CN111368015B - Method and device for compressing map - Google Patents
Method and device for compressing map Download PDFInfo
- Publication number
- CN111368015B CN111368015B CN202010126486.9A CN202010126486A CN111368015B CN 111368015 B CN111368015 B CN 111368015B CN 202010126486 A CN202010126486 A CN 202010126486A CN 111368015 B CN111368015 B CN 111368015B
- Authority
- CN
- China
- Prior art keywords
- map
- frame
- key frame
- points
- map points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Remote Sensing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Embodiments of the present disclosure disclose methods and apparatus for compressing a map. One embodiment of the method comprises: based on the number of the observed map points, performing a key frame determination step on a frame image sequence of the newly added map to obtain a key frame set; merging the determined key frame set into a reference map to obtain a merged map; wherein the key frame determining step comprises: traversing the frame image sequence, determining the frame image with the largest number of map points in the frame image sequence as a key frame, and putting the key frame into a key frame set; determining a frame image of which the observation point is within the compression range of the key frame in the frame image sequence as a contrast frame; and deleting the key frame and the contrast frame from the frame image sequence to obtain an updated frame image sequence in response to that the relative attitude angle of the contrast frame and the key frame is smaller than a first threshold and the number of the common-view map points is larger than a second threshold. The compression of the newly added map is realized, and the adverse effect of the compression process on the map precision is reduced.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of computers, in particular to the technical field of augmented reality, and particularly relates to a method and a device for compressing a map.
Background
In the process of constructing a visual map based on a visual SLAM (synchronous positioning and Mapping) technology, new maps acquired in different environments need to be gradually merged into a reference map so as to update the reference map, so that the capacity of the merged map is increased, and the visual map is not available any more when the capacity exceeds the computing memory of a computer. To avoid this phenomenon, the map needs to be compressed.
The following two methods are generally used in the related art to compress a map: 1) Key frame compression, namely screening key frames based on a simple time sequence relation, and if one key frame is reserved at fixed time intervals, deleting the rest key frames; 2) And (4) map point compression, traversing all key frames, ensuring that each key frame can observe a specified number of map points, and eliminating the rest observed map points which are possibly not reserved due to poor quality or other reasons. Due to the fact that the two compression modes are too simple in the process of thinning the key frames or map points, the accuracy of the compressed map is greatly reduced, and applicable scenes (such as scenes with few map points) are limited.
Disclosure of Invention
The embodiment of the disclosure provides a method and a device for compressing a map.
In a first aspect, an embodiment of the present disclosure provides a method for compressing a map, the method including: based on the number of the observed map points, performing a key frame determination step on a frame image sequence of the newly added map to obtain a key frame set; merging the determined key frame set into a reference map to obtain a merged map; wherein the key frame determining step comprises: traversing the frame image sequence, determining the frame image with the largest number of map points in the frame image sequence as a key frame, and putting the key frame into a key frame set; determining a frame image of which the observation point is within the compression range of the key frame as a comparison frame from the frame image sequence; in response to that the relative attitude angle of the comparison frame and the key frame is smaller than a first threshold value and the number of the common-view map points is larger than a second threshold value, deleting the key frame and the comparison frame from the frame image sequence to obtain an updated frame image sequence; a key frame determination step is performed for the updated sequence of frame images.
In some embodiments, the method further comprises: performing a map point thinning step on the merged map to obtain a compressed merged map; wherein the map point thinning step comprises at least one of the following steps: in response to the observed times of the map points being smaller than a third threshold, deleting the map points with the observed times being smaller than the third threshold; in response to the generation time of the map points being earlier than the preset time, deleting the map points of which the generation time is earlier than the preset time; in response to the reprojection error of the map points being greater than a fourth threshold, deleting map points having a reprojection error greater than the fourth threshold.
In some embodiments, the method further comprises: in response to the number of map points in the compressed merged map being greater than a fifth threshold, performing the following steps to compress the compressed merged map again: constructing a linear programming problem based on the observation times and distribution of map points, wherein the linear programming problem is as follows: acquiring a minimum map point set of a combined map on the premise that a preset number of map points are observed in each frame of image in the combined map; determining a minimum map point set from the remaining map points by solving a linear programming problem; map points that do not exist in the minimum set of map points are deleted.
In some embodiments, the method further comprises: and deleting the descriptors meeting the preset conditions for the descriptors of the rest map points in the merged map.
In some embodiments, deleting the descriptor meeting the preset condition includes: clustering descriptors of the rest map points in the combined map to obtain a clustering result; and deleting the descriptors with the distance from the clustering center to the clustering center larger than the sixth threshold value in each clustering result.
In a second aspect, an embodiment of the present disclosure provides an apparatus for compressing a map, the apparatus including: a key frame compression unit configured to perform a key frame determination step on a frame image sequence of the newly added map based on the number of observed map points to obtain a key frame set; the map merging unit is configured to merge the determined key frame set into a reference map to obtain a merged map; wherein the key frame determining step comprises: traversing the frame image sequence, determining the frame image with the largest number of map points in the frame image sequence as a key frame, and putting the key frame into a key frame set; determining a frame image with an observation point within a compression range of the key frame as a comparison frame from the frame image sequence; in response to that the relative attitude angle of the comparison frame and the key frame is smaller than a first threshold value and the number of the common-view map points is larger than a second threshold value, deleting the key frame and the comparison frame from the frame image sequence to obtain an updated frame image sequence; a key frame determination step is performed for the updated sequence of frame images.
In some embodiments, the apparatus further comprises: a map point compression unit configured to perform a map point thinning step on the merged map to obtain a compressed merged map; wherein the map point thinning step comprises at least one of the following steps: in response to the observed times of the map points being smaller than a third threshold, deleting the map points with the observed times being smaller than the third threshold; in response to the generation time of the map points being earlier than the preset time, deleting the map points of which the generation time is earlier than the preset time; in response to the reprojection error of the map points being greater than a fourth threshold, deleting map points having a reprojection error greater than the fourth threshold.
In some embodiments, the apparatus further comprises: a linear programming unit configured to perform the following steps to compress the compressed merged map again in response to the number of map points in the compressed merged map being greater than a fifth threshold: constructing a linear programming problem based on the observation times and distribution of map points, wherein the linear programming problem is as follows: acquiring a minimum map point set of a combined map on the premise that a preset number of map points are observed in each frame of image in the combined map; determining a minimum map point set from the remaining map points by solving a linear programming problem; map points that do not exist in the minimum set of map points are deleted.
In some embodiments, the apparatus further comprises: and the descriptor compression unit is configured to delete descriptors meeting preset conditions for the descriptors of the map points remaining in the merged map.
In some embodiments, the descriptor compression unit is further configured to: clustering descriptors of the rest map points in the combined map to obtain a clustering result; and deleting the descriptors with the distance from the clustering center to the clustering center larger than the sixth threshold value in each clustering result.
According to the method and the device for compressing the map, the most representative key frame is determined from the frame image sequence of the newly added map based on the spatial characteristics of the map points and the observation points in the frame images, the newly added map is compressed, and meanwhile the adverse effect of the compression process on the map precision is reduced.
Drawings
Other features, objects and advantages of the disclosure will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is an exemplary system architecture diagram in which some embodiments of the present disclosure may be applied;
FIG. 2 is a flow diagram of one embodiment of a method for compressing a map according to the present disclosure;
FIG. 3 is a schematic diagram of one application scenario of a method for compressing a map, in accordance with an embodiment of the present disclosure;
FIG. 4 is a flow diagram of yet another embodiment of a method for compressing a map according to the present disclosure;
FIG. 5 is a schematic block diagram of one embodiment of an apparatus for compressing a map according to the present disclosure;
FIG. 6 is a schematic structural diagram of an electronic device suitable for use in implementing embodiments of the present disclosure.
Detailed Description
The present disclosure is described in further detail below with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 illustrates an exemplary system architecture 100 of a method for compressing a map or an apparatus for compressing a map to which embodiments of the present disclosure may be applied.
As shown in fig. 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may use terminal devices 101, 102, 103 to interact with a server 105 over a network 104 to receive or transmit data. The terminal devices 101, 102, and 103 may send the acquired new map to the server 105, and the server 105 executes the method for compressing the map provided by the embodiment of the present disclosure to obtain a merged map, and send the merged map to the terminal device.
The terminal apparatuses 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices having a display screen and supporting information interaction, including but not limited to smart phones, tablet computers, e-book readers, laptop portable computers, desktop computers, and the like. When the terminal apparatuses 101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented, for example, as multiple software or software modules to provide distributed services, or as a single software or software module. And is not particularly limited herein.
The server 105 may be a server providing various services, such as a background application server providing support for maps displayed on the terminal devices 101, 102, 103. The background application server can update the existing reference map based on the received new map, and send the updated map to the terminal device.
It should be noted that the method for compressing the map provided by the embodiment of the present disclosure may be executed by the terminal devices 101, 102, and 103, or may be executed by the server 105. Accordingly, the means for compressing the map may be provided in the terminal devices 101, 102, 103, or in the server 105. And is not particularly limited herein.
The server may be hardware or software. When the server is hardware, it may be implemented as a distributed server cluster formed by multiple servers, or may be implemented as a single server. When the server is software, it may be implemented as multiple pieces of software or software modules, for example, to provide distributed services, or as a single piece of software or software module. And is not particularly limited herein.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
In the field of augmented reality technology, in order to ensure the accuracy of a visual map constructed based on the visual SLAM technology, new maps acquired under different environments (such as time, lighting conditions, and the like) need to be gradually merged into a reference map, so as to update the reference map. The reference map is a visual map which is processed in advance, and the new map is a new map which is independent of the reference map and is generated by a visual SLAM technology based on newly acquired image data in different environments. The newly added map and the reference map are both composed of frame image sequences, and the frame images comprise acquired images, camera poses and map points, wherein the camera poses comprise position information and posture information of a camera acquiring the images, and the position of the camera is an observation point; map points are also called three-dimensional points, feature points or road signs, and refer to feature points extracted from images based on a visual SLAM technology and used for constructing maps. The key frame is a frame image containing key information in the frame image, for example, a frame image with a large number of observed map points. The capacity of the merged map can be reduced by compressing the frame images in the newly added map and reducing the number of frame images merged into the reference map.
With continued reference to FIG. 2, a flow 200 of one embodiment of a method for compressing a map in accordance with the present disclosure is shown. The method for compressing the map comprises the following steps:
In this embodiment, the key frame determining step includes: traversing the frame image sequence, determining the frame image with the largest number of map points in the frame image sequence as a key frame, and putting the key frame into a key frame set; determining a frame image of which the observation point is within the compression range of the key frame in the frame image sequence as a contrast frame; in response to the fact that the relative attitude angle of the comparison frame and the key frame is smaller than a first threshold value and the number of the common-view map points is larger than a second threshold value, deleting the key frame and the comparison frame from the frame image sequence to obtain an updated frame image sequence; a key frame determination step is performed for the updated sequence of frame images. Wherein the compression range is determined based on the position of the observation point of the key frame and the depth of the map point; the relative pose angle is determined based on the camera poses of the contrast frame and the keyframe.
As an example, the compression range of the key frame may be determined by using the average depth of each map point in the key frame as a compression radius and using the observation point of the key frame as a sphere center to determine a sphere, where an internal space of the sphere is the compression range; the maximum value or the minimum value of the depth of each map point in the key frame can be used as a compression radius, and the observation point of the key frame is used as a sphere center to determine a compression range. As an example, the relative attitude angle may be determined as follows: determining a roll angle, a course angle and a pitch angle of the corresponding observation point from camera poses of the comparison frame and the key frame, respectively determining differences between the roll angle, the course angle and the pitch angle of the comparison frame and the key frame, and determining the maximum value of the differences as a relative pose angle of the comparison frame and the key frame; a weighted sum of the differences of the three angles may also be determined as the relative pose angle of the contrast frame and the keyframe. By compressing the frame image sequence of the newly added map through the step, the key frames containing a large number of map points can be determined, and the similarity degree between the key frames is low, so that the newly added map is compressed while key information is not lost.
The key frame determining step is illustrated below by referring to a scene, and the executing subject determines a frame image i with the largest number of map points observed in a frame image sequence of the newly added map as a key frame, calculates a compression radius r according to an average depth of the map points in the frame image i, then determines a sphere range by taking an observation point of the frame image i as a sphere center and r as a radius, and determines other frame images (for example, frame images a, m, and g) with the observation points in the sphere as comparison frames. Thereafter, the relative pose angles of the contrast frames (frame images a, m, g) and the key frame (frame image i) and the common view points are calculated, respectively, as examples, the relative pose angles of the frame images a, m, g with respect to the key frame are 1 °, 10 °, and 30 °, respectively, and the number of the common view points is 100, 200, and 300, respectively. In this embodiment, the preset first threshold is 20 °, the second threshold is 150, and it is known through comparison that the relative attitude angle of the frame image m with respect to the key frame is smaller than the first threshold and the number of the common view map points is greater than the second threshold, which indicates that the degree of similarity between the frame image m and the key frame is high, then the frame images i and m are deleted from the frame image sequence, and the frame image i is added to the key frame set, so as to obtain an updated frame image sequence. And then, the steps are executed according to the updated frame image sequence until the frame image sequence is cleared, and a key frame set is obtained. It should be noted that the newly added map in this example is a map constructed based on the visual SLAM technology, and therefore the frame image of the newly added map includes feature information of map points and a camera pose, and it belongs to common knowledge in the art to determine the number of map points, the depth of the map points, and the number of co-view map points according to the feature information of the map points, and determine observation points and attitude angles of the frame image according to the camera pose, and details thereof are not repeated here.
In this embodiment, an execution subject (for example, a server shown in fig. 1) of the method for compressing a map may receive a new map from a terminal with which a user performs map construction and compression processing through a wired connection manner or a wireless connection manner. It is noted that the wireless connection means may include, but is not limited to, a 3G/4G connection, a WiFi connection, a bluetooth connection, a WiMAX connection, a Zigbee connection, a UWB (ultra wideband) connection, and other wireless connection means now known or developed in the future.
In a specific example, an operator may acquire image data of a target area through a customized monocular or binocular camera, and then construct a map based on the image data through a visual SLAM technique, i.e., obtain a newly added map. And then sending the new map to an execution subject (such as a server shown in fig. 1) through a network, executing the key frame determination step by the execution subject, removing the frame images with poor representativeness from the frame image sequence in the new map, obtaining a key frame set with a small number, and compressing the frame images.
In this embodiment, based on the key frame set obtained in step 201, the execution subject (for example, the server shown in fig. 1) merges the key frame set into the reference map to obtain a merged map, that is, the update of the base map is completed. Since the newly added map is compressed in step 201, the compression of the volume of the merged map is indirectly realized. As an example, the execution subject may implement the combination of the two methods by using a method based on location prior, or may select a method based on loop detection, and both methods belong to the prior art in the visual SLAM technology and are not described herein again.
In some optional implementation manners of the foregoing embodiment, after the merged map is obtained in step 202, the map points in the merged map may be compressed through the following map point sparsification step, which includes compressing the map points in the original frame image in the reference map, and compressing the map points in the key frame determined in step 201, so as to further compress the capacity of the merged map.
The map point thinning step comprises at least one of the following steps: in response to the observed times of the map points being smaller than a third threshold, deleting the map points with the observed times being smaller than the third threshold; in response to the generation time of the map points being earlier than the preset time, deleting the map points of which the generation time is earlier than the preset time; in response to the reprojection error of the map points being greater than a fourth threshold, deleting map points having a reprojection error greater than the fourth threshold.
The number of times of observing the map point refers to the number of frame images of the map point observed in the merged map, and is positively correlated with the importance degree of the map point in the map. The generation time refers to the time when the map point is observed for the first time, is used for representing the aging of the map point and is inversely related to the importance degree of the map point in the map. The re-projection error is an error obtained by comparing the projection position of the map point observed by the value with the position obtained by projecting the map point according to the current estimated camera pose, is used for representing the map construction precision and is negatively correlated with the importance degree of the map point in the map. Handle
By the map point thinning step, map points with lower importance in the combined map can be deleted, so that compression of the combined map is realized.
In another optional implementation manner of the foregoing embodiment, after the map point thinning step is performed, it is detected that the number of map points still does not meet the requirement, for example, the number of map points is greater than a preset fifth threshold, and then the overall number of map points may be further compressed through the following steps. The method comprises the following specific steps: constructing a linear programming problem based on the observation times and distribution of map points, wherein the linear programming problem is as follows: acquiring a minimum map point set of a combined map on the premise that a preset number of map points are observed in each frame of image in the combined map; determining a minimum map point set from the remaining map points by solving a linear programming problem; map points that do not exist in the minimum set of map points are deleted.
In a specific example of this implementation, the constructed linear programming problem is to determine a minimum map point set on the premise of ensuring that at least k map points are observed in each frame of image, so that the number of map points is not greater than a fifth threshold, and is characterized by the following equation:
subject to Ax+ζ≥b 1
x∈{0，1} N
wherein x is a binary vector used for representing whether the map point is reserved or not; q is the score vector of the map point, where the number of times the map point was observed is used as a scoring criterion; a is binary visibility matrix (binary visibility matrix), and the element a in the matrix A ij Indicating that the ith map point is observed in the ith frame image; b is the number of map points observed least by each key frame, and in combination with the example, the value of b may be set to k; n is the total number of map points desired to be retained, and the value of n may be set to a value less than or equal to a fifth threshold value, in connection with an example.
Then, the above equation is solved by an ILP (integer linear programming) method, and a minimum map point set meeting the requirement can be obtained. And deleting the map points which do not exist in the minimum map point set, so that the number of the map points in the combined map is not more than a fifth threshold value, and the control of the number of the map points of the combined map in the global scope is finished.
With continued reference to fig. 3, fig. 3 is a schematic diagram of an application scenario of the method for compressing a map according to the present embodiment. In the application scenario of fig. 3, a user may perform data interaction with an execution subject through a mobile terminal, for example, obtain map information from the execution subject through a navigation application in a mobile phone. An execution subject (for example, the server 105 shown in fig. 1) determines a key frame set from a frame image sequence of a new map, so as to implement compression of the new map, and then merges the key frame set into a reference map that has already been constructed, so as to update the reference map, thereby obtaining a merged map, and the accuracy and timeliness of the merged map can be ensured. And finally, the combined map is sent to a mobile terminal of the user through a network, so that the user can conveniently navigate or position according to the updated map information.
According to the method provided by the embodiment of the disclosure, the most representative key frame is determined from the frame image sequence of the newly added map based on the number and the spatial characteristics of the map points in the frame image, so that the newly added map is compressed, and meanwhile, the adverse effect of the compression process on the map precision is reduced.
With further reference to fig. 4, a flow 400 of yet another embodiment of a method for compressing a map is shown. The flow 400 of the method for compressing a map comprises the following steps:
And step 402, merging the determined key frame set into a reference map to obtain a merged map. This step corresponds to the step 202, and is not described herein again.
And 403, executing a map point thinning step on the combined map to obtain a compressed combined map. Details of this step have already been discussed in the alternative implementation of the foregoing embodiment, and are not described here again.
In step 408, for the descriptors of the remaining map points in the merged map, the descriptors meeting the preset condition are deleted. For example, when each map point contains a plurality of identical descriptors, only one of the descriptors may be retained, and the remaining repeated descriptors may be deleted; for another example, when each map point includes a plurality of similar descriptors, only some of the descriptors are retained, and the remaining similar descriptors are deleted. In some optional implementation manners of this embodiment, the descriptor may be compressed by a clustering algorithm, and the specific steps include: clustering descriptors of the rest map points in the combined map to obtain a clustering result; and deleting the descriptors with the distance from the clustering center to the clustering center larger than the sixth threshold value in each clustering result. By thinning the descriptors, the capacity of the merged map can be further reduced.
As can be seen from fig. 4, compared with the embodiment corresponding to fig. 2, the flow 400 of the method for compressing a map in this embodiment represents further compression of the merged map through the following three dimensions, specifically including: map points with low importance degree are deleted through a map point thinning step, then the total number of map points of the combined map is controlled by solving a linear programming problem, and finally descriptors corresponding to the rest map points are compressed. Map points with higher importance degree are reserved in the compression process, so that the adverse effect of the compression process on the map precision is reduced.
With further reference to fig. 5, as an implementation of the methods shown in the above-mentioned figures, the present disclosure provides an embodiment of an apparatus for compressing a map, which corresponds to the method embodiment shown in fig. 2, and which is particularly applicable in various electronic devices.
As shown in fig. 5, the apparatus 500 for compressing a map of the present embodiment includes: a key frame compression unit 501 configured to perform a key frame determination step on a frame image sequence of the newly added map based on the number of observed map points to obtain a key frame set; a map merging unit 502 configured to merge the determined key frame set into a reference map, so as to obtain a merged map; wherein the key frame determining step comprises: traversing the frame image sequence, determining the frame image with the largest number of map points in the frame image sequence as a key frame, and putting the key frame into a key frame set; determining a frame image with an observation point within a compression range of the key frame as a comparison frame from the frame image sequence; in response to that the relative attitude angle of the comparison frame and the key frame is smaller than a first threshold value and the number of the common-view map points is larger than a second threshold value, deleting the key frame and the comparison frame from the frame image sequence to obtain an updated frame image sequence; a key frame determination step is performed for the updated sequence of frame images.
In this embodiment, the apparatus 500 further includes: the map point compression unit is configured to execute a map point sparsifying step on the combined map to obtain a compressed combined map; wherein the map point thinning step comprises at least one of the following steps: in response to the observed times of the map points being smaller than a third threshold, deleting the map points with the observed times being smaller than the third threshold; in response to the generation time of the map points being earlier than the preset time, deleting the map points of which the generation time is earlier than the preset time; in response to the reprojection error of the map points being greater than a fourth threshold, deleting map points having a reprojection error greater than the fourth threshold.
In this embodiment, the apparatus 500 further includes: a linear programming unit configured to perform the following steps to compress the compressed merged map again in response to the number of map points in the compressed merged map being greater than a fifth threshold: constructing a linear programming problem based on the observation times and distribution of map points, wherein the linear programming problem is as follows: acquiring a minimum map point set of a combined map on the premise that a preset number of map points are observed in each frame of image in the combined map; determining a minimum map point set from the remaining map points by solving a linear programming problem; map points that do not exist in the minimum set of map points are deleted.
In this embodiment, the apparatus 500 further includes: and the descriptor compression unit is configured to delete descriptors meeting preset conditions for the descriptors of the map points remaining in the merged map.
In this embodiment, the descriptor compression unit is further configured to: clustering descriptors of the rest map points in the combined map to obtain a clustering result; and deleting the descriptors with the distance from the clustering center to the clustering center larger than the sixth threshold value in each clustering result.
Referring now to fig. 6, a schematic diagram of an electronic device (e.g., the server or terminal device of fig. 1) 600 suitable for use in implementing embodiments of the present disclosure is shown. The terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), and the like, and a fixed terminal such as a digital TV, a desktop computer, and the like. The terminal device/server shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 6, the electronic device 600 may include a processing means (e.g., central processing unit, graphics processor, etc.) 601 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices the devices communicate wirelessly or by wire to exchange data. While fig. 6 illustrates an electronic device 600 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 6 may represent one device or may represent multiple devices as desired.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 609, or may be installed from the storage means 608, or may be installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of embodiments of the present disclosure. It should be noted that the computer readable medium of the embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In embodiments of the present disclosure, however, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: based on the number of the observed map points, performing a key frame determination step on a frame image sequence of the newly added map to obtain a key frame set; merging the determined key frame set into a reference map to obtain a merged map; wherein the key frame determining step comprises: traversing the frame image sequence, determining the frame image with the largest number of map points in the frame image sequence as a key frame, and putting the key frame into a key frame set; determining a frame image of which the observation point is within the compression range of the key frame in the frame image sequence as a contrast frame; in response to that the relative attitude angle of the comparison frame and the key frame is smaller than a first threshold value and the number of the common-view map points is larger than a second threshold value, deleting the key frame and the comparison frame from the frame image sequence to obtain an updated frame image sequence; a key frame determination step is performed for the updated sequence of frame images.
Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package partially on the user's computer, partially on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes a key frame compression unit and a map merging unit. The names of these units do not in some cases constitute a limitation on the units themselves, for example, the key frame compression unit may also be described as "a unit that performs a key frame determination step on the frame image sequence of the newly added map based on the number of observed map points to obtain a key frame set".
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept as defined above. For example, the above features and (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.
Claims (12)
1. A method for compressing a map, comprising:
based on the number of the observed map points, performing a key frame determination step on a frame image sequence of the newly added map to obtain a key frame set;
merging the determined key frame set into a reference map to obtain a merged map;
wherein the key frame determining step comprises:
traversing the frame image sequence, determining the frame image with the largest number of map points in the frame image sequence as a key frame, and putting the key frame into a key frame set;
determining a frame image with an observation point within the compression range of the key frame as a contrast frame from the frame image sequence;
in response to that the relative attitude angle of the comparison frame and the key frame is smaller than a first threshold value and the number of common-view map points is larger than a second threshold value, deleting the key frame and the comparison frame from the frame image sequence to obtain an updated frame image sequence;
performing the key frame determination step on the updated sequence of frame images.
2. The method of claim 1, further comprising:
performing a map point thinning step on the merged map to obtain a compressed merged map;
wherein the map point thinning step comprises at least one of the following steps:
in response to the observed times of the map points being smaller than a third threshold, deleting the map points with the observed times being smaller than the third threshold;
in response to the generation time of the map points being earlier than the preset time, deleting the map points of which the generation time is earlier than the preset time;
in response to the reprojection error of the map points being greater than a fourth threshold, deleting map points having a reprojection error greater than the fourth threshold.
3. The method of claim 2, further comprising:
in response to the number of map points in the compressed merged map being greater than a fifth threshold, performing the following steps to compress the compressed merged map again:
constructing a linear programming problem based on the observation times and distribution of map points, wherein the linear programming problem is as follows: acquiring a minimum map point set of the combined map on the premise that a preset number of map points are observed in each frame of image in the combined map;
determining a minimum map point set from the remaining map points by solving the linear programming problem;
deleting map points that do not exist in the minimum set of map points.
4. The method of one of claims 1 to 3, further comprising:
and deleting the descriptors meeting the preset conditions for the descriptors of the rest map points in the merged map.
5. The method of claim 4, wherein deleting descriptors that meet a preset condition comprises:
clustering descriptors of the rest map points in the combined map to obtain a clustering result;
and deleting the descriptors with the distance from the clustering center to the clustering center larger than a sixth threshold value in each clustering result.
6. An apparatus for compressing a map, comprising:
a key frame compression unit configured to perform a key frame determination step on a frame image sequence of the newly added map based on the number of observed map points to obtain a key frame set;
the map merging unit is configured to merge the determined key frame set into a reference map to obtain a merged map;
wherein the key frame determining step comprises:
traversing the frame image sequence, determining the frame image with the largest number of map points in the frame image sequence as a key frame, and putting the key frame into a key frame set;
determining a frame image with an observation point within the compression range of the key frame as a contrast frame from the frame image sequence;
in response to that the relative attitude angle of the comparison frame and the key frame is smaller than a first threshold value and the number of common-view map points is larger than a second threshold value, deleting the key frame and the comparison frame from the frame image sequence to obtain an updated frame image sequence;
performing the key frame determination step on the updated sequence of frame images.
7. The apparatus of claim 6, further comprising:
the map point compression unit is configured to execute a map point sparsifying step on the combined map to obtain a compressed combined map;
wherein the map point thinning step comprises at least one of the following steps:
in response to the observed times of the map points being smaller than a third threshold, deleting the map points with the observed times being smaller than the third threshold;
in response to the generation time of the map points being earlier than the preset time, deleting the map points of which the generation time is earlier than the preset time;
in response to the reprojection error of the map points being greater than a fourth threshold, deleting map points having a reprojection error greater than the fourth threshold.
8. The apparatus of claim 7, further comprising:
a linear programming unit configured to perform the following steps to compress the compressed merged map again in response to the number of map points in the compressed merged map being greater than a fifth threshold:
constructing a linear programming problem based on the observation times and distribution of map points, wherein the linear programming problem is as follows: acquiring a minimum map point set of the merged map on the premise that each frame of image in the merged map observes a preset number of map points;
determining a minimum map point set from the rest map points by solving the linear programming problem;
deleting map points that do not exist in the minimum set of map points.
9. The apparatus of one of claims 6 to 8, further comprising:
and the descriptor compression unit is configured to delete descriptors meeting preset conditions for the descriptors of the map points remaining in the merged map.
10. The apparatus of claim 9, wherein the descriptor compression unit is further configured to:
the descriptors of the remaining map points in the merged map are clustered, obtaining a clustering result;
and deleting the descriptors with the distance from the clustering center to the clustering center larger than a sixth threshold value in each clustering result.
11. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-5.
12. A computer-readable medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010126486.9A CN111368015B (en) | 2020-02-28 | 2020-02-28 | Method and device for compressing map |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010126486.9A CN111368015B (en) | 2020-02-28 | 2020-02-28 | Method and device for compressing map |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111368015A CN111368015A (en) | 2020-07-03 |
| CN111368015B true CN111368015B (en) | 2023-04-07 |
Family
ID=71210076
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010126486.9A Active CN111368015B (en) | 2020-02-28 | 2020-02-28 | Method and device for compressing map |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111368015B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107748569A (en) * | 2017-09-04 | 2018-03-02 | 中国兵器工业计算机应用技术研究所 | Motion control method, device and UAS for unmanned plane |
| CN107862720A (en) * | 2017-11-24 | 2018-03-30 | 北京华捷艾米科技有限公司 | Pose optimization method and pose optimization system based on the fusion of more maps |
| CN107885871A (en) * | 2017-11-24 | 2018-04-06 | 南京华捷艾米软件科技有限公司 | Synchronous superposition method, system, interactive system based on cloud computing |
| KR20180094463A (en) * | 2017-02-15 | 2018-08-23 | 한양대학교 산학협력단 | Method for saving and loading of slam map |
| CN108682027A (en) * | 2018-05-11 | 2018-10-19 | 北京华捷艾米科技有限公司 | VSLAM realization method and systems based on point, line Fusion Features |
| CN109658445A (en) * | 2018-12-14 | 2019-04-19 | 北京旷视科技有限公司 | Network training method, increment build drawing method, localization method, device and equipment |
| CN109816769A (en) * | 2017-11-21 | 2019-05-28 | 深圳市优必选科技有限公司 | Scene based on depth camera ground drawing generating method, device and equipment |
| CN110310326A (en) * | 2019-06-28 | 2019-10-08 | 北京百度网讯科技有限公司 | A kind of pose data processing method, device, terminal and computer readable storage medium |
| CN110501017A (en) * | 2019-08-12 | 2019-11-26 | 华南理工大学 | A kind of Mobile Robotics Navigation based on ORB_SLAM2 ground drawing generating method |
| CN110726406A (en) * | 2019-06-03 | 2020-01-24 | 北京建筑大学 | Improved nonlinear optimization monocular inertial navigation SLAM method |
| CN110782494A (en) * | 2019-10-16 | 2020-02-11 | 北京工业大学 | Visual SLAM method based on point-line fusion |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018048353A1 (en) * | 2016-09-09 | 2018-03-15 | Nanyang Technological University | Simultaneous localization and mapping methods and apparatus |
-
2020
- 2020-02-28 CN CN202010126486.9A patent/CN111368015B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20180094463A (en) * | 2017-02-15 | 2018-08-23 | 한양대학교 산학협력단 | Method for saving and loading of slam map |
| CN107748569A (en) * | 2017-09-04 | 2018-03-02 | 中国兵器工业计算机应用技术研究所 | Motion control method, device and UAS for unmanned plane |
| CN109816769A (en) * | 2017-11-21 | 2019-05-28 | 深圳市优必选科技有限公司 | Scene based on depth camera ground drawing generating method, device and equipment |
| CN107862720A (en) * | 2017-11-24 | 2018-03-30 | 北京华捷艾米科技有限公司 | Pose optimization method and pose optimization system based on the fusion of more maps |
| CN107885871A (en) * | 2017-11-24 | 2018-04-06 | 南京华捷艾米软件科技有限公司 | Synchronous superposition method, system, interactive system based on cloud computing |
| CN108682027A (en) * | 2018-05-11 | 2018-10-19 | 北京华捷艾米科技有限公司 | VSLAM realization method and systems based on point, line Fusion Features |
| CN109658445A (en) * | 2018-12-14 | 2019-04-19 | 北京旷视科技有限公司 | Network training method, increment build drawing method, localization method, device and equipment |
| CN110726406A (en) * | 2019-06-03 | 2020-01-24 | 北京建筑大学 | Improved nonlinear optimization monocular inertial navigation SLAM method |
| CN110310326A (en) * | 2019-06-28 | 2019-10-08 | 北京百度网讯科技有限公司 | A kind of pose data processing method, device, terminal and computer readable storage medium |
| CN110501017A (en) * | 2019-08-12 | 2019-11-26 | 华南理工大学 | A kind of Mobile Robotics Navigation based on ORB_SLAM2 ground drawing generating method |
| CN110782494A (en) * | 2019-10-16 | 2020-02-11 | 北京工业大学 | Visual SLAM method based on point-line fusion |
Non-Patent Citations (2)
| Title |
|---|
| Jintae Kim ; et al.A Robust Client-Server Architecture for Map Information Processing and Transmission for Distributed Visual SLAM.《2019 7th International Conference on Robot Intelligence Technology and Applications (RiTA)》.2019, * |
| 基于深度学习与单目视觉SLAM的三维语义场景建模研究;敖焕轩;《中国优秀硕士学位论文辑》;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111368015A (en) | 2020-07-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111292420B (en) | Method and device for constructing map | |
| CN110413812B (en) | Neural network model training method and device, electronic equipment and storage medium | |
| US20220277481A1 (en) | Panoramic video processing method and apparatus, and storage medium | |
| CN111399956A (en) | Content display method and device applied to display equipment and electronic equipment | |
| CN111784712B (en) | Image processing method, device, equipment and computer readable medium | |
| CN113253319B (en) | Road network extraction and track deviation correction method and system based on vehicle GPS | |
| WO2022171036A1 (en) | Video target tracking method, video target tracking apparatus, storage medium, and electronic device | |
| CN112257582A (en) | Foot posture determination method, device, equipment and computer readable medium | |
| CN110188782B (en) | Image similarity determining method and device, electronic equipment and readable storage medium | |
| CN111652675A (en) | Display method and device and electronic equipment | |
| CN111126159A (en) | Method, apparatus, electronic device, and medium for tracking pedestrian in real time | |
| CN111915532B (en) | Image tracking method and device, electronic equipment and computer readable medium | |
| CN111626990B (en) | Target detection frame processing method and device and electronic equipment | |
| CN111461965B (en) | Picture processing method and device, electronic equipment and computer readable medium | |
| CN111368015B (en) | Method and device for compressing map | |
| CN115086541B (en) | Shooting position determining method, device, equipment and medium | |
| CN115408609A (en) | Parking route recommendation method and device, electronic equipment and computer readable medium | |
| CN112241744B (en) | Image color migration method, device, equipment and computer readable medium | |
| CN111680754B (en) | Image classification method, device, electronic equipment and computer readable storage medium | |
| CN112037280A (en) | Object distance measuring method and device | |
| CN112598732B (en) | Target equipment positioning method, map construction method and device, medium and equipment | |
| CN112884787B (en) | Image clipping method and device, readable medium and electronic equipment | |
| CN118038193B (en) | Visual display method and device for underground cable, electronic equipment and computer medium | |
| CN112668474B (en) | Plane generation method and device, storage medium and electronic equipment | |
| CN114049417B (en) | Virtual character image generation method and device, readable medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |