CN110440802A - P2P air navigation aid, device and the electronic equipment of view-based access control model SLAM - Google Patents
P2P air navigation aid, device and the electronic equipment of view-based access control model SLAM Download PDFInfo
- Publication number
- CN110440802A CN110440802A CN201910618614.9A CN201910618614A CN110440802A CN 110440802 A CN110440802 A CN 110440802A CN 201910618614 A CN201910618614 A CN 201910618614A CN 110440802 A CN110440802 A CN 110440802A
- Authority
- CN
- China
- Prior art keywords
- video
- follower
- access control
- terminal
- control model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/28—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network with correlation of data from several navigational instruments
- G01C21/30—Map- or contour-matching
- G01C21/32—Structuring or formatting of map data
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Automation & Control Theory (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the present invention provides P2P air navigation aid, device and the electronic equipment of a kind of view-based access control model SLAM, wherein this method comprises: obtaining the first video from pilotage people's terminal, first video is the video of the environment on the way in the path shot when pilotage people advances along its path；It based on first video, using monocular vision SLAM algorithm, calculates the camera pose track of the pilotage people and generates three-dimensional map point, and be based on camera pose track and three-dimensional map point, generate the motion profile map of pilotage people；The second video is obtained from follower's terminal in real time, and it is based on second video and the motion profile map, follower is guided to advance in the region comprising the path by follower's terminal, second video is the video for the ambient enviroment that the follower shoots during traveling.The embodiment of the present invention can effectively overcome the dependence to infrastructure, expand the range of application to a greater extent.
Description
Technical field
The present invention relates to location and navigation technology fields, more particularly, to the navigation side P2P of view-based access control model SLAM a kind of
Method, device and electronic equipment.
Background technique
With the substantial increase of the medium-and-large-sized indoor environment in city, indoor pedestrian navigation demand is also being increased rapidly.It leads interior
Boat technology helps to cope with emergency case, while bringing huge convenience to the production and living of people.Currently, indoor navigation technology
There are two types of main modes: navigation mode and user the leading to user (Peer-to-Peer, P2P) based on indoor positioning technologies
Model plane formula.
Under the navigation mode based on indoor positioning technologies, user is first located in indoor global map by system, so
Navigation instruction is provided according to the destination of user afterwards.With the most widely used of Wi-Fi indoor positioning technologies in this mode,
Wi-Fi indoor positioning mostly uses the location fingerprint algorithm based on signal strength, this is needed through artificial on-site land survey (site-
Survey) come construct indoor signal fingerprint map for positioning.However, manpower needed for site-survey and time cost at
Develop and move towards the obstruction of application for Wi-Fi signal fingerprint location technology.Although proposing that a series of crowdsourcing method is come in recent years
The burden of site-survey is reduced, but there are still the defects that need to rely on the infrastructure that Wi-Fi etc. is arranged in advance for this method.
In P2P navigation mode, it is contemplated that the target of navigation is to direct the user to destination to go, and researcher proposes, can
With the information of smart phone record path (such as the collected difference Wi-Fi of institute on path when allowing a user to walk indoors
The step number and turn to situation that fingerprint, pedestrian directly walk), routing information is then shared to by other users by network.Another use
Same path is walked at family if necessary, and the routing information that need to be only shared with the mobile phone tracking of oneself reappears routing information,
Reach same destination.In such navigation mode, the user of record path information is known as pilotage people, tracks path
The user of information is known as follower.Although P2P navigation gets rid of the dependence to global map, existing P2P navigation still needs to
Indoor infrastructure (such as Wi-Fi, bluetooth) is relied on, this is limited its practical application.
Summary of the invention
In order to overcome the above problem or at least be partially solved the above problem, the embodiment of the present invention provides a kind of based on view
Feel that P2P air navigation aid, device and the electronic equipment of SLAM expand to a greater extent effectively to overcome the dependence to infrastructure
The range applied greatly.
In a first aspect, the embodiment of the present invention provides the P2P air navigation aid of view-based access control model SLAM a kind of, comprising:
From pilotage people's terminal obtain the first video, first video be pilotage people along its path advance when shoot described in
The video of the environment on the way in path；
The camera pose track of the pilotage people is calculated simultaneously using monocular vision SLAM algorithm based on first video
Three-dimensional map point is generated, and is based on camera pose track and three-dimensional map point, with generating the motion profile of pilotage people
Figure；
The second video is obtained from follower's terminal in real time, and is based on second video and the motion profile map, is led to
Crossing follower's terminal guides follower to advance in the region comprising the path, and second video is the follower
The video of the ambient enviroment shot during traveling.
Second aspect, the embodiment of the present invention provide the P2P navigation device of view-based access control model SLAM a kind of, comprising:
Module is obtained, for obtaining the first video from pilotage people's terminal, first video is pilotage people along its path row
Into when the video of the environment on the way in the path that shoots；
Calculating and generation module, for calculating the navigator using monocular vision SLAM algorithm based on first video
The camera pose track of person simultaneously generates three-dimensional map point, and is based on camera pose track and three-dimensional map point, generates
The motion profile map of pilotage people；
Navigation instruction module for obtaining the second video from follower's terminal in real time, and is based on second video and institute
Motion profile map is stated, guides follower to advance in the region comprising the path by follower's terminal, described the
Two videos are the video for the ambient enviroment that the follower shoots during traveling.
The third aspect, the embodiment of the present invention provide a kind of electronic equipment, including memory, processor and are stored in described deposit
On reservoir and the computer program that can run on the processor, when the processor executes the computer program, realize
The step of P2P air navigation aid of view-based access control model SLAM described in first aspect as above.
Fourth aspect, the embodiment of the present invention provide a kind of non-transient computer readable storage medium, are stored thereon with calculating
Machine instruction, when the computer instruction is computer-executed, realizes that the P2P of view-based access control model SLAM described in first aspect as above is led
The step of boat method.
P2P air navigation aid, device and the electronic equipment of view-based access control model SLAM provided in an embodiment of the present invention, does not need basis
Facility does not need to provide priori map, the track of pilotage people need to be only constructed by view-based access control model SLAM technology, and utilize and follow
Follower is located in target trajectory and realizes the navigation of user to user by the video of person's shooting, therefore can effectively overcome pair
The dependence of infrastructure expands the range of application to a greater extent.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is this hair
Bright some embodiments for those of ordinary skill in the art without creative efforts, can be with root
Other attached drawings are obtained according to these attached drawings.
Fig. 1 is the flow diagram of the P2P air navigation aid of view-based access control model SLAM provided in an embodiment of the present invention；
Fig. 2 is according to P2P pedestrian navigation system in the P2P air navigation aid of view-based access control model SLAM provided in an embodiment of the present invention
The structural schematic diagram of framework；
Fig. 3 is according to navigation instruction in the P2P air navigation aid of view-based access control model SLAM provided in an embodiment of the present invention and to calculate
The correspondence diagram of angle；
Fig. 4 is the structural schematic diagram of the P2P navigation device of view-based access control model SLAM provided in an embodiment of the present invention；
Fig. 5 is the entity structure schematic diagram of electronic equipment provided in an embodiment of the present invention.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the embodiment of the present invention, instead of all the embodiments.Based on the embodiment in the embodiment of the present invention, ability
Domain those of ordinary skill every other embodiment obtained without making creative work, belongs to the present invention
The range of embodiment protection.
The embodiment of the present invention causes to answer for needing to rely on indoor infrastructure when carrying out P2P navigation in the prior art
With limited problem, does not need infrastructure, do not need to provide priori map, need to only be constructed by view-based access control model SLAM technology
The track of pilotage people, and follower is located in target trajectory to realize user to user's using the video of follower's shooting
Navigation, therefore can effectively overcome the dependence to infrastructure, expand the range of application to a greater extent.It below will be especially by
Multiple embodiments carry out expansion explanation and introduction to the embodiment of the present invention.
Fig. 1 is the flow diagram of the P2P air navigation aid of view-based access control model SLAM provided in an embodiment of the present invention, such as Fig. 1 institute
Show, this method comprises:
S101 obtains the first video from pilotage people's terminal, and the first video is the road shot when pilotage people advances along its path
The video of the environment on the way of diameter.
It is to be understood that pilotage people can reach from current location along certain path in the case where determining destination
To the destination, which is the path of pilotage people.During advancing along the path, pilotage people can use its mobile phone
The camera captured in real-time having is the first video by way of the video on ground on the way environment.This may include neck in environment on the way
Any visible dynamic object and stationary body in boat person's ambient enviroment.The embodiment of the present invention can be by taking in pilotage people's mobile phone
Pilotage people's terminal of band obtains the first video of its shooting from the mobile phone of pilotage people.It is understood that the acquisition process can
To be the process actively obtained from pilotage people's terminal, it is also possible to the process that pilotage people's terminal actively issues.
S102 is calculated the camera pose track of pilotage people and generated based on the first video using monocular vision SLAM algorithm
Three-dimensional map point, and it is based on camera pose track and three-dimensional map point, generate the motion profile map of pilotage people.
It is to be understood that on the basis of obtaining the first video, with monocular vision SLAM algorithm according to the first video meter
Camera pose track and three-dimensional map point are calculated, is initialized as the information in the path, namely to the first video by monocular SLAM
After calculating, the track of camera is tracked by visual odometry, and generate three-dimensional map point according to calculating, construct the fortune of pilotage people
Dynamic rail slash figure.
S103, in real time from follower's terminal obtain the second video, and be based on the second video and motion profile map, by with
Follower is guided to advance in the region comprising path with person's terminal, the second video is the week that follower shoots during traveling
The video in collarette border.
After constructing the motion profile map of pilotage people, follower can be along the same motion profile of pilotage people to chase after
Identical visit is asked to experience.The embodiment of the present invention can guide follower according to neck by follower's terminal that follower holds
The running track of boat person is advanced.Specifically, follower during traveling, also can shoot ambient enviroment using its mobile phone camera
Video, as the second video.
The embodiment of the present invention passes through the second video that follower's terminal obtains user's shooting first, same according to second video
When combine the first video, the positioning to follower's current time can be realized.Later can according to follower when prelocalization and
The actual motion profile map of pilotage people determines the direction of travel of follower's next step, and passes through follower's terminal for the traveling
Direction is supplied to follower, so that follower advances within a certain area along the operating path of pilotage people.
The P2P air navigation aid of view-based access control model SLAM provided in an embodiment of the present invention does not need infrastructure, does not need to provide
Priori map need to only construct the track of pilotage people by view-based access control model SLAM technology, and will using the video of follower's shooting
Follower is located in target trajectory realizes the navigation of user to user, thus can effectively overcome to infrastructure according to
Rely, expands the range of application to a greater extent.
It is appreciated that current vision SLAM is often to study in rigid environment, rigid environment refers in environment
Object will not all move or deformation occurs, so the appearance of environment will not change.And in practical application, it needs to pedestrian
The environment to navigate is often non-rigid environment, for example the walking about of someone, commodity change changing for even light in market
Change etc. can all influence the appearance of scene.These non-rigid environmental factors will affect SLAM system to the calculating of camera pose and three
The building for tieing up point map, so that the applicability of vision SLAM can be weakened significantly.Therefore, on the basis of the various embodiments described above, In
Before the step of calculating the camera pose track of pilotage people and generating three-dimensional map point, the method for the embodiment of the present invention can also be wrapped
It includes:
The dynamic prospect in the first video is extracted and removed, to retain the static environment factor in the first video, and is based on
Static environment factor extracts static nature point；Correspondingly, being based on static nature point, using monocular vision SLAM algorithm, phase is calculated
Seat in the plane appearance track simultaneously generates three-dimensional map point.
That is, the embodiment of the present invention is in practical application, extracting and removing acquisition in order to overcome the non-rigid of environment
The first video in dynamic prospect, as in environment pedestrian and other often by mobile object, it is static to leave behind those
Environmental factor, and characteristic point is therefrom extracted, for constructing track and three-dimensional map point.
It is understood that can also extract and remove first when obtaining the second video to relocate to follower
Obtain the second video in dynamic prospect, leave behind static environment factor, and therefrom extract characteristic point with from the first video
The characteristic point of extraction carries out matching primitives, to realize reorientation.
It is wherein optional according to the above embodiments, it extracts and is specifically wrapped the step of removing the dynamic prospect in the first video
It includes: using MaskR-CNN algorithm, extracting the connected domain of dynamic object in the first video, and utilize the mirror surface and light of YOLO system
Sliding Surface testing tool, extracts the relevant connected domain of mirror surface in the first video；Extract the connected domain and mirror surface of all dynamic objects
Characteristic point in relevant connected domain constitutes dynamic prospect and rejects.
The embodiment of the present invention uses MaskR-CNN algorithm first, extracts in the first video (being also possible to the second video)
The connected domain of dynamic object, while using the detection algorithm of the mirror surface and smooth surface proposed in YOLO system, extract first
The relevant connected domain of mirror surface in video (being also possible to the second video).Then the embodiment of the present invention is by the feature in these connected domains
Point is all extracted and is weeded out, then remaining characteristic point is extracted from rigid environment, is also just applicable in vision
SLAM.
The embodiment of the present invention filters non-rigid environmental factor by semantic segmentation, therefore can adapt to the room of high dynamic
Interior navigational environment.
It is wherein optional, it calculates the camera pose track of pilotage people and generates three-dimensional map point to the movement for generating pilotage people
The step of track map, is specifically handled in the server, and is utilizing MaskR-CNN algorithm, extracts dynamic in the first video
In the step of connected domain of object, 1 frame MaskR-CNN is used for the frame synchronization operation of continuous multiframe SLAM.
It is to be understood that vision SLAM needs biggish calculation power when carrying out P2P navigation according to the above embodiments, from
Feature point extraction, the Feature Points Matching of front end, to the tracking of pose, the optimization of rear end, map structuring, and to operation result
Real-time rendering etc. is all a test to the real-time of system, and pure mobile terminal is difficult to meet such calculating requirement.And
And after having used Mask R-CNN and vision SLAM, for each video frame, Mask R-CNN and vision SLAM are strings
What row carried out, therefore the real-time of system has bigger challenge.But qualified navigation system is required to real time execution again,
To obtain good user experience.For this purpose, the embodiment of the present invention also carries out the Real-time Design of navigation system.
Specifically, on the one hand, it is contemplated that since navigation system is carried out using vision SLAM, the embodiment of the present invention can
Will be mutually related from framework, SLAM modules attempt decoupling, only do necessary calculating, increase the efficiency of system.It is another
Aspect, since the frame per second of Mask R-CNN is lower than SLAM, the embodiment of the present invention can allow the two to cooperate, guarantee the reality of system
Shi Yunhang.
With reference to Fig. 2, to be led according to P2P pedestrian in the P2P air navigation aid of view-based access control model SLAM provided in an embodiment of the present invention
The structural schematic diagram for system architecture of navigating.That is, after the mobile phone of pilotage people first takes the video of whole section of track, by video
It is transmitted to server, servers off-line constructs track.In the server, all frames all need the Mask through removing dynamic environment factor
The detection of R-CNN and YOLO mirror-reflection pre-processes, and after monocular SLAM initialization, the rail of camera is tracked by VO
Mark, and three-dimensional map point is generated, construct track map.
Later, follower realizes real-time navigation using the method for smart phone and server communication, in server end meter
All SLAM and Mask R-CNN.The collected each frame of follower, is all uploaded to server, and server completes reorientation
After the calculating of navigation instruction, navigation instruction is returned to the smart phone of follower.In server end, by non-rigid environment
It rejects, the mobile phone of follower is located in the map of track using reorientation, and calculate navigation instruction.
Simultaneously, it is contemplated that the operation frame per second of SLAM is attained by least 10 frames/second on most of servers, and MaskR-
CNN is often less than this frame per second (such as 5 frames/second).In order to which (average visual of 10 frames/second, human eye is temporary for the requirement that reaches real-time navigation
Stay the time), the embodiment of the present invention proposes the strategy that 1 frame Mask R-CNN is used for the frame synchronization of continuous multiframe SLAM.So
The real time execution of navigation system can be effectively ensured, and can guarantee the accuracy of follower's reorientation to a certain extent.
It is wherein optional according to the above embodiments, it is based on the second video and motion profile map, passes through follower's terminal
It guides follower to specifically include the step of advancing in the region for including path: being based on the second video and the first video, to following
Person relocates, and the result based on reorientation and motion profile map, generates the navigation instruction to follower；Navigation is referred to
Order is sent to follower's terminal, to guide follower to advance in the region comprising path.
It is to be understood that the embodiment of the present invention obtains the second video of its captured in real-time from follower's terminal, second view
The image of environment on the way is contained in follower's traveling process in frequency.Therefore, follower's current shooting by that will obtain in real time
The second video carry out characteristic matching with the first video obtained from pilotage people's terminal, can according to the motion profile of pilotage people
Figure realizes the reorientation to follower, the result relocated.
On the basis of obtaining follower's reorientation, then it can be generated further according to the reorientation and motion profile map
Navigation instruction, the navigation instruction can indicate how follower advances to track the motion profile of pilotage people.When it is implemented, then
The navigation instruction can be sent to follower's terminal, follower can then get what how this continued on by follower's terminal
Instruction.
Wherein optional, the step of relocating to follower, specifically includes: obtain in the first video with the second video
The middle immediate frame of present frame obtains camera pose of the camera pose as present frame of key frame as key frame, real
Now relocate.
The step of navigation instruction of follower, is specifically included correspondingly, generating: obtain in the first video key frame it
The data frame of given quantity afterwards, and the displacement based on all data frames calculate average displacement amount；Camera based on present frame
Pose and average displacement calculate the direction of travel of follower's next step, and are based on direction of travel and motion profile map, generate
Navigation instruction.
It is to be understood that the embodiment of the present invention can realize the calculating and acquisition of navigation instruction.Specifically, in the system of completion
After positioning, the embodiment of the present invention is obtained in the video frame of pilotage people with the immediate frame of the present frame of follower first as pass
The camera pose for taking the frame is denoted as by key framecT.Then, system takes out 10 frames after the key frame, by them
Displacement be averaging, obtain average displacement amount, be denoted as tnext(t hereinnextHomogeneous coordinates are used, i.e., in three-dimensional coordinate
Add one-dimensional constant value 1 again afterwards, constitutes [x y z 1]T), determine it as the follower place to be reached in next step.So tnextIncRelative pose relationship under T may be expressed as:ctnext=cT*tnext, reacted next direction that follower should advance.
It is optional later, it can basisctnextX coordinate and z coordinate, calculate direction and the camera imaging plane of the traveling
Angle: α=arctan (z/x).
It is wherein optional, can include 4 kinds with the instruction of predefined follower, be respectively advance, turn left, turning right and
Offset warning.Then it can be defined according to the range of above-mentioned angle: if π/3 π/3 < α < 2, provide " advance " instruction, if 0 < α < π/
3, then the instruction of " right-hand rotation " is provided, if 2 π/3 < α < π, provides the instruction of left-hand rotation.As shown in figure 3, for according to the embodiment of the present invention
The correspondence diagram of navigation instruction and calculating angle, Fig. 3 are shown in the P2P air navigation aid of the view-based access control model SLAM of offer
In the angle value of the direction of travel that is calculated according to above formula and camera imaging plane in different range, it is corresponding provide lead
Navigate the corresponding relationship instructed.
In view of user is when actually using navigation system, it may appear that the case where deviating from set pilotage people track.Cause
This is on the basis of the various embodiments described above, in the step for guiding follower to advance in the region comprising path by follower's terminal
After rapid, if the method for the embodiment of the present invention can also include: to be matched in the second video and the first video in reorientation
Characteristic point be less than given threshold value, then determine follower's bias motion track map, and by enabling auxiliary VO module, tracking with
With person's terminal, to guide follower to return to the motion profile of pilotage people.
Specifically, illustrating follower if being only matched to seldom characteristic point in the reorientation of follower and navigating
There is offset in the track of person, the auxiliary visual odometry module preset at this moment will be immediately turned on, to follower's
Mobile phone is tracked, and follower is guided to return to the track of pilotage people.After follower returns track, navigate from the place of offset
It resumes operation.
Based on identical design, the embodiment of the present invention provides the P2P of view-based access control model SLAM a kind of according to the above embodiments
Navigation device, the device for realizing the P2P navigation of view-based access control model SLAM in the above embodiments.Therefore, in above-mentioned each reality
The description and definition in the P2P air navigation aid of the view-based access control model SLAM of example are applied, can be used for each execution in the embodiment of the present invention
The understanding of module specifically refers to above-described embodiment, is not repeating herein.
One embodiment according to an embodiment of the present invention, structure such as Fig. 4 institute of the P2P navigation device of view-based access control model SLAM
Show, be the structural schematic diagram of the P2P navigation device of view-based access control model SLAM provided in an embodiment of the present invention, which can be used for reality
The P2P navigation of view-based access control model SLAM in existing above-mentioned each method embodiment, the device include: to obtain module 401, calculate and generate mould
Block 402 and navigation instruction module 403.Wherein:
It obtains module 401 and is used to obtain the first video from pilotage people's terminal, the first video is that pilotage people advances along its path
When the video of the environment on the way in path that shoots；It calculates and is used to be based on the first video with generation module 402, utilize monocular vision
SLAM algorithm calculates the camera pose track of pilotage people and generates three-dimensional map point, and based on camera pose track and dimensionally
Figure point generates the motion profile map of pilotage people；Navigation instruction module 403 is for obtaining the second view from follower's terminal in real time
Frequently, and it is based on the second video and motion profile map, guides follower in the region expert comprising path by follower's terminal
Into the second video is the video for the ambient enviroment that follower shoots during traveling.
Specifically, firstly, obtain module 401 can be by pilotage people's terminal for being carried in pilotage people's mobile phone, from navigator
The first video of its shooting is obtained in the mobile phone of person.It is understood that the acquisition process can be from pilotage people's terminal actively
The process of acquisition is also possible to the process that pilotage people's terminal actively issues.
Later, it calculates with generation module 402 on the basis of obtaining the first video, with monocular vision SLAM algorithm according to this
First video calculates camera pose track and three-dimensional map point, as the information of the paths, namely to the first video by single
After mesh SLAM initialization calculates, the track of camera is tracked by visual odometry, and generate three-dimensional map point, structure according to calculating
Build the motion profile map of pilotage people.
Finally, navigation instruction module 403 first passes through the second video that follower's terminal obtains user's shooting, and according to this
Two videos realize the positioning to follower's current time in combination with the first video.Navigation instruction module 403 is according to following later
Person's works as prelocalization and the actual motion profile map of pilotage people, determines the direction of travel of follower's next step, and by following
The direction of travel is supplied to follower by person's terminal, so that follower goes within a certain area along the operating path of pilotage people
Into.
The P2P navigation device of view-based access control model SLAM provided in an embodiment of the present invention, by the way that corresponding execution module is arranged, no
It needs infrastructure, need to provide priori map, the track of pilotage people need to be only constructed by view-based access control model SLAM technology, and
Follower is located in target trajectory using the video that follower shoots and realizes the navigation of user to user, therefore can be had
Effect overcomes the dependence to infrastructure, expands the range of application to a greater extent.
It is understood that can be by hardware processor (hardware processor) come real in the embodiment of the present invention
Each relative program module in the device of existing the various embodiments described above.Also, the P2P of the view-based access control model SLAM of the embodiment of the present invention is led
The device that navigates utilizes above-mentioned each program module, can be realized the P2P flow of navigation of the view-based access control model SLAM of above-mentioned each method embodiment,
In for realizing above-mentioned each method embodiment when the P2P navigation of view-based access control model SLAM, what the device of the embodiment of the present invention generated
Beneficial effect is identical as corresponding above-mentioned each method embodiment, can refer to above-mentioned each method embodiment, details are not described herein again.
As the another aspect of the embodiment of the present invention, the present embodiment provides a kind of electronics according to the above embodiments and sets
Standby, which includes memory, processor and is stored in the computer that can be run on the memory and on the processor
Program when the processor executes the computer program, realizes the navigation side P2P of view-based access control model SLAM described in as above each embodiment
The step of method.
Further, the electronic equipment of the embodiment of the present invention can also include communication interface and bus.With reference to Fig. 5, for this
The entity structure schematic diagram for the electronic equipment that inventive embodiments provide, comprising: at least one processor 501, at least one processing
Device 502, communication interface 503 and bus 504.
Wherein, memory 501, processor 502 and communication interface 503 complete mutual communication by bus 504, communicate
Interface 503 is for the information transmission between the electronic equipment and subscriber terminal equipment；Being stored in memory 501 can be in processor
The computer program run on 502 when processor 502 executes the computer program, realizes the base as described in the various embodiments described above
In the P2P air navigation aid of vision SLAM the step of.
It is to be understood that including at least memory 501, processor 502, communication interface 503 and bus in the electronic equipment
504, and memory 501, processor 502 and communication interface 503 form mutual communication connection by bus 504, and can be complete
At mutual communication, the program as processor 502 reads the P2P air navigation aid of view-based access control model SLAM from memory 501 refers to
Enable etc..In addition, communication interface 503 can also realize the communication connection between the electronic equipment and subscriber terminal equipment, and can be complete
It is transmitted at mutual information, such as realizes the P2P navigation of view-based access control model SLAM by communication interface 503.
When electronic equipment is run, processor 502 calls the program instruction in memory 501, real to execute above-mentioned each method
Apply method provided by example, for example, obtain the first video from pilotage people's terminal, the first video is pilotage people along its path row
Into when the video of the environment on the way in path that shoots；Calculate pilotage people's using monocular vision SLAM algorithm based on the first video
Camera pose track simultaneously generates three-dimensional map point, and is based on camera pose track and three-dimensional map point, generates the movement of pilotage people
Track map；The second video is obtained from follower's terminal in real time, and is based on the second video and motion profile map, passes through follower
Terminal guides follower to advance in the region comprising path etc..
Program instruction in above-mentioned memory 501 can be realized and as independent by way of SFU software functional unit
Product when selling or using, can store in a computer readable storage medium.Alternatively, realizing that above-mentioned each method is implemented
This can be accomplished by hardware associated with program instructions for all or part of the steps of example, and program above-mentioned can store to be calculated in one
In machine read/write memory medium, when being executed, execution includes the steps that above-mentioned each method embodiment to the program；And storage above-mentioned
Medium includes: USB flash disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random
Access Memory, RAM), the various media that can store program code such as magnetic or disk.
The embodiment of the present invention provides a kind of non-transient computer readable storage medium also according to the various embodiments described above, deposits thereon
Computer instruction is contained, when which is computer-executed, realizes the view-based access control model as described in the various embodiments described above
The step of P2P air navigation aid of SLAM, for example, obtain the first video from pilotage people's terminal, the first video is pilotage people edge
The video of the environment on the way in the path that its path is shot when advancing；It is calculated based on the first video using monocular vision SLAM algorithm
The camera pose track of pilotage people simultaneously generates three-dimensional map point, and is based on camera pose track and three-dimensional map point, generates and navigates
The motion profile map of person；The second video is obtained from follower's terminal in real time, and is based on the second video and motion profile map, is led to
Crossing follower's terminal guides follower to advance in the region comprising path, and the second video is that follower shoots during traveling
Ambient enviroment video etc..
Electronic equipment provided in an embodiment of the present invention and non-transient computer readable storage medium, by executing above-mentioned each reality
The P2P air navigation aid for applying view-based access control model SLAM described in example does not need infrastructure, does not need to provide priori map, need to only lead to
View-based access control model SLAM technology is crossed to construct the track of pilotage people, and follower is located in target using the video that follower shoots
The navigation of user to user is realized in track, therefore can effectively overcome the dependence to infrastructure, is expanded to a greater extent
The range of application.
It is understood that the embodiment of device described above, electronic equipment and storage medium is only schematic
, wherein unit may or may not be physically separated as illustrated by the separation member, it can both be located at one
Place, or may be distributed on heterogeneous networks unit.Some or all of modules can be selected according to actual needs
To achieve the purpose of the solution of this embodiment.Those of ordinary skill in the art are without paying creative labor
To understand and implement.
By the description of embodiment of above, those skilled in the art is it will be clearly understood that each embodiment can borrow
Help software that the mode of required general hardware platform is added to realize, naturally it is also possible to pass through hardware.Based on this understanding, above-mentioned
Substantially the part that contributes to existing technology can be embodied in the form of software products technical solution in other words, the meter
Calculation machine software product may be stored in a computer readable storage medium, such as USB flash disk, mobile hard disk, ROM, RAM, magnetic disk or light
Disk etc., including some instructions, with so that a computer equipment (such as personal computer, server or network equipment etc.)
Execute method described in certain parts of above-mentioned each method embodiment or embodiment of the method.
In addition, those skilled in the art are it should be understood that in the application documents of the embodiment of the present invention, term
"include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that including a series of elements
Process, method, article or equipment not only include those elements, but also including other elements that are not explicitly listed, or
Person is to further include for elements inherent to such a process, method, article, or device.In the absence of more restrictions, by
The element that sentence "including a ..." limits, it is not excluded that in the process, method, article or apparatus that includes the element
There is also other identical elements.
In the specification of the embodiment of the present invention, numerous specific details are set forth.It should be understood, however, that the present invention is implemented
The embodiment of example can be practiced without these specific details.In some instances, it is not been shown in detail well known
Methods, structures and technologies, so as not to obscure the understanding of this specification.Similarly, it should be understood that in order to simplify implementation of the present invention
Example is open and helps to understand one or more of the various inventive aspects, above to the exemplary embodiment of the embodiment of the present invention
Description in, each feature of the embodiment of the present invention is grouped together into single embodiment, figure or descriptions thereof sometimes
In.
However, the disclosed method should not be interpreted as reflecting the following intention: i.e. the claimed invention is implemented
Example requires features more more than feature expressly recited in each claim.More precisely, such as claims institute
As reflection, inventive aspect is all features less than single embodiment disclosed above.Therefore, it then follows specific embodiment party
Thus claims of formula are expressly incorporated in the specific embodiment, wherein each claim itself is real as the present invention
Apply the separate embodiments of example.
Finally, it should be noted that above embodiments are only to illustrate the technical solution of the embodiment of the present invention, rather than it is limited
System；Although the embodiment of the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art it is understood that
It is still possible to modify the technical solutions described in the foregoing embodiments, or part of technical characteristic is carried out etc.
With replacement；And these are modified or replaceed, each embodiment skill of the embodiment of the present invention that it does not separate the essence of the corresponding technical solution
The spirit and scope of art scheme.
Claims (10)
1. a kind of P2P air navigation aid of view-based access control model SLAM characterized by comprising
The first video is obtained from pilotage people's terminal, first video is the path shot when pilotage people advances along its path
Environment on the way video；
It calculates the camera pose track of the pilotage people using monocular vision SLAM algorithm based on first video and generates
Three-dimensional map point, and it is based on camera pose track and three-dimensional map point, generate the motion profile map of pilotage people；
The second video is obtained from follower's terminal in real time, and is based on second video and the motion profile map, passes through institute
Stating follower's terminal guides follower to advance in the region comprising the path, and second video is that the follower is expert at
Into the video of the ambient enviroment shot in the process.
2. the P2P air navigation aid of view-based access control model SLAM according to claim 1, which is characterized in that described in the calculating
The camera pose track of pilotage people and the step of generate three-dimensional map point before, further includes:
The dynamic prospect in first video is extracted and removes, to retain the static environment factor in first video, and
Based on the static environment factor, static nature point is extracted；
Correspondingly, calculating camera pose track simultaneously using the monocular vision SLAM algorithm based on the static nature point
Generate the three-dimensional map point.
3. the P2P air navigation aid of view-based access control model SLAM according to claim 2, which is characterized in that the extraction simultaneously removes
The step of dynamic prospect in first video, specifically includes:
Using MaskR-CNN algorithm, the connected domain of dynamic object in first video is extracted, and utilizes the mirror surface of YOLO system
With smooth surface detection instrument, the relevant connected domain of mirror surface in first video is extracted；
The characteristic point in the connected domain and the relevant connected domain of the mirror surface of all dynamic objects is extracted, the dynamic is constituted
Prospect is simultaneously rejected.
4. the P2P air navigation aid of view-based access control model SLAM according to claim 1, which is characterized in that described based on described the
Two videos and the motion profile map guide follower in the region expert comprising the path by follower's terminal
Into the step of specifically include:
Based on second video and first video, the follower is relocated, and based on the reorientation
As a result with the motion profile map, the navigation instruction to the follower is generated；
The navigation instruction is sent to follower's terminal, to guide the follower in the region comprising the path
It advances.
5. the P2P air navigation aid of view-based access control model SLAM according to claim 4, which is characterized in that described to be followed to described
The step of person relocates specifically includes:
It obtains in first video with the immediate frame of present frame in second video as key frame, and described in acquisition
Camera pose of the camera pose of key frame as the present frame, realizes the reorientation；
Correspondingly, the step of navigation instruction generated to the follower, specifically includes:
The data frame of the given quantity in first video after the key frame is obtained, and is based on all data frames
Displacement, calculate average displacement amount；
Camera pose and the average displacement amount based on the present frame, calculate the direction of travel of follower's next step,
And it is based on the direction of travel and the motion profile map, generate the navigation instruction.
6. the P2P air navigation aid of view-based access control model SLAM according to claim 3, which is characterized in that described to calculate the neck
The camera pose track of boat person and generate three-dimensional map point to it is described generate pilotage people motion profile map the step of specifically exist
It is handled in server, and utilizes MaskR-CNN algorithm described, extract the connected domain of dynamic object in first video
The step of in, 1 frame MaskR-CNN is used for the frame synchronization operation of continuous multiframe SLAM.
7. the P2P air navigation aid of view-based access control model SLAM according to claim 4, which is characterized in that described by described
After the step of follower's terminal guides follower to advance in the region for including the path, further includes:
If the characteristic point being matched in second video and first video in the reorientation is less than given threshold value,
Then determine that the follower deviates the motion profile map, and by enabling preset auxiliary visual odometry module, tracking
Follower's terminal, to guide the follower to return to the motion profile of the pilotage people.
8. a kind of P2P navigation device of view-based access control model SLAM characterized by comprising
Module is obtained, for obtaining the first video from pilotage people's terminal, when first video is that pilotage people advances along its path
The video of the environment on the way in the path of shooting；
Calculating and generation module, for calculating the pilotage people's using monocular vision SLAM algorithm based on first video
Camera pose track simultaneously generates three-dimensional map point, and is based on camera pose track and three-dimensional map point, generates and navigates
The motion profile map of person；
Navigation instruction module for obtaining the second video from follower's terminal in real time, and is based on second video and the fortune
Dynamic rail slash figure guides follower to advance in the region comprising the path by follower's terminal, second view
Frequency is the video for the ambient enviroment that the follower shoots during traveling.
9. a kind of electronic equipment, including memory, processor and it is stored on the memory and can transports on the processor
Capable computer program, which is characterized in that when the processor executes the computer program, realize as in claim 1 to 7
The step of P2P air navigation aid of described in any item view-based access control model SLAM.
10. a kind of non-transient computer readable storage medium, is stored thereon with computer instruction, which is characterized in that the calculating
When machine instruction is computer-executed, the navigation side P2P of the view-based access control model SLAM as described in any one of claims 1 to 7 is realized
The step of method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910618614.9A CN110440802A (en) | 2019-07-10 | 2019-07-10 | P2P air navigation aid, device and the electronic equipment of view-based access control model SLAM |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910618614.9A CN110440802A (en) | 2019-07-10 | 2019-07-10 | P2P air navigation aid, device and the electronic equipment of view-based access control model SLAM |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110440802A true CN110440802A (en) | 2019-11-12 |
Family
ID=68430037
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910618614.9A Pending CN110440802A (en) | 2019-07-10 | 2019-07-10 | P2P air navigation aid, device and the electronic equipment of view-based access control model SLAM |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110440802A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102062605A (en) * | 2009-11-16 | 2011-05-18 | 高德软件有限公司 | Navigation method and device |
| CN104949679A (en) * | 2014-03-31 | 2015-09-30 | 高德信息技术有限公司 | Navigation information determination method and navigation information determination device |
| CN105277203A (en) * | 2014-06-30 | 2016-01-27 | 高德信息技术有限公司 | Navigation action generation method, navigation method and device |
| CN106289296A (en) * | 2016-09-05 | 2017-01-04 | 广州极飞科技有限公司 | A kind of method and apparatus of road guide |
| CN108168539A (en) * | 2017-12-21 | 2018-06-15 | 儒安科技有限公司 | A kind of blind man navigation method based on computer vision, apparatus and system |
| CN108235246A (en) * | 2017-12-19 | 2018-06-29 | 清华大学 | A kind of indoor orientation method and system |
| CN109828580A (en) * | 2019-02-27 | 2019-05-31 | 华南理工大学 | A kind of Mobile Robot Formation's tracking and controlling method based on separate type ultrasonic wave |
-
2019
- 2019-07-10 CN CN201910618614.9A patent/CN110440802A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102062605A (en) * | 2009-11-16 | 2011-05-18 | 高德软件有限公司 | Navigation method and device |
| CN104949679A (en) * | 2014-03-31 | 2015-09-30 | 高德信息技术有限公司 | Navigation information determination method and navigation information determination device |
| CN105277203A (en) * | 2014-06-30 | 2016-01-27 | 高德信息技术有限公司 | Navigation action generation method, navigation method and device |
| CN106289296A (en) * | 2016-09-05 | 2017-01-04 | 广州极飞科技有限公司 | A kind of method and apparatus of road guide |
| CN108235246A (en) * | 2017-12-19 | 2018-06-29 | 清华大学 | A kind of indoor orientation method and system |
| CN108168539A (en) * | 2017-12-21 | 2018-06-15 | 儒安科技有限公司 | A kind of blind man navigation method based on computer vision, apparatus and system |
| CN109828580A (en) * | 2019-02-27 | 2019-05-31 | 华南理工大学 | A kind of Mobile Robot Formation's tracking and controlling method based on separate type ultrasonic wave |
Non-Patent Citations (1)
| Title |
|---|
| ERQUN DONG 等: "Pair-Navi: Peer-to-Peer Indoor Navigation with Mobile Visual SLAM", 《IEEE INFOCOM 2019-IEEE CONFERENCE ON COMPUTER COMMUNICATIONS》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112435325B (en) | VI-SLAM and depth estimation network-based unmanned aerial vehicle scene density reconstruction method | |
| US11328158B2 (en) | Visual-inertial positional awareness for autonomous and non-autonomous tracking | |
| CN109166149B (en) | Positioning and three-dimensional line frame structure reconstruction method and system integrating binocular camera and IMU | |
| US10366508B1 (en) | Visual-inertial positional awareness for autonomous and non-autonomous device | |
| US10410328B1 (en) | Visual-inertial positional awareness for autonomous and non-autonomous device | |
| CN110070615A (en) | A kind of panoramic vision SLAM method based on polyphaser collaboration | |
| Strasdat et al. | Double window optimisation for constant time visual SLAM | |
| CN104200523B (en) | A kind of large scene three-dimensional rebuilding method for merging additional information | |
| CN107967457A (en) | A kind of place identification for adapting to visual signature change and relative positioning method and system | |
| CN113706699B (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN106840148A (en) | Wearable positioning and path guide method based on binocular camera under outdoor work environment | |
| WO2016210227A1 (en) | Aligning 3d point clouds using loop closures | |
| CN106327571A (en) | Three-dimensional face modeling method and three-dimensional face modeling device | |
| CN102663722A (en) | Moving object segmentation using depth images | |
| Yang et al. | Reactive obstacle avoidance of monocular quadrotors with online adapted depth prediction network | |
| Zhang et al. | Bundle adjustment for monocular visual odometry based on detections of traffic signs | |
| Chiu et al. | Sub-meter vehicle navigation using efficient pre-mapped visual landmarks | |
| Khan et al. | Towards monocular neural facial depth estimation: Past, present, and future | |
| Jaramillo et al. | 6-DoF pose localization in 3D point-cloud dense maps using a monocular camera | |
| CN110440802A (en) | P2P air navigation aid, device and the electronic equipment of view-based access control model SLAM | |
| Sujiwo et al. | Robust and accurate monocular vision-based localization in outdoor environments of real-world robot challenge | |
| Muhlbauer et al. | Navigation through urban environments by visual perception and interaction | |
| Gu et al. | Unsupervised cycle optimization learning for single-view depth and camera pose with Kalman filter | |
| Ren et al. | LiveHPS: LiDAR-based Scene-level Human Pose and Shape Estimation in Free Environment | |
| CN111583331A (en) | Method and apparatus for simultaneous localization and mapping |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191112 |