CN103196430A - Mapping navigation method and system based on flight path and visual information of unmanned aerial vehicle - Google Patents

Mapping navigation method and system based on flight path and visual information of unmanned aerial vehicle Download PDF

Info

Publication number
CN103196430A
CN103196430A CN2013101538286A CN201310153828A CN103196430A CN 103196430 A CN103196430 A CN 103196430A CN 2013101538286 A CN2013101538286 A CN 2013101538286A CN 201310153828 A CN201310153828 A CN 201310153828A CN 103196430 A CN103196430 A CN 103196430A
Authority
CN
China
Prior art keywords
unmanned plane
information
flight path
visual
mapping
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN2013101538286A
Other languages
Chinese (zh)
Other versions
CN103196430B (en
Inventor
戴琼海
刘慧�
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tsinghua University
Original Assignee
Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tsinghua University filed Critical Tsinghua University
Priority to CN201310153828.6A priority Critical patent/CN103196430B/en
Publication of CN103196430A publication Critical patent/CN103196430A/en
Application granted granted Critical
Publication of CN103196430B publication Critical patent/CN103196430B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Navigation (AREA)

Abstract

The invention provides a mapping navigation method and system based on the flight path and visual information of an unmanned aerial vehicle. The method comprises the following steps of: collecting visual images of the unmanned aerial vehicle; analyzing the visual patterns by an unmanned aerial vehicle image understanding system so as to obtain feature information and semantic text information of the visual images; determining the flight path of the unmanned aerial vehicle according to the feature information and the semantic text information of the visual images; generating a mapping relation data base according to the visual images and the corresponding flight path, wherein the mapping relation data base is generated in real time or loaded in advance according to the image understanding system; and navigating the unmanned aerial vehicle according to the mapping relation data base. According to the method, the flight path and the visual images are associated to generate the mapping relation data base, and people learn according to the mapping relation data base, so that the accuracy of the navigation of the unmanned aerial vehicle is improved, and meanwhile, and the flight security of the unmanned aerial vehicle is improved.

Description

Based on the flight path of unmanned plane and mapping air navigation aid and the system of visual information
Technical field
The present invention relates to the unmanned plane technical field, particularly a kind of based on the flight path of unmanned plane and mapping air navigation aid and the system of visual information.
Background technology
At the complicacy that concerns between the huge property of the diversity of complicated day by day information environment and expression forms of information, information content, the information, and require real-time to information processing, information fusion is as the informix treatment technology, and its importance highlights day by day.By can improve the quality of data to the fusion of different information, obtain useful complete information to greatest extent to realize the identification to environment, the detection of target, location, functions such as independent navigation.The visual navigation of unmanned plane is a kind of novel air navigation aid that comprehensive vision, flight, blank pipe and the communication technology are organically combined, and is that the typical case of information fusion uses.Existing in the visual navigation needs sensing equipment many, and the visual information data volume is big, and the Data Fusion real-time requires problems such as height.
At present, merge the method for navigating or following the tracks of at flight path and mainly be divided into two classes, one class is centralized flight path fusion method, another kind of is distributed flight path fusion method, and mostly fusion method is to merge the method that generates low error flight path or the problem that solves track association at multisensor or multi-source information, as: the fusion method of simple variance convex combination method, weighting fusion method, multiple step format filtering etc.But prior art does not have to consider the understanding to visual information basically, is short of to some extent improving the accurate visual navigation of unmanned plane aspect, and the method that flight path and visual information is merged to mate mutually mapping yet there are no.The mode that solves handling problem with existing flight path integration technology is different, the present invention is in the visual navigation flight environment of unmanned plane, its flight path and visual information are merged mutually, make unmanned plane have the function of mapping study, can improve the accuracy of unmanned plane image understanding system effectively, improve real-time and matching, for realizing that unmanned plane is independently located, path planning provides reliable hedging to keep away the barrier navigation information with safe flight.
Summary of the invention
Purpose of the present invention is intended to solve at least one of above-mentioned technological deficiency.
For this reason, one object of the present invention is to propose the mapping air navigation aid of a kind of flight path based on unmanned plane and visual information.
Another object of the present invention is to propose the mapping navigational system of a kind of flight path based on unmanned plane and visual information.
For achieving the above object, the embodiment of one aspect of the present invention proposes the mapping air navigation aid of a kind of flight path based on unmanned plane and visual information, may further comprise the steps:
Gather the visual pattern of unmanned plane; Utilize the unmanned plane image understanding system to analyze visual pattern to obtain characteristic information and the semantic text information of visual pattern; Characteristic information and semantic text information according to described visual pattern are determined described unmanned plane during flying track; Generate the mapping relations database according to described visual pattern and corresponding flight path, wherein, described mapping relations database can comprise the flight path information of described unmanned plane in different visual patterns, attitude, speed, strategy and corresponding visual pattern in order to retrieval study and/or transmission, and described mapping relations database data can generate in real time or be written in advance according to image understanding system; And
According to described mapping relations database described unmanned plane is navigated.
In one embodiment of the present of invention, the step of described unmanned plane being navigated according to described mapping relations database, further comprise: when visual pattern matches in visual pattern that described unmanned plane is gathered and the described mapping relations database, adopt the flight strategy corresponding with visual pattern in the described mapping relations database that described unmanned plane is navigated.
In one embodiment of the present of invention, also comprise: when the matching degree of visual pattern that described unmanned plane is gathered and the visual pattern in the described mapping relations database during greater than threshold value, by the mapping mode of learning flight strategy corresponding with described visual pattern described unmanned plane is navigated.
In one embodiment of the present of invention, the corresponding flight strategy of the visual pattern that matching degree is the highest navigates to described unmanned plane as final flight strategy.
In one embodiment of the present of invention, described semantic text information comprises: the shape of object in the described visual pattern, size, motion state, with the relative distance of described unmanned plane and the status information of background context.
In one embodiment of the present of invention; storage unmanned plane during flying flight path and corresponding visual information in the described mapping relations database; and described unmanned plane during flying flight path and corresponding visual information are classified according to the feature of flight path or visual information, the time sequencing that flight path takes place, and unmanned plane is retrieved according to different types of data in retrieving.
In one embodiment of the present of invention, described unmanned plane positions with corresponding visual information according to the flight track of historical flight track with corresponding visual information, current time, and the described visual information according to current acquisition is shone upon study, realizes flight path prediction and the path planning of described unmanned plane.
In one embodiment of the present of invention, described mapping mode of learning further comprises: according to feature recognition result and/or semantic relevant explanation, retrieval and described relevant flight path and visual information like the current data class that obtains from the mapping relations database, and according to shining upon study with the highest flight path and the visual information of described feature recognition result and/or semantic relevant explanation matching degree.
In one embodiment of the present of invention, when described unmanned plane has map maybe can obtain map, generate flight path information by described map, unmanned plane can directly carry out navigation flight according to the trace information that is generated by map, for the prevention environment changes, the trace information that is generated by map can be complementary to merge to generate and hang down error flight path information described unmanned plane is navigated with the prediction flight path information that merges mapping study according to flight path and visual information.
In one embodiment of the present of invention, described visual information comprises: characteristic information and semantic text information.
Method according to the embodiment of the invention,, flight path generates the mapping relations database by being associated with visual pattern, and learn according to this mapping relations database, thereby improved the accuracy that unmanned plane is navigated, the security that has improved unmanned plane during flying simultaneously.
For achieving the above object, embodiments of the invention propose the mapping navigational system of a kind of flight path based on unmanned plane and visual information on the other hand, comprising: acquisition module, for the visual pattern of gathering unmanned plane; Analysis module is used for analyzing the semantic text information that described visual pattern reaches with the characteristic information that obtains described visual pattern and the characteristic information of described visual pattern mates; Determination module is used for the flight path of determining described unmanned plane according to characteristic information and the semantic text information of visual pattern; Generation module, be used for generating the mapping relations database according to described visual pattern and corresponding flight path, wherein, described mapping relations database can comprise that the flight path information of described unmanned plane in different visual patterns, attitude, speed, strategy and corresponding visual pattern are in order to retrieval study and/or transmission, the mapping relations database data can generate in real time according to image understanding system, also can be written in advance; And navigation module, be used for according to described mapping relations database described unmanned plane being navigated.
In one embodiment of the present of invention, when described navigation module also is used for the visual pattern gathered when described unmanned plane and described mapping relations database visual pattern and matches, adopt the flight strategy corresponding with visual pattern in the described mapping relations database that described unmanned plane is navigated.
In one embodiment of the present of invention, also comprise: the study navigation module, the matching degree of visual pattern that is used for visual pattern that described unmanned plane gathers and described mapping relations database is navigated to described unmanned plane by the mapping mode of learning flight strategy corresponding with described visual pattern during greater than threshold value.
In one embodiment of the present of invention, described study navigation module also is used for the corresponding flight strategy of the visual pattern that matching degree is the highest and as final flight strategy described unmanned plane is navigated.
In one embodiment of the present of invention, described semantic text information comprises: the shape of object in the described visual pattern, size, motion state, with the relative distance of described unmanned plane and the status information of background context.
System according to the embodiment of the invention,, flight path generates the mapping relations database by being associated with visual pattern, and learn according to this mapping relations database, thereby improved the accuracy that unmanned plane is navigated, the security that has improved unmanned plane during flying simultaneously.
The aspect that the present invention adds and advantage part in the following description provide, and part will become obviously from the following description, or recognize by practice of the present invention.
Description of drawings
Above-mentioned and/or the additional aspect of the present invention and advantage are from obviously and easily understanding becoming the description of embodiment below in conjunction with accompanying drawing, wherein:
Fig. 1 is according to an embodiment of the invention based on the process flow diagram of the mapping air navigation aid of the flight path of unmanned plane and visual information;
Fig. 2 is the frame diagram according to the unmanned plane image understanding system of the embodiment of the invention;
Fig. 3 (a) (b) (c) (d) determines the synoptic diagram of flight path for unmanned plane according to an embodiment of the invention;
Fig. 4 is according to an embodiment of the invention based on the structured flowchart of the mapping navigational system of the flight path of unmanned plane and visual information; And
Fig. 5 is in accordance with another embodiment of the present invention based on the structured flowchart of the mapping navigational system of the flight path of unmanned plane and visual information.
Embodiment
Describe embodiments of the invention below in detail, the example of embodiment is shown in the drawings, and wherein identical or similar label is represented identical or similar elements or the element with identical or similar functions from start to finish.Be exemplary below by the embodiment that is described with reference to the drawings, only be used for explaining the present invention, and can not be interpreted as limitation of the present invention.
Fig. 1 is according to an embodiment of the invention based on the process flow diagram of the mapping air navigation aid of the flight path of unmanned plane and visual information.As shown in Figure 1, according to the embodiment of the invention based on the flight path of unmanned plane and the mapping air navigation aid of visual information, may further comprise the steps:
Step S101, the visual pattern of collection unmanned plane.
Particularly, gather the visual pattern of unmanned plane in flight course in real time by the airborne visual sensing equipment of unmanned plane, for example, video camera etc.Can also obtain the required video image information of visual navigation from other unmanned planes or equipment.
Step S102 utilizes the unmanned plane image understanding system to analyze visual pattern obtaining the characteristic information of visual pattern, and from the iconic model storehouse image, semantic of search and the characteristic information coupling of visual pattern with the generative semantics text message.Semantic text information comprises: the shape of object in the visual pattern, size, with the relative distance of unmanned plane and the status information of background context etc.
Particularly, handle and identify to obtain the characteristic information of video image by the video image of gathering being carried out image.Search in the iconic model storehouse then, wherein, this iconic model storehouse comprises that abundant image makes the video image that obtains is identified to obtain image, semantic.Afterwards again unmanned plane according to image, semantic generative semantics text message.
Fig. 2 is the frame diagram according to the unmanned plane image understanding system of the embodiment of the invention.As shown in Figure 2, the unmanned plane image understanding system comprises: the real-time vision information acquisition module is used for gathering the required video image information of visual navigation and/or obtaining the required video image information of visual navigation by communication from other unmanned planes or equipment by airborne visual sensing equipment.The image characteristics extraction module is used for obtaining by image processing and recognition technology the feature of video image information.The iconic model storehouse is used for the memory image model information, retrieves identification for the image, semantic generation module.The dictionary knowledge base is used for the lexical or textual analysis between storage keyword and the keyword, retrieves identification for the image, semantic generation module.Flight path and visual information merge the dependency rule that mapping block is used for setting up flight path and the corresponding visual information of flight path, and storage unmanned plane during flying flight path and corresponding visual information are learnt in order to retrieval and/or transmitted.The image, semantic generation module is used for the fusion information generative semantics text message according to feature and flight path and the visual information of video image information.Fig. 2 is the image understanding system synoptic diagram of the fusion mapping mode of unmanned plane during flying flight path and visual information in the visual navigation of the embodiment of the invention; Unmanned plane is gathered the required video image information of visual navigation by the airborne visual sensing equipment of real-time vision information acquisition module acquisition and/or is obtained by communication from other unmanned planes or the next required video image information of visual navigation of device transmission; Obtain the feature of video image information by image processing and recognition technology in the image characteristics extraction module; The gained characteristic information is identified with the graph image coupling in iconic model storehouse simultaneously, and can obtain crucial semantic interpretation by the image, semantic generation module; According to feature recognition result and/or semantic relevant explanation, the retrieval flight path merges similar relevant flight path and the visual information of storing in the mapping block with visual information, according to shining upon study with the highest flight path and the visual information of current feature recognition result and/or semantic relevant explanation matching degree, obtain the prediction flight path information of current visual information correspondence, can further merge mutually with the explanation results of image, semantic generation module to current visual information, generate the unmanned plane navigation instruction and continue flight.
In one embodiment of the invention; storage unmanned plane during flying flight path and corresponding visual information in the mapping relations database; and unmanned plane during flying flight path and corresponding visual information are classified according to the feature of flight path or visual information, the time sequencing that flight path takes place, and unmanned plane is retrieved according to different types of data in retrieving.Unmanned plane positions with corresponding visual information according to the flight track of historical flight track with corresponding visual information, current time, and shines upon study according to the visual information of current acquisition, realizes flight path prediction and the path planning of unmanned plane.
Step S103 determines the unmanned plane during flying track according to characteristic information and the semantic text information of visual pattern.
Particularly, moving situation from semantic text information institute acquired disturbance thing, for example, static, constant speed or speed change move, the shape of barrier, for example, circle, ball, square etc. also have semantic text information such as slit, circular hole, door, window, tree, mountain, house, bird, flying object to adopt corresponding flight path in addition.For example, according to crossing from the barrier top with the relative height of unmanned plane or mode such as cut-through thing is determined the flight path of unmanned plane.Fig. 3 determines the synoptic diagram of flight path for unmanned plane according to an embodiment of the invention.Shown in the (a) and (b) of Fig. 3, (c), (d), as when running into slit, can adopt the mode of flying nonstop to determine flight path; Run into tree or buildings, and the width according to barrier adopts the mode that detours to determine flight path should not fly over the time; When barrier being arranged in the air, adopt the mode of low flight to determine flight path; When running into the turning that buildings or mountain etc. form, turning flight then.Detailed process can represent that retrieving reference locus is S by following date expression r=f r(x, y, z, I r, t), its corresponding visual information is I r=F r(x', y', z', λ, t), current visual information to be matched is I n=F n(x', y', z', λ, t), matching degree is MD n=g (F r, F n), wherein, g is expressed as matching process, if MD nD c, d cBe the matching degree threshold value, then by reference locus can proper anterior optic information prediction locus information be S n=f n(x, y, z, I n, t).X wherein, y, z are the trajectory range coordinate, I rExpression reference picture intensity, I nRepresent current image intensity to be matched, t represents the time, x', and y', z' is the image space coordinate, λ represents optical wavelength.
Step S104, generate the mapping relations database according to visual pattern and corresponding flight path, wherein, the mapping relations database can comprise the flight path information of unmanned plane in different visual patterns, attitude, speed, strategy and corresponding visual pattern in order to retrieval study and/or transmission, and the mapping relations database data can generate in real time or be written in advance according to image understanding system.
Particularly, adopt corresponding flight strategy to determine the flight path of unmanned plane to different visual patterns.This visual pattern and corresponding flight strategy are formed one group of information, a plurality of different visual patterns are obtained a plurality of flight strategies to generate the mapping relations database.As shown in Figure 3, with slit with fly nonstop to strategy and form one group of information; The tree that should not fly over or buildings and the strategy that detours are formed one group of information.Thus, make up by the visual pattern mapping relations database corresponding with the flight strategy.
In addition, the foundation of flight strategy, can also be according to the unmanned plane during flying time, set up the detailed corresponding relation of each historical track points and its place's visual information, as locating the index of visual information with the track points position coordinates as this, also can extract track points and the visual information thereof of crucial historical juncture and set up corresponding relation, perhaps extract crucial flight path line segment and key visual information thereof and set up corresponding relation.Wherein when index flight path visual information, can index the attitude that unmanned plane should the historical juncture simultaneously, highly, speed and obtain the information such as computation rule of this historical flight path of moment according to this place's visual information.Can according to flight path feature or zone as: turn, soar, fly nonstop to, fly at low altitude, be diversion etc. or the text message of visual information feature as the barrier of the dynamic barrier of high or low static-obstacle thing, high or low certain speed, definite shape (as: circle, ball, side etc.), slit, tree/mountain/house/bird/flying object etc., set up the correlation computations rule of flight path and its place's visual information.
In one embodiment of the invention, the visual pattern in the visual pattern that unmanned plane is gathered and the mapping relations database compares, if its matching degree during greater than threshold value, will be navigated to unmanned plane by the study flight strategy corresponding with visual pattern.For example, just it varies in size for two trees that run into before and after the one tree unmanned plane, and unmanned plane can be learnt previous flight strategy unmanned plane was navigated this moment.In matching process, the corresponding flight strategy of the visual pattern that matching degree is the highest navigates to unmanned plane as final flight strategy.
According to feature recognition result and/or semantic relevant explanation, retrieval and relevant flight path and visual information like the current data class that obtains from the mapping relations database, and according to shining upon study with the highest flight path and the visual information of feature recognition result and/or semantic relevant explanation matching degree.Particularly, can be according to shining upon study with the highest flight path and the visual information of current feature recognition result and/or semantic relevant explanation matching degree, specifically get matching degree and surpass the policing rule that certain threshold value and the higher or the highest flight path of matching degree are generated by its visual information, the corresponding flight path that obtains current visual information is replaced in the related data substitution of current visual information, be unmanned plane and merge the prediction flight path information that mapping block study obtains by flight path and visual information.When matching degree surpasses the higher or the highest flight path of certain threshold value and matching degree with visual information during above one, the desirable prediction flight path information that each utilizes the highest flight path of matching degree and visual information mapping study to obtain, under the situation that time delay allows, the fusion association algorithm that also can design many flight path information of forecastings merges the final prediction flight path information that obtains.When within a certain period of time, when not retrieving flight path that matching degree surpasses certain threshold value with its visual information, then export specific character and represent not learn not obtain predicting flight path information.
In one embodiment of the invention, when unmanned plane has map maybe can obtain map, the track that generates by map, and according to flight path and flight path and visual information unmanned plane is navigated.Visual information comprises: characteristic information and semantic text information.
Step S105 navigates to unmanned plane according to the mapping relations database.
Particularly, the visual pattern information that unmanned plane is gathered in visual pattern and the mapping relations database in real time compares, when visual pattern matches in the visual pattern of gathering and the mapping relations database, adopt the flight strategy corresponding with visual pattern in the mapping relations database that unmanned plane is navigated.
Method according to the embodiment of the invention,, flight path generates the mapping relations database by being associated with visual pattern, and learn according to this mapping relations database, thereby improved the accuracy that unmanned plane is navigated, the security that has improved unmanned plane during flying simultaneously.
In one embodiment of the invention, map is being arranged in advance or having the numerical map generator to produce under the situation of cartographic information, unmanned plane can directly carry out navigation flight according to the trace information that is generated by map, for the prevention environment changes, the trace information that is generated by map can be complementary to merge to generate and hang down error flight path information and carry out navigation flight with the prediction flight path information that merges mapping block mapping study according to flight path and visual information.Also can directly cartographic information be fused in the mapping relations database that unmanned aerial vehicle flight path and visual information merges mapping block or in the graph image model bank of unmanned plane image understanding system and generate visual navigation information with identification and coupling.
Fig. 4 is according to an embodiment of the invention based on the structured flowchart of the mapping navigational system of the flight path of unmanned plane and visual information.As shown in Figure 4, comprise acquisition module 100, analysis module 200, determination module 300, generation module 400 and navigation module 500 according to the embodiment of the invention based on the flight path of unmanned plane and the mapping navigational system of visual information.
Acquisition module 100 is used for gathering the visual pattern of unmanned plane.
Particularly, acquisition module 100 is gathered the visual pattern of unmanned plane in flight course in real time by the airborne visual sensing equipment of unmanned plane, for example, and video camera etc.Can also obtain the required video image information of visual navigation from other unmanned planes or equipment.
Analysis module 200 is for analyzing the semantic text information that visual pattern reaches with the characteristic information that obtains visual pattern and the characteristic information of visual pattern mates.
Particularly, analysis module 200 handles and identifies to obtain the characteristic information of video image by the video image of gathering being carried out image.Search in the iconic model storehouse then, wherein, this iconic model storehouse comprises that abundant image makes the video image that obtains is identified to obtain image, semantic.Afterwards again unmanned plane according to image, semantic generative semantics text message.
Determination module 300 is for the flight path of determining unmanned plane according to characteristic information and the semantic text information of visual pattern.
Particularly, determination module 300 is from the moving situation of semantic text information institute acquired disturbance thing, for example, static, constant speed or speed change moves, the shape of barrier, for example, circle, ball, square etc. also have semantic text information such as slit, circular hole, door, window, tree, mountain, house, bird, flying object to adopt corresponding flight path in addition.For example, according to crossing from the barrier top with the relative height of unmanned plane or mode such as cut-through thing is determined the flight path of unmanned plane.Fig. 3 determines the synoptic diagram of flight path for unmanned plane according to an embodiment of the invention.Shown in the (a) and (b) of Fig. 3, (c), (d), as when running into slit, can adopt the mode of flying nonstop to determine flight path; Run into tree or buildings, and the width according to barrier adopts the mode that detours to determine flight path should not fly over the time; When barrier being arranged in the air, adopt the mode of low flight to determine flight path; When running into the turning that buildings or mountain etc. form, turning flight then.Detailed process can represent that retrieving reference locus is S by following date expression r=f r(x, y, z, I r, t), its corresponding visual information is I r=F r(x', y', z', λ, t), current visual information to be matched is I n=F n(x', y', z', λ, t), matching degree is MD n=g (F r, F n), wherein, g is expressed as matching process, if MD nD c, d cBe the matching degree threshold value, then by reference locus can proper anterior optic information prediction locus information be S n=f n(x, y, z, I n, t).Wherein, x, y, z are the trajectory range coordinate, I rExpression reference picture intensity, I nRepresent current image intensity to be matched, t represents the time, x', and y', z' is the image space coordinate, λ represents optical wavelength.
Generation module 400 is used for generating the mapping relations database according to visual pattern and corresponding flight path, and wherein, the mapping relations database data can generate in real time according to image understanding system, also can be written in advance.Semantic text information comprises: the shape of object in the visual pattern, size, with the relative distance of unmanned plane and the status information of background context.
Particularly, 300 pairs of different visual patterns of determination module adopt corresponding flight strategy to determine the flight path of unmanned plane, and by generation module 400 this visual pattern and corresponding flight strategy are formed one group of information, a plurality of different visual patterns are obtained a plurality of flights strategies with generation mapping relations database.As shown in Figure 3, with slit with fly nonstop to strategy and form one group of information; The tree that should not fly over or buildings and the strategy that detours are formed one group of information.Thus, make up by the visual pattern mapping relations database corresponding with the flight strategy.
In addition, the foundation of flight strategy, can also be according to the unmanned plane during flying time, set up the detailed corresponding relation of each historical track points and its place's visual information, as locating the index of visual information with the track points position coordinates as this, also can extract track points and the visual information thereof of crucial historical juncture and set up corresponding relation, perhaps extract crucial flight path line segment and key visual information thereof and set up corresponding relation.Wherein when index flight path visual information, can index the attitude that unmanned plane should the historical juncture simultaneously, highly, speed and obtain the information such as computation rule of this historical flight path of moment according to this place's visual information.Can according to flight path feature or zone as: turn, soar, fly nonstop to, fly at low altitude, be diversion etc. or the text message of visual information feature as the barrier of the dynamic barrier of high or low static-obstacle thing, high or low certain speed, definite shape (as: circle, ball, side etc.), slit, tree/mountain/house/bird/flying object etc., set up the correlation computations rule of flight path and its place's visual information.
Navigation module 500 is used for according to the mapping relations database unmanned plane being navigated.
Particularly, the visual pattern information that navigation module 500 will be gathered in visual pattern and the mapping relations database in real time compares, when visual pattern matches in the visual pattern of gathering and the mapping relations database, adopt the flight strategy corresponding with visual pattern in the mapping relations database that unmanned plane is navigated.
Fig. 5 is in accordance with another embodiment of the present invention based on the structured flowchart of the mapping navigational system of the flight path of unmanned plane and visual information.As shown in Figure 5, the mapping navigational system based on the flight path of unmanned plane and visual information according to the embodiment of the invention also comprises study navigation module 600.
The matching degree of visual pattern that study navigation module 600 is used for visual pattern that unmanned plane gathers and mapping relations database is navigated to unmanned plane by the mapping mode of learning flight strategy corresponding with visual pattern during greater than threshold value.For example, just it varies in size for two trees that run into before and after the one tree unmanned plane, and unmanned plane can be learnt previous flight strategy unmanned plane was navigated this moment.In matching process, the corresponding flight strategy of the visual pattern that matching degree is the highest navigates to unmanned plane as final flight strategy.
In one embodiment of the invention, the study navigation module 600 corresponding flight strategies of visual pattern that matching degree is the highest navigate to unmanned plane as final flight strategy.
In one embodiment of the invention, analysis module 200 is according to feature recognition result and/or semantic relevant explanation, retrieval and relevant flight path and visual information like the current data class that obtains from the mapping relations database are learnt navigation module 600 bases and feature recognition result and/or semantic relevant explanation matching degree the highest flight path and visual information and are shone upon study.Particularly, study navigation module 600 is according to shining upon study with the highest flight path and the visual information of current feature recognition result and/or semantic relevant explanation matching degree, specifically get matching degree and surpass the policing rule that certain threshold value and the higher or the highest flight path of matching degree are generated by its visual information, study navigation module 600 is the corresponding flight path of the current visual information of related data substitution displacement acquisition of current visual information unmanned plane and merges the prediction flight path information that mapping block study obtains by flight path and visual information.When matching degree surpasses the higher or the highest flight path of certain threshold value and matching degree with visual information during above one, the desirable prediction flight path information that each utilizes the highest flight path of matching degree and visual information mapping study to obtain, under the situation that time delay allows, the fusion association algorithm that also can design many flight path information of forecastings merges the final prediction flight path information that obtains.If within a certain period of time, when not retrieving flight path that matching degree surpasses certain threshold value with its visual information, study navigation module 600 output specific characters represent not learn not obtain predicting flight path information.
In one embodiment of the invention, when unmanned plane has map maybe can obtain map, the track that generates by map, and according to flight path and flight path and visual information unmanned plane is navigated.Visual information comprises: characteristic information and semantic text information.
System according to the embodiment of the invention,, flight path generates the mapping relations database by being associated with visual pattern, and learn according to this mapping relations database, thereby improved the accuracy that unmanned plane is navigated, the security that has improved unmanned plane during flying simultaneously.
Although illustrated and described embodiments of the invention above, be understandable that, above-described embodiment is exemplary, can not be interpreted as limitation of the present invention, those of ordinary skill in the art can change above-described embodiment under the situation that does not break away from principle of the present invention and aim within the scope of the invention, modification, replacement and modification.

Claims (15)

1. one kind based on the flight path of unmanned plane and the mapping air navigation aid of visual information, it is characterized in that, may further comprise the steps:
Gather the visual pattern of unmanned plane;
Utilize the unmanned plane image understanding system to analyze visual pattern to obtain characteristic information and the semantic text information of visual pattern;
Characteristic information and semantic text information according to described visual pattern are determined described unmanned plane during flying track;
Generate the mapping relations database according to described visual pattern and corresponding flight path, wherein, described mapping relations database can comprise the flight path information of described unmanned plane in different visual patterns, attitude, speed, strategy and corresponding visual pattern in order to retrieval study and/or transmission, and described mapping relations database data can generate in real time or be written in advance according to image understanding system; And
According to described mapping relations database described unmanned plane is navigated.
2. as claimed in claim 1ly it is characterized in that based on the flight path of unmanned plane and the mapping air navigation aid of visual information,, further comprise the step that described unmanned plane navigates according to described mapping relations database:
When visual pattern matches in visual pattern that described unmanned plane is gathered and the described mapping relations database, adopt the flight strategy corresponding with visual pattern in the described mapping relations database that described unmanned plane is navigated.
3. as claimed in claim 1ly it is characterized in that based on the flight path of unmanned plane and the mapping air navigation aid of visual information, also comprise:
When the matching degree of visual pattern that described unmanned plane is gathered and the visual pattern in the described mapping relations database during greater than threshold value, by the mapping mode of learning flight strategy corresponding with described visual pattern described unmanned plane is navigated.
4. as claimed in claim 3ly it is characterized in that based on the flight path of unmanned plane and the mapping air navigation aid of visual information that the corresponding flight strategy of the visual pattern that matching degree is the highest navigates to described unmanned plane as final flight strategy.
5. as claimed in claim 1 based on the flight path of unmanned plane and the mapping air navigation aid of visual information, it is characterized in that described semantic text information comprises: the shape of object in the described visual pattern, size, motion state, with the relative distance of described unmanned plane and the status information of background context.
6. as claimed in claim 1 based on the flight path of unmanned plane and the mapping air navigation aid of visual information; it is characterized in that; storage unmanned plane during flying flight path and corresponding visual information in the described mapping relations database; and described unmanned plane during flying flight path and corresponding visual information are classified according to the feature of flight path or visual information, the time sequencing that flight path takes place, and unmanned plane is retrieved according to different types of data in retrieving.
7. as claimed in claim 1 based on the flight path of unmanned plane and the mapping air navigation aid of visual information, it is characterized in that, described unmanned plane positions with corresponding visual information according to the flight track of historical flight track with corresponding visual information, current time, and the described visual information according to current acquisition is shone upon study, realizes flight path prediction and the path planning of described unmanned plane.
8. as claimed in claim 1ly it is characterized in that based on the flight path of unmanned plane and the mapping air navigation aid of visual information described mapping mode of learning further comprises:
According to feature recognition result and/or semantic relevant explanation, retrieval and described relevant flight path and visual information like the current data class that obtains from the mapping relations database, and according to shining upon study with the highest flight path and the visual information of described feature recognition result and/or semantic relevant explanation matching degree.
9. as claimed in claim 1 based on the flight path of unmanned plane and the mapping air navigation aid of visual information, it is characterized in that, when described unmanned plane has map maybe can obtain map, generate flight path information by described map, unmanned plane can directly carry out navigation flight according to the trace information that is generated by map, for the prevention environment changes, the trace information that is generated by map can be complementary to merge to generate and hang down error flight path information described unmanned plane is navigated with the prediction flight path information that merges mapping study according to flight path and visual information.
10. described based on the flight path of unmanned plane and the mapping air navigation aid of visual information as claim 6 to 9, it is characterized in that described visual information comprises: characteristic information and semantic text information.
11. one kind based on the flight path of unmanned plane and the mapping navigational system of visual information, it is characterized in that, comprising:
Acquisition module is for the visual pattern of gathering unmanned plane;
Analysis module is used for analyzing the semantic text information that described visual pattern reaches with the characteristic information that obtains described visual pattern and the characteristic information of described visual pattern mates;
Determination module is used for the flight path of determining described unmanned plane according to characteristic information and the semantic text information of visual pattern;
Generation module, be used for generating the mapping relations database according to described visual pattern and corresponding flight path, wherein, described mapping relations database can comprise that the flight path information of described unmanned plane in different visual patterns, attitude, speed, strategy and corresponding visual pattern are in order to retrieval study and/or transmission, the mapping relations database data can generate in real time according to image understanding system, also can be written in advance; And
Navigation module is used for according to described mapping relations database described unmanned plane being navigated.
12. it is as claimed in claim 11 based on the flight path of unmanned plane and the mapping navigational system of visual information, it is characterized in that, when described navigation module also is used for the visual pattern gathered when described unmanned plane and described mapping relations database visual pattern and matches, adopt the flight strategy corresponding with visual pattern in the described mapping relations database that described unmanned plane is navigated.
13. as claimed in claim 11ly it is characterized in that based on the flight path of unmanned plane and the mapping navigational system of visual information, also comprise:
The study navigation module, the matching degree of visual pattern that is used for visual pattern that described unmanned plane gathers and described mapping relations database is navigated to described unmanned plane by the mapping mode of learning flight strategy corresponding with described visual pattern during greater than threshold value.
14. it is as claimed in claim 13 based on the flight path of unmanned plane and the mapping navigational system of visual information, it is characterized in that described study navigation module also is used for the corresponding flight strategy of the visual pattern that matching degree is the highest and as final flight strategy described unmanned plane is navigated.
15. it is as claimed in claim 11 based on the flight path of unmanned plane and the mapping navigational system of visual information, it is characterized in that described semantic text information comprises: the shape of object in the described visual pattern, size, motion state, with the relative distance of described unmanned plane and the status information of background context.
CN201310153828.6A 2013-04-27 2013-04-27 Based on the flight path of unmanned plane and the mapping navigation method and system of visual information Active CN103196430B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310153828.6A CN103196430B (en) 2013-04-27 2013-04-27 Based on the flight path of unmanned plane and the mapping navigation method and system of visual information

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310153828.6A CN103196430B (en) 2013-04-27 2013-04-27 Based on the flight path of unmanned plane and the mapping navigation method and system of visual information

Publications (2)

Publication Number Publication Date
CN103196430A true CN103196430A (en) 2013-07-10
CN103196430B CN103196430B (en) 2015-12-09

Family

ID=48719134

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310153828.6A Active CN103196430B (en) 2013-04-27 2013-04-27 Based on the flight path of unmanned plane and the mapping navigation method and system of visual information

Country Status (1)

Country Link
CN (1) CN103196430B (en)

Cited By (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103941748A (en) * 2014-04-29 2014-07-23 百度在线网络技术(北京)有限公司 Autonomous navigation method and system and map modeling method and system
CN105955291A (en) * 2016-04-29 2016-09-21 深圳市哈博森科技有限公司 Unmanned plane flight route track recording and automatic flight control mode
CN106297237A (en) * 2016-08-17 2017-01-04 联想(北京)有限公司 A kind of control method and electronic equipment
CN106355866A (en) * 2016-11-14 2017-01-25 徐志勇 Unmanned aerial vehicle detection image storage analytic system
CN106647807A (en) * 2016-12-29 2017-05-10 上海资誉电子科技有限公司 Method and system of coping strategies generation of unmanned aerial vehicle
CN107278262A (en) * 2016-11-14 2017-10-20 深圳市大疆创新科技有限公司 Generation method, control device and the unmanned vehicle of flight path
CN107450593A (en) * 2017-08-30 2017-12-08 清华大学 A kind of unmanned plane autonomous navigation method and system
CN107444665A (en) * 2017-07-24 2017-12-08 长春草莓科技有限公司 A kind of unmanned plane Autonomous landing method
CN107993308A (en) * 2017-09-08 2018-05-04 北京航空航天大学 A kind of stand alone type unmanned plane during flying safety monitoring and information management system
CN108088438A (en) * 2017-12-05 2018-05-29 普达迪泰(天津)智能装备科技有限公司 A kind of unmanned plane vision navigation test method and system
CN108496129A (en) * 2017-04-28 2018-09-04 深圳市大疆创新科技有限公司 A kind of facility detection method and control device based on aircraft
CN108513643A (en) * 2017-08-31 2018-09-07 深圳市大疆创新科技有限公司 A kind of paths planning method, aircraft, flight system
CN109064467A (en) * 2018-08-20 2018-12-21 贵州宜行智通科技有限公司 Analysis method, device and the electronic equipment of community security defence
US10168696B2 (en) 2016-03-31 2019-01-01 International Business Machines Corporation Dynamic analysis of real-time restrictions for remote controlled vehicles
CN109275027A (en) * 2018-09-26 2019-01-25 Tcl海外电子(惠州)有限公司 Speech output method, electronic playback devices and the storage medium of video
WO2019043704A1 (en) * 2017-08-29 2019-03-07 Wajnberg Adam Drone escort system
CN109765922A (en) * 2019-03-07 2019-05-17 安徽省川佰科技有限公司 A kind of unmanned plane during flying track auxiliary method of adjustment
CN109799838A (en) * 2018-12-21 2019-05-24 金季春 A kind of training method and system
CN110162083A (en) * 2018-02-12 2019-08-23 维布络有限公司 The method and system of three-dimensional structure inspection and maintenance task is executed using unmanned plane
CN110940320A (en) * 2019-07-19 2020-03-31 华北电力大学(保定) Open stock ground monitored control system based on unmanned aerial vehicle cruises
CN111209899A (en) * 2019-12-31 2020-05-29 科大讯飞股份有限公司 Rescue material delivery method, system, device and storage medium
CN111316066A (en) * 2017-09-13 2020-06-19 Wing航空有限责任公司 Backup navigation system for unmanned aerial vehicle
CN112668652A (en) * 2020-12-31 2021-04-16 哈尔滨工业大学 Method and system for identifying cluster array and motion trend in unmanned equipment confrontation
CN115248877A (en) * 2022-09-22 2022-10-28 中国电子科技集团公司第十五研究所 Multi-mode-based track text matching method
CN117452969A (en) * 2023-12-06 2024-01-26 南京瑞蓝世光电传感技术研究院有限公司 Unmanned aerial vehicle navigation method based on multi-mode data processing

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101619985A (en) * 2009-08-06 2010-01-06 上海交通大学 Service robot autonomous navigation method based on deformable topological map
CN102426019A (en) * 2011-08-25 2012-04-25 航天恒星科技有限公司 Unmanned aerial vehicle scene matching auxiliary navigation method and system
CN102436738A (en) * 2011-09-26 2012-05-02 同济大学 Traffic monitoring device based on unmanned aerial vehicle (UAV)
CN102853830A (en) * 2012-09-03 2013-01-02 东南大学 Robot vision navigation method based on general object recognition

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101619985A (en) * 2009-08-06 2010-01-06 上海交通大学 Service robot autonomous navigation method based on deformable topological map
CN102426019A (en) * 2011-08-25 2012-04-25 航天恒星科技有限公司 Unmanned aerial vehicle scene matching auxiliary navigation method and system
CN102436738A (en) * 2011-09-26 2012-05-02 同济大学 Traffic monitoring device based on unmanned aerial vehicle (UAV)
CN102853830A (en) * 2012-09-03 2013-01-02 东南大学 Robot vision navigation method based on general object recognition

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
吴显亮等: "无人机视觉导航研究综述", 《***仿真学报》, vol. 22, 28 February 2010 (2010-02-28) *

Cited By (45)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9981742B2 (en) 2014-04-29 2018-05-29 Baidu Online Network Technology (Beijing) Co., Ltd. Autonomous navigation method and system, and map modeling method and system
WO2015165266A1 (en) * 2014-04-29 2015-11-05 百度在线网络技术(北京)有限公司 Autonomous navigation method and system, and map modeling method and system
CN103941748B (en) * 2014-04-29 2016-05-25 百度在线网络技术(北京)有限公司 Autonomous navigation method and system and Map building method and system
CN103941748A (en) * 2014-04-29 2014-07-23 百度在线网络技术(北京)有限公司 Autonomous navigation method and system and map modeling method and system
US10606258B2 (en) 2016-03-31 2020-03-31 International Business Machines Corporation Dynamic analysis of real-time restrictions for remote controlled vehicles
US10168696B2 (en) 2016-03-31 2019-01-01 International Business Machines Corporation Dynamic analysis of real-time restrictions for remote controlled vehicles
CN105955291A (en) * 2016-04-29 2016-09-21 深圳市哈博森科技有限公司 Unmanned plane flight route track recording and automatic flight control mode
CN106297237B (en) * 2016-08-17 2021-07-16 联想(北京)有限公司 Control method and electronic equipment
CN106297237A (en) * 2016-08-17 2017-01-04 联想(北京)有限公司 A kind of control method and electronic equipment
CN113074733A (en) * 2016-11-14 2021-07-06 深圳市大疆创新科技有限公司 Flight trajectory generation method, control device and unmanned aerial vehicle
CN107278262A (en) * 2016-11-14 2017-10-20 深圳市大疆创新科技有限公司 Generation method, control device and the unmanned vehicle of flight path
CN106355866A (en) * 2016-11-14 2017-01-25 徐志勇 Unmanned aerial vehicle detection image storage analytic system
CN106647807A (en) * 2016-12-29 2017-05-10 上海资誉电子科技有限公司 Method and system of coping strategies generation of unmanned aerial vehicle
CN113791641A (en) * 2017-04-28 2021-12-14 深圳市大疆创新科技有限公司 Aircraft-based facility detection method and control equipment
WO2018195955A1 (en) * 2017-04-28 2018-11-01 深圳市大疆创新科技有限公司 Aircraft-based facility detection method and control device
CN113791641B (en) * 2017-04-28 2024-06-18 深圳市大疆创新科技有限公司 Facility detection method and control equipment based on aircraft
CN108496129A (en) * 2017-04-28 2018-09-04 深圳市大疆创新科技有限公司 A kind of facility detection method and control device based on aircraft
CN108496129B (en) * 2017-04-28 2021-10-01 深圳市大疆创新科技有限公司 Aircraft-based facility detection method and control equipment
CN107444665A (en) * 2017-07-24 2017-12-08 长春草莓科技有限公司 A kind of unmanned plane Autonomous landing method
CN107444665B (en) * 2017-07-24 2020-06-09 长春草莓科技有限公司 Unmanned aerial vehicle autonomous landing method
IL272979A (en) * 2017-08-29 2020-04-30 Wajnberg Adam Drone escort system
IL272979B2 (en) * 2017-08-29 2023-05-01 Wajnberg Adam Drone escort system
WO2019043704A1 (en) * 2017-08-29 2019-03-07 Wajnberg Adam Drone escort system
US11513233B2 (en) 2017-08-29 2022-11-29 Adam Wajnberg et al. Drone escort system
CN107450593A (en) * 2017-08-30 2017-12-08 清华大学 A kind of unmanned plane autonomous navigation method and system
CN107450593B (en) * 2017-08-30 2020-06-12 清华大学 Unmanned aerial vehicle autonomous navigation method and system
CN108513643A (en) * 2017-08-31 2018-09-07 深圳市大疆创新科技有限公司 A kind of paths planning method, aircraft, flight system
CN107993308A (en) * 2017-09-08 2018-05-04 北京航空航天大学 A kind of stand alone type unmanned plane during flying safety monitoring and information management system
CN111316066B (en) * 2017-09-13 2024-03-15 Wing航空有限责任公司 Standby navigation system for unmanned aerial vehicle
CN111316066A (en) * 2017-09-13 2020-06-19 Wing航空有限责任公司 Backup navigation system for unmanned aerial vehicle
CN108088438B (en) * 2017-12-05 2021-05-14 普达迪泰(天津)智能装备科技有限公司 Unmanned aerial vehicle visual navigation test method and system
CN108088438A (en) * 2017-12-05 2018-05-29 普达迪泰(天津)智能装备科技有限公司 A kind of unmanned plane vision navigation test method and system
CN110162083A (en) * 2018-02-12 2019-08-23 维布络有限公司 The method and system of three-dimensional structure inspection and maintenance task is executed using unmanned plane
CN109064467A (en) * 2018-08-20 2018-12-21 贵州宜行智通科技有限公司 Analysis method, device and the electronic equipment of community security defence
CN109275027A (en) * 2018-09-26 2019-01-25 Tcl海外电子(惠州)有限公司 Speech output method, electronic playback devices and the storage medium of video
CN109799838A (en) * 2018-12-21 2019-05-24 金季春 A kind of training method and system
CN109765922A (en) * 2019-03-07 2019-05-17 安徽省川佰科技有限公司 A kind of unmanned plane during flying track auxiliary method of adjustment
CN110940320A (en) * 2019-07-19 2020-03-31 华北电力大学(保定) Open stock ground monitored control system based on unmanned aerial vehicle cruises
CN111209899B (en) * 2019-12-31 2023-06-02 科大讯飞股份有限公司 Rescue material delivery method, system, device and storage medium
CN111209899A (en) * 2019-12-31 2020-05-29 科大讯飞股份有限公司 Rescue material delivery method, system, device and storage medium
CN112668652A (en) * 2020-12-31 2021-04-16 哈尔滨工业大学 Method and system for identifying cluster array and motion trend in unmanned equipment confrontation
CN115248877B (en) * 2022-09-22 2023-01-17 中国电子科技集团公司第十五研究所 Multi-mode-based track text matching method
CN115248877A (en) * 2022-09-22 2022-10-28 中国电子科技集团公司第十五研究所 Multi-mode-based track text matching method
CN117452969A (en) * 2023-12-06 2024-01-26 南京瑞蓝世光电传感技术研究院有限公司 Unmanned aerial vehicle navigation method based on multi-mode data processing
CN117452969B (en) * 2023-12-06 2024-07-02 西藏创博通航科技有限公司 Unmanned aerial vehicle navigation method based on multi-mode data processing

Also Published As

Publication number Publication date
CN103196430B (en) 2015-12-09

Similar Documents

Publication Publication Date Title
CN103196430B (en) Based on the flight path of unmanned plane and the mapping navigation method and system of visual information
US9911340B2 (en) Real-time system for multi-modal 3D geospatial mapping, object recognition, scene annotation and analytics
CN106767812B (en) A kind of indoor semantic map updating method and system based on Semantic features extraction
Cieslewski et al. Point cloud descriptors for place recognition using sparse visual information
Emmi et al. A hybrid representation of the environment to improve autonomous navigation of mobile robots in agriculture
Li et al. Street tree segmentation from mobile laser scanning data
CN103278170A (en) Mobile robot cascading map building method based on remarkable scenic spot detection
CN106228539A (en) Multiple geometric primitive automatic identifying method in a kind of three-dimensional point cloud
JP2023036796A (en) Positioning method and positioning device by lane line and characteristic point, electronic apparatus, storage medium, computer program, and autonomous driving vehicle
CN112101527B (en) Method and device for identifying lane change, electronic equipment and storage medium
CN115203352B (en) Lane level positioning method and device, computer equipment and storage medium
KR20200028210A (en) System for structuring observation data and platform for mobile mapping or autonomous vehicle
CN110986945A (en) Local navigation method and system based on semantic height map
CN115147790A (en) Vehicle future trajectory prediction method based on graph neural network
CN104463909A (en) Visual target tracking method based on credibility combination map model
Li et al. An efficient point cloud place recognition approach based on transformer in dynamic environment
CN112733971B (en) Pose determination method, device and equipment of scanning equipment and storage medium
Regier et al. Classifying obstacles and exploiting knowledge about classes for efficient humanoid navigation
Karaoğuz et al. An integrated model of autonomous topological spatial cognition
CN115311867B (en) Tunnel scene positioning method and device, computer equipment and storage medium
CN112785564A (en) Pedestrian detection tracking system and method based on mechanical arm
Wen et al. Efficient indoor positioning with visual experiences via lifelong learning
Garforth et al. Lost in the woods? Place recognition for navigation in difficult forest environments
CN112380314B (en) Road network information processing method and device, storage medium and electronic equipment
Prabhu et al. UAVs for forestry: Metric-semantic mapping and diameter estimation with autonomous aerial robots

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant