CN108225273A - A kind of real-time runway detection method based on sensor priori - Google Patents
A kind of real-time runway detection method based on sensor priori Download PDFInfo
- Publication number
- CN108225273A CN108225273A CN201611153162.4A CN201611153162A CN108225273A CN 108225273 A CN108225273 A CN 108225273A CN 201611153162 A CN201611153162 A CN 201611153162A CN 108225273 A CN108225273 A CN 108225273A
- Authority
- CN
- China
- Prior art keywords
- runway
- image
- search
- template
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/182—Network patterns, e.g. roads or rivers
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to embedded computer technical field of image processing, more particularly to a kind of real-time runway detection method based on sensor priori.Existing runway detection algorithm mainly for satellite image and it is lower regard high-altitude Aerial Images, be not inconsistent, and be detected generally be directed to still image with the forward sight Aerial Images needed for airborne synthetic vision system, operation time is longer, it is impossible to meet requirement of real-time.Thus, this invention proposes a kind of real-time runway detection algorithms based on sensor priori, the information such as flight position that this method provides airborne sensor and posture are also included in calculating, so that the region of search of runway detection reduces significantly, so as to quick and precisely detect the post position in image.
Description
Technical field
The invention belongs to embedded computer technical field of image processing, are known more particularly to one kind based on sensor priori
The real-time runway detection method known.
Background technology
With the development of airborne avionics technology, be born many flight assisting systems.Wherein enhancing visual system can carry
Perception of the pilot to external environment is risen, enhances flight safety.Runway detection technology plays important in visual system is enhanced
Effect.But existing runway detection method mainly for satellite image and it is lower regard high-altitude Aerial Images, with airborne Synthetic vision
Forward sight Aerial Images needed for system are not inconsistent, and be detected generally be directed to still image, and operation time is longer, it is impossible to meet
Requirement of real-time.This just needs a kind of runway detection method of real-time high-efficiency, can accurately quickly detect out in realtime graphic
Runway.
The present invention provides a kind of real-time runway detection methods based on sensor priori to solve the above problems.
Invention content
The purpose of the present invention is:
Solve the problems, such as the runway detection in airborne realtime graphic.
The present invention technical solution be:
Existing runway detection method, majority is using runway as one kind of target, using general target identification side
Method, major defect are:1) computation complexity is higher.The processes such as the lookup and matching of algorithm are carried out for entire image, fortune
It is higher to calculate complexity;2) method based on template, it is low and inaccurate that template obtains efficiency.The extraction of template relies on artificial mark mostly
Fixed and template library, the former efficiency is low and the latter is not accurate enough.
But in mobile system, there are many prioris, such as camera posture, camera position, imaging parameters, these letters
Breath can instruct us to generate the shape of template of runway, and locating template position in the picture, so as to reduce search range, reduce
Operand.
In visual system is enhanced, there is different types of sensor, the flight including the information such as navigation attitude, position is provided
Information.Wherein, aspect sensor can provide the pitching, rolling and yaw angle of aircraft, and GPS can then provide aircraft and exist
Coordinate in three-dimensional world.In addition, it is also solid that shooting image opportunity, which carries the parameter of video camera and the coordinate of airfield runway angle point,
It is fixed and known.
Described in summary, the algorithm first step that this patent proposes can calculate the mould of airfield runway ideally
Plate shape and position, and then determine a search range, reduce the calculation amount of runway detection.Specific calculating process is as follows:
A kind of real-time runway detection method based on sensor priori, this method include the following steps:
The first step:It is assumed that the runway angle point P in world coordinate systemW=(xW, yW, zW)T, require transformation into camera coordinates system
In a point PC=(xC, yC, zC)T, then transfer process needs primary rotationAdd primary translation TW, obtain camera coordinates system
In corner location it is as follows:
Wherein TWIt can be directly obtained from GPS data:
TW=(xW, yW, zW)T(4-10)
And spin matrixCalculating process it is then slightly more complex;Pass through three attitude angles of aircraft:Rolling φ, pitching θ
With yaw ψ, by rotating three times, primary complete rotary course may finally be completed;So spin matrixIt is by three parts
Obtained from spin matrix is multiplied:
For simplicity the cx=cos (x) in formula, sx=sin (x);
Second step:Angle point is goed off the course final according to the image-forming principle of camera and the inside and outside parameter of shooting camera, calculating
Coordinate position on image;The equivalent geometrical relationship of specific imaging calculating process;
A point P in objective plane is projected in the P ' points of imaging plane, is finally converged in focal point and other points;According to
The P that one step calculatesC=(xC, yC, zC)T, obtained by the similarity relation of triangle:
Wherein, ximgAnd yimgIt is coordinate of the target point on imaging plane, PsIt is the pixel length of side of imaging plane, H0And W0
It is the pixel number on imaging plane height and width direction respectively;Thus x is solvedimgAnd yimg;
Third walks:According to the coordinate position of runway angle point on the image, the shape for the template that determines to go off the course and position, and with
This delimit region of search;Specific calculating process is as follows:
The coordinate of four angle points of airfield runway on the final is calculated respectively, is sequentially connected with, has obtained preferable shape
Airfield runway template under state;In practical applications, have by aspect sensor and the GPS data obtained certain
Error except the airfield runway template position calculated, delimit a block search region, and controlled with two parameter HE and WE
The size of region of search, HE and WE take 1/2 length of runway template height and width respectively;
4th step:In region of search, lines detection is carried out to image, obtains the more obvious rectilinear of feature;Specifically
Lines detection is carried out using LSD algorithm, key step has:1) image gradient calculates;2) gradient puppet sorts;3) increasing of gradient neighborhood
Long or merging;4) fitting a straight line;
5th step:Using the rectilinear of generation and runway template, the matching of runway is carried out in region of search, it is final to calculate
It goes off the course position;The specific method for using directionality Chamfer Matching carries out the matching of runway, meter in region of search
Calculating formula is:
Wherein U={ uiBe template edge line image, V={ vjBe image to be detected edge line image, U and V
Between directionality Chamfer distances dDCM(U, V) is defined as the average distance of every bit and edge nearest in V in U, φ (x)
The edge direction at marginal point x is represented, λ is the weight parameter between location entries and direction;dDCMThe position of (U, V) minimum
As final search result.
Present invention has the advantage that:
This algorithm is directed to aircraft closely into the forward sight Aerial Images of landing period, convenient for applying in airborne enhancing visual system
In.The priori provided using airborne sensor of this algorithm can prejudge go off the course shape of template and runway in airborne reality
When image in position, so as to reduce region of search, only in region of search carry out runway matching, so as to substantially increase inspection
The accuracy of survey, reduces operand, has achieved the effect that accurate detection is goed off the course in real time.
Description of the drawings
Fig. 1 is the principle of the present invention block diagram, and algorithm is divided into five steps.
Fig. 2 is the camera imaging principle schematic simplified in the present invention.
Fig. 3 is the schematic diagram for delimiting region of search in the present invention according to runway template.
Specific embodiment
In airborne enhancing synthetic vision system, to need to detect the runway information in forward sight visible light video in real time
Identification.By taking the system as an example, the method described in concrete application patent.Have in airborne enhancing synthetic vision system and regard all the way
Frequency inputs, and video exports all the way, as follows to the specific runway detection step of every frame input picture:
The first step:It is assumed that the runway angle point P in world coordinate systemW=(xW, yW, zW)T, require transformation into camera coordinates system
In a point PC=(xC, yC, zC)T, then transfer process needs primary rotationAdd primary translation TW, obtain camera coordinates system
In corner location it is as follows:
Wherein TWIt can be directly obtained from GPS data:
TW=(xW, yW, zW)T(4-10)
And spin matrixCalculating process it is then slightly more complex;Pass through three attitude angles of aircraft:Rolling φ, pitching θ
With yaw ψ, by rotating three times, primary complete rotary course may finally be completed;So spin matrixIt is by three parts
Obtained from spin matrix is multiplied:
For simplicity the cx=cos (x) in formula, sx=sin (x);
Second step:Angle point is goed off the course final according to the image-forming principle of camera and the inside and outside parameter of shooting camera, calculating
Coordinate position on image;The equivalent geometrical relationship of specific imaging calculating process;
A point P in objective plane is projected in the P ' points of imaging plane, is finally converged in focal point and other points;According to
The P that one step calculatesC=(xC, yC, zC)T, obtained by the similarity relation of triangle:
Wherein, ximgAnd yimgIt is coordinate of the target point on imaging plane, PsIt is the pixel length of side of imaging plane, H0And W0
It is the pixel number on imaging plane height and width direction respectively;Thus x is solvedimgAnd yimg;
Third walks:According to the coordinate position of runway angle point on the image, the shape for the template that determines to go off the course and position, and with
This delimit region of search;Specific calculating process is as follows:
The coordinate of four angle points of airfield runway on the final is calculated respectively, is sequentially connected with, has obtained preferable shape
Airfield runway template under state;In practical applications, have by aspect sensor and the GPS data obtained certain
Error except the airfield runway template position calculated, delimit a block search region, and controlled with two parameter HE and WE
The size of region of search, HE and WE take 1/2 length of runway template height and width respectively;
4th step:In region of search, lines detection is carried out to image, obtains the more obvious rectilinear of feature;Specifically
Lines detection is carried out using LSD algorithm, key step has:1) image gradient calculates;2) gradient puppet sorts;3) increasing of gradient neighborhood
Long or merging;4) fitting a straight line;
5th step:Using the rectilinear of generation and runway template, the matching of runway is carried out in region of search, it is final to calculate
It goes off the course position;The specific method for using directionality Chamfer Matching carries out the matching of runway, meter in region of search
Calculating formula is:
Wherein U={ uiBe template edge line image, V={ vjBe image to be detected edge line image, U and V
Between directionality Chamfer distances dDCM(U, V) is defined as the average distance of every bit and edge nearest in V in U, φ (x)
The edge direction at marginal point x is represented, λ is the weight parameter between location entries and direction;dDCMThe position of (U, V) minimum
As final search result;And marked in the corresponding position of input picture, it is exported as output image.
Claims (1)
1. a kind of real-time runway detection method based on sensor priori, this method include the following steps:
The first step:It is assumed that the runway angle point P in world coordinate systemW=(xW, yW, zW)T, require transformation into camera coordinates system
One point PC=(xC, yC, zC)T, then transfer process needs primary rotationAdd primary translation TW, obtain in camera coordinates system
Corner location is as follows:
Wherein TWIt can be directly obtained from GPS data:
TW=(xW, yW, zW)T(4-10)
And spin matrixCalculating process it is then slightly more complex;Pass through three attitude angles of aircraft:Rolling φ, pitching θ and partially
Navigate ψ, by rotating three times, may finally complete primary complete rotary course;So spin matrixIt is to be rotated by three parts
Obtained from matrix multiple:
For simplicity the cx=cos (x) in formula, sx=sin (x);
Second step:Angle point is goed off the course in final image according to the image-forming principle of camera and the inside and outside parameter of shooting camera, calculating
On coordinate position;The equivalent geometrical relationship of specific imaging calculating process;
A point P in objective plane is projected in the P ' points of imaging plane, is finally converged in focal point and other points;According to the first step
The P calculatedC=(xC, yC, zC)T, obtained by the similarity relation of triangle:
Wherein, ximgAnd yimgIt is coordinate of the target point on imaging plane, PsIt is the pixel length of side of imaging plane, H0And W0Respectively
It is the pixel number on imaging plane height and width direction;Thus x is solvedimgAnd yimg;
Third walks:According to the coordinate position of runway angle point on the image, the shape for the template that determines to go off the course and position, and with this stroke
Determine region of search;Specific calculating process is as follows:
The coordinate of four angle points of airfield runway on the final is calculated respectively, is sequentially connected with, obtained ideally
Airfield runway template;In practical applications, there is certain mistake by aspect sensor and the GPS data obtained
Difference except the airfield runway template position calculated, delimit a block search region, and searched to control with two parameters HE and WE
The size in rope region, HE and WE take 1/2 length of runway template height and width respectively;
4th step:In region of search, lines detection is carried out to image, obtains the more obvious rectilinear of feature;It is specific to use
LSD algorithm carries out lines detection, and key step has:1) image gradient calculates;2) gradient puppet sorts;3) growth of gradient neighborhood or
Merge;4) fitting a straight line;
5th step:Using the rectilinear of generation and runway template, the matching of runway is carried out in region of search, finally calculates race
Road position;The specific method for using directionality Chamfer Matching, carries out the matching of runway in region of search, calculates public
Formula is:
Wherein U={ uiBe template edge line image, V={ vjBe image to be detected edge line image, between U and V
Directionality Chamfer distances dDCM(U, V) is defined as the average distance of every bit and edge nearest in V in U, and φ (x) is represented
Edge direction at marginal point x, λ are the weight parameters between location entries and direction;dDCMThe position of (U, V) minimum is
Final search result.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201611153162.4A CN108225273B (en) | 2016-12-14 | 2016-12-14 | Real-time runway detection method based on sensor priori knowledge |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201611153162.4A CN108225273B (en) | 2016-12-14 | 2016-12-14 | Real-time runway detection method based on sensor priori knowledge |
Publications (2)
Publication Number | Publication Date |
---|---|
CN108225273A true CN108225273A (en) | 2018-06-29 |
CN108225273B CN108225273B (en) | 2020-06-30 |
Family
ID=62639019
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201611153162.4A Active CN108225273B (en) | 2016-12-14 | 2016-12-14 | Real-time runway detection method based on sensor priori knowledge |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN108225273B (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109238265A (en) * | 2018-07-20 | 2019-01-18 | 民航中南空管设备工程公司 | A kind of airfield runway location measurement method |
CN109341685A (en) * | 2018-12-04 | 2019-02-15 | 中国航空工业集团公司西安航空计算技术研究所 | A kind of fixed wing aircraft vision auxiliary landing navigation method based on homograph |
CN109341700A (en) * | 2018-12-04 | 2019-02-15 | 中国航空工业集团公司西安航空计算技术研究所 | Fixed wing aircraft vision assists landing navigation method under a kind of low visibility |
CN110274622A (en) * | 2019-06-13 | 2019-09-24 | 北京环奥体育发展有限公司 | A kind of in-site measurement, when calculating marathon race racing track complexity calculation method |
CN113295164A (en) * | 2021-04-23 | 2021-08-24 | 四川腾盾科技有限公司 | Unmanned aerial vehicle visual positioning method and device based on airport runway |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101777181A (en) * | 2010-01-15 | 2010-07-14 | 西安电子科技大学 | Ridgelet bi-frame system-based SAR image airfield runway extraction method |
WO2013051967A2 (en) * | 2011-08-31 | 2013-04-11 | Kirillov Andrey Porfir Evich | Method for visual landing and kirillov device for visualizing takeoff or landing of an aircraft |
US20130120585A1 (en) * | 2011-11-15 | 2013-05-16 | Canon Kabushiki Kaisha | Automatic tracking camera system |
CN103162669A (en) * | 2013-03-01 | 2013-06-19 | 西北工业大学 | Detection method of airport area through aerial shooting image |
CN103577697A (en) * | 2013-11-12 | 2014-02-12 | 中国民用航空总局第二研究所 | FOD detection method based on road surface point cloud data |
CN105302146A (en) * | 2014-07-25 | 2016-02-03 | 空中客车运营简化股份公司 | Method and system for automatic autonomous landing of an aircraft |
-
2016
- 2016-12-14 CN CN201611153162.4A patent/CN108225273B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101777181A (en) * | 2010-01-15 | 2010-07-14 | 西安电子科技大学 | Ridgelet bi-frame system-based SAR image airfield runway extraction method |
WO2013051967A2 (en) * | 2011-08-31 | 2013-04-11 | Kirillov Andrey Porfir Evich | Method for visual landing and kirillov device for visualizing takeoff or landing of an aircraft |
US20130120585A1 (en) * | 2011-11-15 | 2013-05-16 | Canon Kabushiki Kaisha | Automatic tracking camera system |
CN103162669A (en) * | 2013-03-01 | 2013-06-19 | 西北工业大学 | Detection method of airport area through aerial shooting image |
CN103577697A (en) * | 2013-11-12 | 2014-02-12 | 中国民用航空总局第二研究所 | FOD detection method based on road surface point cloud data |
CN105302146A (en) * | 2014-07-25 | 2016-02-03 | 空中客车运营简化股份公司 | Method and system for automatic autonomous landing of an aircraft |
Non-Patent Citations (3)
Title |
---|
ERIC FREW ETC.: "Vision-Based Road-Following Using a Small Autonomous Aircraft", 《2004 IEEE AEROSPACE CONFERENCE PROCEEDINGS》 * |
J. SCHONEFELD ETC.: "Runway incursion prevention systems: A review of runway incursion avoidance and alerting system approaches", 《PROGRESS IN AEROSPACE SCIENCES》 * |
邸男 等: "提取直线特征实现机场跑道实时检测", 《光学精密工程》 * |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109238265A (en) * | 2018-07-20 | 2019-01-18 | 民航中南空管设备工程公司 | A kind of airfield runway location measurement method |
CN109238265B (en) * | 2018-07-20 | 2020-08-11 | 民航中南空管设备工程公司 | Airport runway position measuring method |
CN109341685A (en) * | 2018-12-04 | 2019-02-15 | 中国航空工业集团公司西安航空计算技术研究所 | A kind of fixed wing aircraft vision auxiliary landing navigation method based on homograph |
CN109341700A (en) * | 2018-12-04 | 2019-02-15 | 中国航空工业集团公司西安航空计算技术研究所 | Fixed wing aircraft vision assists landing navigation method under a kind of low visibility |
CN110274622A (en) * | 2019-06-13 | 2019-09-24 | 北京环奥体育发展有限公司 | A kind of in-site measurement, when calculating marathon race racing track complexity calculation method |
CN110274622B (en) * | 2019-06-13 | 2021-08-27 | 北京环奥体育发展有限公司 | Method for calculating difficulty degree of track during marathon competition by field measurement and calculation |
CN113295164A (en) * | 2021-04-23 | 2021-08-24 | 四川腾盾科技有限公司 | Unmanned aerial vehicle visual positioning method and device based on airport runway |
Also Published As
Publication number | Publication date |
---|---|
CN108225273B (en) | 2020-06-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10515458B1 (en) | Image-matching navigation method and apparatus for aerial vehicles | |
EP3407294B1 (en) | Information processing method, device, and terminal | |
CN101598556B (en) | Unmanned aerial vehicle vision/inertia integrated navigation method in unknown environment | |
CN108225273A (en) | A kind of real-time runway detection method based on sensor priori | |
US20200357141A1 (en) | Systems and methods for calibrating an optical system of a movable object | |
Liu et al. | An off-board vision system for relative attitude measurement of aircraft | |
Ma et al. | A novel autonomous aerial refueling drogue detection and pose estimation method based on monocular vision | |
CN109341686A (en) | A kind of tightly coupled aircraft lands position and orientation estimation method of view-based access control model-inertia | |
CN109978954A (en) | The method and apparatus of radar and camera combined calibrating based on cabinet | |
Vetrella et al. | RGB-D camera-based quadrotor navigation in GPS-denied and low light environments using known 3D markers | |
Zhao et al. | Homography-based camera pose estimation with known gravity direction for UAV navigation | |
Liu et al. | Runway extraction in low visibility conditions based on sensor fusion method | |
US20130093880A1 (en) | Height Measurement Apparatus And Method | |
Deng et al. | A binocular vision-based measuring system for UAVs autonomous aerial refueling | |
Steffes et al. | Optical terrain relative navigation approaches to lunar orbit, descent and landing | |
Angermann et al. | High precision approaches enabled by an optical-based navigation system | |
Burdziakowski | Towards precise visual navigation and direct georeferencing for MAV using ORB-SLAM2 | |
Kim et al. | Target detection and position likelihood using an aerial image sensor | |
Zhuge et al. | 3-D motion trajectory measurement for the target through a multi-UAVs system | |
Zhang et al. | Tracking and position of drogue for autonomous aerial refueling | |
Wang et al. | A real-time sensor guided runway detection method for forward-looking aerial images | |
Al-Kaff | Vision-based navigation system for unmanned aerial vehicles | |
Roozing et al. | Low-cost vision-based 6-DOF MAV localization using IR beacons | |
Li-Chee-Ming et al. | Determination of UAS trajectory in a known environment from FPV video | |
Hu et al. | Toward high-quality magnetic data survey using UAV: development of a magnetic-isolated vision-based positioning system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |