CN108225273A - A kind of real-time runway detection method based on sensor priori - Google Patents

A kind of real-time runway detection method based on sensor priori Download PDF

Info

Publication number
CN108225273A
CN108225273A CN201611153162.4A CN201611153162A CN108225273A CN 108225273 A CN108225273 A CN 108225273A CN 201611153162 A CN201611153162 A CN 201611153162A CN 108225273 A CN108225273 A CN 108225273A
Authority
CN
China
Prior art keywords
runway
image
search
template
region
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201611153162.4A
Other languages
Chinese (zh)
Other versions
CN108225273B (en
Inventor
程岳
李亚晖
谢建春
张磊
文鹏程
白林亭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian Aeronautics Computing Technique Research Institute of AVIC
Original Assignee
Xian Aeronautics Computing Technique Research Institute of AVIC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian Aeronautics Computing Technique Research Institute of AVIC filed Critical Xian Aeronautics Computing Technique Research Institute of AVIC
Priority to CN201611153162.4A priority Critical patent/CN108225273B/en
Publication of CN108225273A publication Critical patent/CN108225273A/en
Application granted granted Critical
Publication of CN108225273B publication Critical patent/CN108225273B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/182—Network patterns, e.g. roads or rivers

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The invention belongs to embedded computer technical field of image processing, more particularly to a kind of real-time runway detection method based on sensor priori.Existing runway detection algorithm mainly for satellite image and it is lower regard high-altitude Aerial Images, be not inconsistent, and be detected generally be directed to still image with the forward sight Aerial Images needed for airborne synthetic vision system, operation time is longer, it is impossible to meet requirement of real-time.Thus, this invention proposes a kind of real-time runway detection algorithms based on sensor priori, the information such as flight position that this method provides airborne sensor and posture are also included in calculating, so that the region of search of runway detection reduces significantly, so as to quick and precisely detect the post position in image.

Description

A kind of real-time runway detection method based on sensor priori
Technical field
The invention belongs to embedded computer technical field of image processing, are known more particularly to one kind based on sensor priori The real-time runway detection method known.
Background technology
With the development of airborne avionics technology, be born many flight assisting systems.Wherein enhancing visual system can carry Perception of the pilot to external environment is risen, enhances flight safety.Runway detection technology plays important in visual system is enhanced Effect.But existing runway detection method mainly for satellite image and it is lower regard high-altitude Aerial Images, with airborne Synthetic vision Forward sight Aerial Images needed for system are not inconsistent, and be detected generally be directed to still image, and operation time is longer, it is impossible to meet Requirement of real-time.This just needs a kind of runway detection method of real-time high-efficiency, can accurately quickly detect out in realtime graphic Runway.
The present invention provides a kind of real-time runway detection methods based on sensor priori to solve the above problems.
Invention content
The purpose of the present invention is：
Solve the problems, such as the runway detection in airborne realtime graphic.
The present invention technical solution be：
Existing runway detection method, majority is using runway as one kind of target, using general target identification side Method, major defect are：1) computation complexity is higher.The processes such as the lookup and matching of algorithm are carried out for entire image, fortune It is higher to calculate complexity；2) method based on template, it is low and inaccurate that template obtains efficiency.The extraction of template relies on artificial mark mostly Fixed and template library, the former efficiency is low and the latter is not accurate enough.
But in mobile system, there are many prioris, such as camera posture, camera position, imaging parameters, these letters Breath can instruct us to generate the shape of template of runway, and locating template position in the picture, so as to reduce search range, reduce Operand.
In visual system is enhanced, there is different types of sensor, the flight including the information such as navigation attitude, position is provided Information.Wherein, aspect sensor can provide the pitching, rolling and yaw angle of aircraft, and GPS can then provide aircraft and exist Coordinate in three-dimensional world.In addition, it is also solid that shooting image opportunity, which carries the parameter of video camera and the coordinate of airfield runway angle point, It is fixed and known.
Described in summary, the algorithm first step that this patent proposes can calculate the mould of airfield runway ideally Plate shape and position, and then determine a search range, reduce the calculation amount of runway detection.Specific calculating process is as follows：
A kind of real-time runway detection method based on sensor priori, this method include the following steps：
The first step：It is assumed that the runway angle point P in world coordinate systemW=(xW, yW, zW)T, require transformation into camera coordinates system In a point PC=(xC, yC, zC)T, then transfer process needs primary rotationAdd primary translation TW, obtain camera coordinates system In corner location it is as follows：
Wherein TWIt can be directly obtained from GPS data：
TW=(xW, yW, zW)T(4-10)
And spin matrixCalculating process it is then slightly more complex；Pass through three attitude angles of aircraft：Rolling φ, pitching θ With yaw ψ, by rotating three times, primary complete rotary course may finally be completed；So spin matrixIt is by three parts Obtained from spin matrix is multiplied：
For simplicity the cx=cos (x) in formula, sx=sin (x)；
Second step：Angle point is goed off the course final according to the image-forming principle of camera and the inside and outside parameter of shooting camera, calculating Coordinate position on image；The equivalent geometrical relationship of specific imaging calculating process；
A point P in objective plane is projected in the P ' points of imaging plane, is finally converged in focal point and other points；According to The P that one step calculatesC=(xC, yC, zC)T, obtained by the similarity relation of triangle：
Wherein, ximgAnd yimgIt is coordinate of the target point on imaging plane, PsIt is the pixel length of side of imaging plane, H0And W0 It is the pixel number on imaging plane height and width direction respectively；Thus x is solvedimgAnd yimg；
Third walks：According to the coordinate position of runway angle point on the image, the shape for the template that determines to go off the course and position, and with This delimit region of search；Specific calculating process is as follows：
The coordinate of four angle points of airfield runway on the final is calculated respectively, is sequentially connected with, has obtained preferable shape Airfield runway template under state；In practical applications, have by aspect sensor and the GPS data obtained certain Error except the airfield runway template position calculated, delimit a block search region, and controlled with two parameter HE and WE The size of region of search, HE and WE take 1/2 length of runway template height and width respectively；
4th step：In region of search, lines detection is carried out to image, obtains the more obvious rectilinear of feature；Specifically Lines detection is carried out using LSD algorithm, key step has：1) image gradient calculates；2) gradient puppet sorts；3) increasing of gradient neighborhood Long or merging；4) fitting a straight line；
5th step：Using the rectilinear of generation and runway template, the matching of runway is carried out in region of search, it is final to calculate It goes off the course position；The specific method for using directionality Chamfer Matching carries out the matching of runway, meter in region of search Calculating formula is：
Wherein U={ uiBe template edge line image, V={ vjBe image to be detected edge line image, U and V Between directionality Chamfer distances dDCM(U, V) is defined as the average distance of every bit and edge nearest in V in U, φ (x) The edge direction at marginal point x is represented, λ is the weight parameter between location entries and direction；dDCMThe position of (U, V) minimum As final search result.
Present invention has the advantage that：
This algorithm is directed to aircraft closely into the forward sight Aerial Images of landing period, convenient for applying in airborne enhancing visual system In.The priori provided using airborne sensor of this algorithm can prejudge go off the course shape of template and runway in airborne reality When image in position, so as to reduce region of search, only in region of search carry out runway matching, so as to substantially increase inspection The accuracy of survey, reduces operand, has achieved the effect that accurate detection is goed off the course in real time.
Description of the drawings
Fig. 1 is the principle of the present invention block diagram, and algorithm is divided into five steps.
Fig. 2 is the camera imaging principle schematic simplified in the present invention.
Fig. 3 is the schematic diagram for delimiting region of search in the present invention according to runway template.
Specific embodiment
In airborne enhancing synthetic vision system, to need to detect the runway information in forward sight visible light video in real time Identification.By taking the system as an example, the method described in concrete application patent.Have in airborne enhancing synthetic vision system and regard all the way Frequency inputs, and video exports all the way, as follows to the specific runway detection step of every frame input picture：
The first step：It is assumed that the runway angle point P in world coordinate systemW=(xW, yW, zW)T, require transformation into camera coordinates system In a point PC=(xC, yC, zC)T, then transfer process needs primary rotationAdd primary translation TW, obtain camera coordinates system In corner location it is as follows：
Wherein TWIt can be directly obtained from GPS data：
TW=(xW, yW, zW)T(4-10)
And spin matrixCalculating process it is then slightly more complex；Pass through three attitude angles of aircraft：Rolling φ, pitching θ With yaw ψ, by rotating three times, primary complete rotary course may finally be completed；So spin matrixIt is by three parts Obtained from spin matrix is multiplied：
For simplicity the cx=cos (x) in formula, sx=sin (x)；
Second step：Angle point is goed off the course final according to the image-forming principle of camera and the inside and outside parameter of shooting camera, calculating Coordinate position on image；The equivalent geometrical relationship of specific imaging calculating process；
A point P in objective plane is projected in the P ' points of imaging plane, is finally converged in focal point and other points；According to The P that one step calculatesC=(xC, yC, zC)T, obtained by the similarity relation of triangle：
Wherein, ximgAnd yimgIt is coordinate of the target point on imaging plane, PsIt is the pixel length of side of imaging plane, H0And W0 It is the pixel number on imaging plane height and width direction respectively；Thus x is solvedimgAnd yimg；
Third walks：According to the coordinate position of runway angle point on the image, the shape for the template that determines to go off the course and position, and with This delimit region of search；Specific calculating process is as follows：
The coordinate of four angle points of airfield runway on the final is calculated respectively, is sequentially connected with, has obtained preferable shape Airfield runway template under state；In practical applications, have by aspect sensor and the GPS data obtained certain Error except the airfield runway template position calculated, delimit a block search region, and controlled with two parameter HE and WE The size of region of search, HE and WE take 1/2 length of runway template height and width respectively；
4th step：In region of search, lines detection is carried out to image, obtains the more obvious rectilinear of feature；Specifically Lines detection is carried out using LSD algorithm, key step has：1) image gradient calculates；2) gradient puppet sorts；3) increasing of gradient neighborhood Long or merging；4) fitting a straight line；
5th step：Using the rectilinear of generation and runway template, the matching of runway is carried out in region of search, it is final to calculate It goes off the course position；The specific method for using directionality Chamfer Matching carries out the matching of runway, meter in region of search Calculating formula is：
Wherein U={ uiBe template edge line image, V={ vjBe image to be detected edge line image, U and V Between directionality Chamfer distances dDCM(U, V) is defined as the average distance of every bit and edge nearest in V in U, φ (x) The edge direction at marginal point x is represented, λ is the weight parameter between location entries and direction；dDCMThe position of (U, V) minimum As final search result；And marked in the corresponding position of input picture, it is exported as output image.

Claims (1)

1. a kind of real-time runway detection method based on sensor priori, this method include the following steps：
The first step：It is assumed that the runway angle point P in world coordinate systemW=(xW, yW, zW)T, require transformation into camera coordinates system One point PC=(xC, yC, zC)T, then transfer process needs primary rotationAdd primary translation TW, obtain in camera coordinates system Corner location is as follows：
Wherein TWIt can be directly obtained from GPS data：
TW=(xW, yW, zW)T(4-10)
And spin matrixCalculating process it is then slightly more complex；Pass through three attitude angles of aircraft：Rolling φ, pitching θ and partially Navigate ψ, by rotating three times, may finally complete primary complete rotary course；So spin matrixIt is to be rotated by three parts Obtained from matrix multiple：
For simplicity the cx=cos (x) in formula, sx=sin (x)；
Second step：Angle point is goed off the course in final image according to the image-forming principle of camera and the inside and outside parameter of shooting camera, calculating On coordinate position；The equivalent geometrical relationship of specific imaging calculating process；
A point P in objective plane is projected in the P ' points of imaging plane, is finally converged in focal point and other points；According to the first step The P calculatedC=(xC, yC, zC)T, obtained by the similarity relation of triangle：
Wherein, ximgAnd yimgIt is coordinate of the target point on imaging plane, PsIt is the pixel length of side of imaging plane, H0And W0Respectively It is the pixel number on imaging plane height and width direction；Thus x is solvedimgAnd yimg；
Third walks：According to the coordinate position of runway angle point on the image, the shape for the template that determines to go off the course and position, and with this stroke Determine region of search；Specific calculating process is as follows：
The coordinate of four angle points of airfield runway on the final is calculated respectively, is sequentially connected with, obtained ideally Airfield runway template；In practical applications, there is certain mistake by aspect sensor and the GPS data obtained Difference except the airfield runway template position calculated, delimit a block search region, and searched to control with two parameters HE and WE The size in rope region, HE and WE take 1/2 length of runway template height and width respectively；
4th step：In region of search, lines detection is carried out to image, obtains the more obvious rectilinear of feature；It is specific to use LSD algorithm carries out lines detection, and key step has：1) image gradient calculates；2) gradient puppet sorts；3) growth of gradient neighborhood or Merge；4) fitting a straight line；
5th step：Using the rectilinear of generation and runway template, the matching of runway is carried out in region of search, finally calculates race Road position；The specific method for using directionality Chamfer Matching, carries out the matching of runway in region of search, calculates public Formula is：
Wherein U={ uiBe template edge line image, V={ vjBe image to be detected edge line image, between U and V Directionality Chamfer distances dDCM(U, V) is defined as the average distance of every bit and edge nearest in V in U, and φ (x) is represented Edge direction at marginal point x, λ are the weight parameters between location entries and direction；dDCMThe position of (U, V) minimum is Final search result.
CN201611153162.4A 2016-12-14 2016-12-14 Real-time runway detection method based on sensor priori knowledge Active CN108225273B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611153162.4A CN108225273B (en) 2016-12-14 2016-12-14 Real-time runway detection method based on sensor priori knowledge

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611153162.4A CN108225273B (en) 2016-12-14 2016-12-14 Real-time runway detection method based on sensor priori knowledge

Publications (2)

Publication Number Publication Date
CN108225273A true CN108225273A (en) 2018-06-29
CN108225273B CN108225273B (en) 2020-06-30

Family

ID=62639019

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611153162.4A Active CN108225273B (en) 2016-12-14 2016-12-14 Real-time runway detection method based on sensor priori knowledge

Country Status (1)

Country Link
CN (1) CN108225273B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109238265A (en) * 2018-07-20 2019-01-18 民航中南空管设备工程公司 A kind of airfield runway location measurement method
CN109341685A (en) * 2018-12-04 2019-02-15 中国航空工业集团公司西安航空计算技术研究所 A kind of fixed wing aircraft vision auxiliary landing navigation method based on homograph
CN109341700A (en) * 2018-12-04 2019-02-15 中国航空工业集团公司西安航空计算技术研究所 Fixed wing aircraft vision assists landing navigation method under a kind of low visibility
CN110274622A (en) * 2019-06-13 2019-09-24 北京环奥体育发展有限公司 A kind of in-site measurement, when calculating marathon race racing track complexity calculation method
CN113295164A (en) * 2021-04-23 2021-08-24 四川腾盾科技有限公司 Unmanned aerial vehicle visual positioning method and device based on airport runway

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101777181A (en) * 2010-01-15 2010-07-14 西安电子科技大学 Ridgelet bi-frame system-based SAR image airfield runway extraction method
WO2013051967A2 (en) * 2011-08-31 2013-04-11 Kirillov Andrey Porfir Evich Method for visual landing and kirillov device for visualizing takeoff or landing of an aircraft
US20130120585A1 (en) * 2011-11-15 2013-05-16 Canon Kabushiki Kaisha Automatic tracking camera system
CN103162669A (en) * 2013-03-01 2013-06-19 西北工业大学 Detection method of airport area through aerial shooting image
CN103577697A (en) * 2013-11-12 2014-02-12 中国民用航空总局第二研究所 FOD detection method based on road surface point cloud data
CN105302146A (en) * 2014-07-25 2016-02-03 空中客车运营简化股份公司 Method and system for automatic autonomous landing of an aircraft

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101777181A (en) * 2010-01-15 2010-07-14 西安电子科技大学 Ridgelet bi-frame system-based SAR image airfield runway extraction method
WO2013051967A2 (en) * 2011-08-31 2013-04-11 Kirillov Andrey Porfir Evich Method for visual landing and kirillov device for visualizing takeoff or landing of an aircraft
US20130120585A1 (en) * 2011-11-15 2013-05-16 Canon Kabushiki Kaisha Automatic tracking camera system
CN103162669A (en) * 2013-03-01 2013-06-19 西北工业大学 Detection method of airport area through aerial shooting image
CN103577697A (en) * 2013-11-12 2014-02-12 中国民用航空总局第二研究所 FOD detection method based on road surface point cloud data
CN105302146A (en) * 2014-07-25 2016-02-03 空中客车运营简化股份公司 Method and system for automatic autonomous landing of an aircraft

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
ERIC FREW ETC.: "Vision-Based Road-Following Using a Small Autonomous Aircraft", 《2004 IEEE AEROSPACE CONFERENCE PROCEEDINGS》 *
J. SCHONEFELD ETC.: "Runway incursion prevention systems: A review of runway incursion avoidance and alerting system approaches", 《PROGRESS IN AEROSPACE SCIENCES》 *
邸男 等: "提取直线特征实现机场跑道实时检测", 《光学精密工程》 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109238265A (en) * 2018-07-20 2019-01-18 民航中南空管设备工程公司 A kind of airfield runway location measurement method
CN109238265B (en) * 2018-07-20 2020-08-11 民航中南空管设备工程公司 Airport runway position measuring method
CN109341685A (en) * 2018-12-04 2019-02-15 中国航空工业集团公司西安航空计算技术研究所 A kind of fixed wing aircraft vision auxiliary landing navigation method based on homograph
CN109341700A (en) * 2018-12-04 2019-02-15 中国航空工业集团公司西安航空计算技术研究所 Fixed wing aircraft vision assists landing navigation method under a kind of low visibility
CN110274622A (en) * 2019-06-13 2019-09-24 北京环奥体育发展有限公司 A kind of in-site measurement, when calculating marathon race racing track complexity calculation method
CN110274622B (en) * 2019-06-13 2021-08-27 北京环奥体育发展有限公司 Method for calculating difficulty degree of track during marathon competition by field measurement and calculation
CN113295164A (en) * 2021-04-23 2021-08-24 四川腾盾科技有限公司 Unmanned aerial vehicle visual positioning method and device based on airport runway

Also Published As

Publication number Publication date
CN108225273B (en) 2020-06-30

Similar Documents

Publication Publication Date Title
US10515458B1 (en) Image-matching navigation method and apparatus for aerial vehicles
EP3407294B1 (en) Information processing method, device, and terminal
CN101598556B (en) Unmanned aerial vehicle vision/inertia integrated navigation method in unknown environment
CN108225273A (en) A kind of real-time runway detection method based on sensor priori
US20200357141A1 (en) Systems and methods for calibrating an optical system of a movable object
Liu et al. An off-board vision system for relative attitude measurement of aircraft
Ma et al. A novel autonomous aerial refueling drogue detection and pose estimation method based on monocular vision
CN109341686A (en) A kind of tightly coupled aircraft lands position and orientation estimation method of view-based access control model-inertia
CN109978954A (en) The method and apparatus of radar and camera combined calibrating based on cabinet
Vetrella et al. RGB-D camera-based quadrotor navigation in GPS-denied and low light environments using known 3D markers
Zhao et al. Homography-based camera pose estimation with known gravity direction for UAV navigation
Liu et al. Runway extraction in low visibility conditions based on sensor fusion method
US20130093880A1 (en) Height Measurement Apparatus And Method
Deng et al. A binocular vision-based measuring system for UAVs autonomous aerial refueling
Steffes et al. Optical terrain relative navigation approaches to lunar orbit, descent and landing
Angermann et al. High precision approaches enabled by an optical-based navigation system
Burdziakowski Towards precise visual navigation and direct georeferencing for MAV using ORB-SLAM2
Kim et al. Target detection and position likelihood using an aerial image sensor
Zhuge et al. 3-D motion trajectory measurement for the target through a multi-UAVs system
Zhang et al. Tracking and position of drogue for autonomous aerial refueling
Wang et al. A real-time sensor guided runway detection method for forward-looking aerial images
Al-Kaff Vision-based navigation system for unmanned aerial vehicles
Roozing et al. Low-cost vision-based 6-DOF MAV localization using IR beacons
Li-Chee-Ming et al. Determination of UAS trajectory in a known environment from FPV video
Hu et al. Toward high-quality magnetic data survey using UAV: development of a magnetic-isolated vision-based positioning system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant