CN112464886A - Aircraft identification tracking method - Google Patents

Aircraft identification tracking method Download PDF

Info

Publication number
CN112464886A
CN112464886A CN202011465723.0A CN202011465723A CN112464886A CN 112464886 A CN112464886 A CN 112464886A CN 202011465723 A CN202011465723 A CN 202011465723A CN 112464886 A CN112464886 A CN 112464886A
Authority
CN
China
Prior art keywords
aircraft
prediction
boundary box
information
detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011465723.0A
Other languages
Chinese (zh)
Inventor
钱久超
薛建伟
潘敏婷
朱肖光
刘佩林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jiaotong University
Original Assignee
Shanghai Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Jiaotong University filed Critical Shanghai Jiaotong University
Priority to CN202011465723.0A priority Critical patent/CN112464886A/en
Publication of CN112464886A publication Critical patent/CN112464886A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G06V20/42—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an aircraft identification and tracking method, which comprises the following steps: detecting and identifying video data containing an aircraft by using YOLOv3, and if the video data are judged to be the aircraft, recording boundary box information and center point information of the aircraft; constructing a flight state vector of the aircraft, wherein the flight state vector comprises the central point information and the boundary box information; predicting the motion trail of the aircraft by the flight state vector through a Kalman filter; correcting the motion trajectory prediction of the aircraft. The invention determines a specific range in the data frame by comparing the detected position information of the specific image frame with the information predicted by the Kalman filter, thereby improving the tracking speed. In addition, when the frame information difference is found to be large in the key of prediction and detection, the frame information can be adjusted in time, and the method is a method with a feedback mechanism and acting on the identification and tracking of the aircraft.

Description

Aircraft identification tracking method
Technical Field
The invention relates to the field of computer vision processing, in particular to a method for identifying and tracking objects in video data.
Background
The tracking and identification of the aircraft is an important technology in the national aviation safety field, and particularly has an important significance for the identification and tracking of some fighters, so that the national airspace safety is enhanced by some scientific and technological means in the military field, and the national attention is also paid to the aircraft. As a very important part of the field of computer vision, target tracking can implement a technology for identifying and tracking an aircraft in an acquired video image.
In recent years, with the development of deep learning techniques and computer capabilities, various applications based on deep learning are more and more varied, and in particular, various algorithms for deep learning are widely applied in the field of computer vision, and the target detection, identification and tracking techniques based on deep learning are rapidly developed. Compared with the traditional method, the recognition tracking method based on deep learning has the advantages that the accuracy is greatly improved, and the tracking performance is greatly improved. The current mainstream object detection strategies are divided into two categories, namely region suggestions and region distribution. However, the accuracy of aircraft tracking is mainly affected by complex environments, the speed of the aircraft tracking is high, and the difference between a frame image and a frame image is large, so that the visual tracking algorithm still has some challenging problems, such as sudden motion, attitude change, object shielding, background clutter, illumination or viewpoint change and the like, which can cause the performance reduction of the aircraft tracking, even the aircraft tracking fails, and these are external factors affecting the aircraft tracking effect. Therefore, how to identify and track the aircraft more reliably and accurately is a problem to be solved at present.
Disclosure of Invention
The invention aims to solve the technical problem of how to more reliably and accurately identify and track an aircraft, and provides an aircraft identification and tracking method.
The invention solves the technical problems through the following technical scheme:
an aircraft identification tracking method, the method comprising:
detecting and identifying video data containing an aircraft by using YOLOv3, and if the video data are judged to be the aircraft, recording boundary box information and center point information of the aircraft;
constructing a flight state vector of the aircraft, wherein the flight state vector comprises the central point information and the boundary box information;
predicting the motion trail of the aircraft by the flight state vector through a Kalman filter:
when the track of the central point between the adjacent images of the aircraft is a straight line, the position vector of the central point of the aircraft adopts a linear model, Kalman gain calculation is carried out according to the prediction result of the next moment and the detection result of the current moment, the prediction result of the previous moment and the detection result of the current moment are calculated, and the uncertainty of the prediction of the motion track of the current moment is determined;
when the track of the central point between the adjacent images of the aircraft is a nonlinear track, an extended Kalman filter is adopted for prediction, a nonlinear model is adopted for the position vector of the central point of the aircraft, and then the uncertainty of the motion track prediction at the current moment is determined;
combining the linear model prediction part and the nonlinear model prediction part to predict the motion trail of the aircraft;
correcting the motion trajectory prediction of the aircraft;
judging whether the boundary box can detect the aircraft, when the deviation between the prediction result and the detection result is large, limiting the effective range according to the size of a target object to improve the detection speed, if the overlapping value of the prediction boundary box and the detection boundary box of the aircraft is larger than a predefined threshold value, modifying the position and the size of the current detection boundary box according to the prediction boundary box of the previous frame, and if not, re-determining the boundary box by taking the aircraft as the center and inputting the boundary box into a detection network for training.
Preferably, the video data is processed into a video with a fixed length, and is combined into a data set to train the detection network, where the data set includes: a training set for training and a validation set for validation.
Further, when the position vector of the central point of the aircraft is approximated by a linear model, the state transition matrix and the observation matrix are constants; when the position vector of the aircraft center point is approximated by a nonlinear model, the state transition matrix and the observation matrix are respectively:
Figure BDA0002834101870000031
Figure BDA0002834101870000032
wherein, FkIs a state change matrix, where HkIs an observation matrix. Wherein
Figure BDA0002834101870000033
For the observed values of the input parameters, the lower subscript k is denoted as the kth time, where ukAre control vectors.
Further, the correction includes a bezel correction formula:
Figure BDA0002834101870000034
wherein deltab,δaRespectively representing the confidence of the detection result between the adjacent images, the higher the value of the confidence, the higher the accuracy of the model is, wherein wa，wb，wcRespectively representing the widths, h, of the adjacent images and the corrected bounding boxa,hb,hcRespectively representing the heights of the adjacent images and the corrected bounding box, wherein (x)a,ya),(xb,yb),(xc,yc) Respectively representing adjacent images and correctedThe horizontal and vertical height of the bounding box.
On the basis of the common knowledge in the field, the above preferred conditions can be combined randomly to obtain the preferred embodiments of the invention.
The positive progress effects of the invention are as follows: the position information detected by a specific image frame is compared with the information predicted by the Kalman filter to establish a specific range in the data frame, so that the tracking speed is improved. In addition, when the frame information difference is found to be large in the key of prediction and detection, the frame information can be adjusted in time, and the method is a method with a feedback mechanism and acting on the identification and tracking of the aircraft.
Drawings
FIG. 1 is a flowchart of a method in an embodiment of a method for identifying and tracking an aircraft according to the invention;
fig. 2 is a schematic diagram of a frame correction method according to an embodiment of an aircraft identification and tracking method of the present invention.
Detailed Description
To facilitate an understanding of the present application, the present application will now be described more fully with reference to the accompanying drawings. Preferred embodiments of the present application are shown in the drawings. This application may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the description of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
FIG. 1 shows a flowchart of a method for identifying and tracking an aircraft according to the present invention:
s01: detecting and identifying video data containing an aircraft by using YOLOv3, and if the video data are judged to be the aircraft, recording boundary box information and center point information of the aircraft;
in one example, YOLOV3 first compresses the input picture to 416 × 416, extracts features from the input image through a feature extraction network to obtain a feature map with a certain size, such as 13 × 13, then divides the input image into 13 × 13 grids, and then predicts a target in GT (real value) if the central coordinate of the target falls in which grid. Each mesh predicts 3 bounding boxes. The type of the detected object is then determined based on the values of a frame, wherein the frame contains four values v ═ (v ═ v)x,vy,vw,vh) For subsequent calculation, where vx,vyCoordinates representing the center point x and y of the object, vw,vhShowing the width and height of the bounding box;
dividing the ROI (Region Of interest) into four sub-regions, i.e., upper left, lower left, upper right, and lower right, which are used as a score map, the vertices Of the frame can be represented by the following methods, respectively.
Figure BDA0002834101870000051
Figure BDA0002834101870000052
Figure BDA0002834101870000053
Figure BDA0002834101870000054
Wherein B istlVertex representing the upper left corner, BtrVertex representing the upper right corner, BblVertex representing the lower left corner, BbrThe vertex representing the lower right corner.
S02: constructing a flight state vector of the aircraft, wherein the flight state vector comprises the central point information and the boundary box information;
in one example, a flight state vector of an aircraft is constructed
Figure BDA0002834101870000055
For describing the state of motion of an aircraft, in which,
Figure BDA0002834101870000056
for describing the position of the center point of the object,
Figure BDA0002834101870000057
for describing the proportion of the frame and the longitudinal and transverse proportion of the aircraft.
S03: predicting the motion trail of the aircraft by the flight state vector through a Kalman filter;
in one example, (1)
Figure BDA0002834101870000058
The approximation can be performed by a linear model, because the motion track between adjacent images can be identified by straight lines, and the parameters are separately extracted from the constructed state vector to describe the motion target.
xk＝Axk-1+Buk+wk (1)
zk＝Hxk+vk (2)
Wherein x represents the state vector of the system, z is the observed value, and A is the state transition matrix, and because the invention is applied to a non-controllable system, the control parameters formed by B and u can be ignored. Where H is the observation model, i.e., the observation matrix, the true state can be mapped to the observation space. w and v represent noise during state update and observation, respectively. The lower subscript k for all the above parameters indicates the time at kth.
(2) According to the principle of the kalman filter, it is known that:
Figure BDA0002834101870000061
Figure BDA0002834101870000062
wherein, P is an error covariance matrix between the predicted value and the actual value, and is used for representing the uncertainty of the prediction result, and Q is the newly increased uncertainty in the prediction process. Equation 3 represents the current state determined together with the external input at the previous moment, and equation 4 represents the new uncertainty Q due to the previously existing uncertainty.
(3) Calculating Kalman gain K and the predicted value of the kth time
Figure BDA0002834101870000063
Can be calculated from the following formula:
Figure BDA0002834101870000064
Figure BDA0002834101870000065
where equation 5 represents the uncertainty of the prediction result
Figure BDA0002834101870000066
And the uncertainty R of the observation, the kalman gain K is calculated. Formula 6 shows that the prediction results are weighted and averaged to obtain the state estimation of the current time.
(4) Updating
Figure BDA0002834101870000067
Representing the uncertainty of this state estimate.
And predicting the parts of s and r which are not suitable for the linear model by using an extended Kalman filter. The state vector is created in the same way as in the second step, the state matrix at this time is no longer a constant matrix, and the update is represented by:
Figure BDA0002834101870000068
wherein, the matrix FkAnd HkDerived from a Jacobian matrix, where FkCorresponds to the matrix A in the Kalman filter, and HkCorresponding to the constant matrix H.
And the inference of the nonlinear part is added into a linear inference system, namely the motion state of the airplane can be described.
S04: correcting the motion trajectory prediction of the aircraft;
as shown in fig. 1 and 2, in one example, it is determined whether the frame can correctly detect the airplane, and if the overlap degree between the predicted frame of the airplane and the actual detected frame of the airplane in the specific image is greater than a predefined threshold, the state variable is updated again according to the detected frame information, and the specific correction formula is as follows:
Figure BDA0002834101870000071
wherein deltab,δaRespectively representing the confidence of the detection result between the front image and the rear image, the higher the value of the confidence, the higher the accuracy of the model is, wherein wa，wb，wcRespectively representing the widths, h, of the adjacent images and the corrected bounding boxa,hb,hcRespectively representing the heights of the adjacent images and the corrected bounding box, wherein (x)a,ya),(xb,yb),(xc,yc) The horizontal and vertical heights of the front and rear images and the corrected bounding box are respectively indicated.
In one example, video data containing airplane motion is gathered, uniformly processed into fixed-length video, and composed into a data set, wherein 80% of the data is input into the detection network as a training set and verified through 20% of the data.
While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that this is by way of example only, and that the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the spirit and scope of the invention, and these changes and modifications are within the scope of the invention.

Claims (4)

1. An aircraft identification tracking method, characterized in that the method comprises:
detecting and identifying video data containing an aircraft by using YOLOv3, and if the video data are judged to be the aircraft, recording boundary box information and center point information of the aircraft;
constructing a flight state vector of the aircraft, wherein the flight state vector comprises the central point information and the boundary box information;
predicting the motion trail of the aircraft by the flight state vector through a Kalman filter:
when the track of the central point between the adjacent images of the aircraft is a straight line, the position vector of the central point of the aircraft adopts a linear model, Kalman gain calculation is carried out according to the prediction result of the next moment and the detection result of the current moment, the prediction result of the previous moment and the detection result of the current moment are calculated, and the uncertainty of the prediction of the motion track of the current moment is determined;
when the track of the central point between the adjacent images of the aircraft is a nonlinear track, an extended Kalman filter is adopted for prediction, a nonlinear model is adopted for the position vector of the central point of the aircraft, and then the uncertainty of the motion track prediction at the current moment is determined;
combining the linear model prediction part and the nonlinear model prediction part to predict the motion trail of the aircraft;
correcting the motion trajectory prediction of the aircraft;
judging whether the boundary box can detect the aircraft, when the deviation between the prediction result and the detection result is large, limiting the effective range according to the size of a target object to improve the detection speed, if the overlapping value of the prediction boundary box and the detection boundary box of the aircraft is larger than a predefined threshold value, modifying the position and the size of the current detection boundary box according to the prediction boundary box of the previous frame, and if not, re-determining the boundary box by taking the aircraft as the center and inputting the boundary box into a detection network for training.
2. The aircraft identification and tracking method according to claim 1, wherein the video data is processed into fixed-length video and composed into a data set to train the detection network, the data set comprising: a training set for training and a validation set for validation.
3. The method of claim 2, wherein the state transition matrix and the observation matrix are constants when the vector of the aircraft center point position is approximated by a linear model; when the position vector of the aircraft center point is approximated by a nonlinear model, the state transition matrix and the observation matrix are respectively:
Figure FDA0002834101860000021
Figure FDA0002834101860000022
wherein, FkIs a state change matrix, where HkIs an observation matrix. Wherein
Figure FDA0002834101860000023
For the observed values of the input parameters, the lower subscript k is denoted as the kth time, where ukAre control vectors.
4. The method of claim 2, wherein said correction comprises a frame correction formula:
Figure FDA0002834101860000024
wherein deltab,δaRespectively representing the confidence of the detection result between the adjacent images, the higher the value of the confidence, the higher the accuracy of the model is, wherein wa，wb，wcRespectively representing the widths, h, of the adjacent images and the corrected bounding boxa,hb,hcRespectively representing the heights of the adjacent images and the corrected bounding box, wherein (x)a,ya),(xb,yb),(xc,yc) Respectively representing the horizontal and vertical heights of the adjacent images and the corrected bounding box.
CN202011465723.0A 2020-12-14 2020-12-14 Aircraft identification tracking method Pending CN112464886A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011465723.0A CN112464886A (en) 2020-12-14 2020-12-14 Aircraft identification tracking method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011465723.0A CN112464886A (en) 2020-12-14 2020-12-14 Aircraft identification tracking method

Publications (1)

Publication Number Publication Date
CN112464886A true CN112464886A (en) 2021-03-09

Family

ID=74804071

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011465723.0A Pending CN112464886A (en) 2020-12-14 2020-12-14 Aircraft identification tracking method

Country Status (1)

Country Link
CN (1) CN112464886A (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108492324A (en) * 2018-01-27 2018-09-04 天津大学 Aircraft method for tracing based on fully-connected network and Kalman filter
CN110532889A (en) * 2019-08-02 2019-12-03 南京理工大学 Track foreign matter detecting method based on rotor unmanned aircraft and YOLOv3
CN110738690A (en) * 2019-09-16 2020-01-31 南京理工大学 unmanned aerial vehicle video middle vehicle speed correction method based on multi-target tracking framework
CN110751099A (en) * 2019-10-22 2020-02-04 东南大学 Unmanned aerial vehicle aerial video track high-precision extraction method based on deep learning
CN110956653A (en) * 2019-11-29 2020-04-03 中国科学院空间应用工程与技术中心 Satellite video dynamic target tracking method with fusion of correlation filter and motion estimation
CN111681259A (en) * 2020-05-17 2020-09-18 天津理工大学 Vehicle tracking model establishing method based on Anchor-free mechanism detection network

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108492324A (en) * 2018-01-27 2018-09-04 天津大学 Aircraft method for tracing based on fully-connected network and Kalman filter
CN110532889A (en) * 2019-08-02 2019-12-03 南京理工大学 Track foreign matter detecting method based on rotor unmanned aircraft and YOLOv3
CN110738690A (en) * 2019-09-16 2020-01-31 南京理工大学 unmanned aerial vehicle video middle vehicle speed correction method based on multi-target tracking framework
CN110751099A (en) * 2019-10-22 2020-02-04 东南大学 Unmanned aerial vehicle aerial video track high-precision extraction method based on deep learning
CN110956653A (en) * 2019-11-29 2020-04-03 中国科学院空间应用工程与技术中心 Satellite video dynamic target tracking method with fusion of correlation filter and motion estimation
CN111681259A (en) * 2020-05-17 2020-09-18 天津理工大学 Vehicle tracking model establishing method based on Anchor-free mechanism detection network

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
庄建: "《深度学习图像识别技术》", 29 February 2020, 机械工业出版社 *

Similar Documents

Publication Publication Date Title
Li et al. Adaptive pyramid mean shift for global real-time visual tracking
CN110688965B (en) IPT simulation training gesture recognition method based on binocular vision
CN105678338B (en) Target tracking method based on local feature learning
CN108492324B (en) Airplane tracking method based on full-connection network and Kalman filter
CN108876820B (en) Moving target tracking method under shielding condition based on mean shift
CN104820996A (en) Target tracking method based on self-adaptive blocks of video
CN106952294B (en) A kind of video tracing method based on RGB-D data
CN106600613B (en) Improvement LBP infrared target detection method based on embedded gpu
CN112927211B (en) Universal attack countermeasure method based on depth three-dimensional detector, storage medium and terminal
CN109448023A (en) A kind of satellite video Small object method for real time tracking of combination space confidence map and track estimation
CN111192294A (en) Target tracking method and system based on target detection
Iraei et al. Object tracking with occlusion handling using mean shift, Kalman filter and edge histogram
CN111354022A (en) Target tracking method and system based on kernel correlation filtering
CN115239760B (en) Target tracking method, system, equipment and storage medium
CN108230313B (en) SAR image target detection method based on component adaptive selection and discrimination model
CN111539993B (en) Space target visual tracking method based on segmentation
CN112464886A (en) Aircraft identification tracking method
CN110826575A (en) Underwater target identification method based on machine learning
Wang et al. Pmds-slam: Probability mesh enhanced semantic slam in dynamic environments
CN110147747B (en) Correlation filtering tracking method based on accumulated first-order derivative high-confidence strategy
WO2022222087A1 (en) Method and apparatus for generating adversarial patch
CN112465865A (en) Multi-target tracking method based on background modeling and IoU matching
JPH07320068A (en) Method and processor for image processing
CN105654514A (en) Image target tracking method
CN111860264A (en) Multi-task instance level road scene understanding algorithm based on gradient equilibrium strategy

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210309