CN109448023B - Satellite video small target real-time tracking method - Google Patents

Satellite video small target real-time tracking method Download PDF

Info

Publication number
CN109448023B
CN109448023B CN201811235772.8A CN201811235772A CN109448023B CN 109448023 B CN109448023 B CN 109448023B CN 201811235772 A CN201811235772 A CN 201811235772A CN 109448023 B CN109448023 B CN 109448023B
Authority
CN
China
Prior art keywords
tracking
target
filter
new
tracking filter
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811235772.8A
Other languages
Chinese (zh)
Other versions
CN109448023A (en
Inventor
眭海刚
陈旭
孙向东
付宏博
杨威
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan University WHU
Original Assignee
Wuhan University WHU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan University WHU filed Critical Wuhan University WHU
Priority to CN201811235772.8A priority Critical patent/CN109448023B/en
Publication of CN109448023A publication Critical patent/CN109448023A/en
Application granted granted Critical
Publication of CN109448023B publication Critical patent/CN109448023B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/262—Analysis of motion using transform domain methods, e.g. Fourier domain methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)
  • Radar Systems Or Details Thereof (AREA)

Abstract

The invention discloses a satellite video small target real-time tracking method combining a spatial confidence map and trajectory estimation, which comprises the steps of initializing a tracking filter and initializing parameters of a Kalman filter, regressing the Kalman filter by utilizing a tracked target position, and predicting a new target position; performing Fourier transform on the extracted features of the next frame of image, utilizing the constructed tracking filter, solving the maximum response value of the result through image convolution to serve as a new position of a target, and extracting features by taking the new position as a center to update the tracking filter; and circularly transmitting the next frame of image, judging whether the set prediction condition is met, and performing corresponding processing to extract a new position. The method fully considers the characteristics of few pixels occupied by the target relative to the background, no obvious shape characteristics, high similarity between the color and the background and the like in the satellite video, not only ensures the effectiveness of the tracking result, but also has real-time tracking speed.

Description

Satellite video small target real-time tracking method
Technical Field
The invention relates to the technical field of satellite video tracking, in particular to a satellite video small target real-time tracking method combining a spatial confidence map and trajectory estimation.
Background
The video satellite is a novel earth observation satellite appearing in recent years, and compared with the traditional satellite, the video satellite can continuously observe a certain area and generate dynamic video information with high time and high spatial resolution. Currently, the NASA international space station has released a satellite video with a resolution of 1m, and a commercial satellite No. jilin transmitted in china can shoot a satellite video with a resolution of 0.72 m. Compared with a ground video, the moving target in the satellite video has the characteristics of difficult shielding, wide visible range and the like, so that the high-resolution satellite video analysis has great potential in the aspects of motion analysis, traffic monitoring, suspicious target supervision and the like. Therefore, the satellite video data is utilized to track the moving target, and the method has strong theoretical guidance and practical application significance.
The current video target tracking method mainly comprises a generation class and a discrimination class. The generation method converts the target tracking problem into a search problem, describes the expression characteristics of the target by using a generation model, and then minimizes the reconstruction error by searching candidate targets. The method is easy to drift when the target changes violently or is shielded because background information is ignored. The discrimination method uses the target area as a positive sample and the background area as a negative sample, trains a classifier by a machine learning method, and finds the optimal area by the trained classifier in the next frame. The biggest difference from the generation method is that the background information is utilized during training, and the classifier can better distinguish the foreground from the background, so the effect is generally better than that of the generation method. Among the discrimination algorithms, the most concerned and most effective ones at present are the correlation filtering algorithm and the deep learning algorithm. The related filtering algorithm is represented by KCF, DCF and CN, and the calculation is carried out by converting the property of the cyclic matrix into the Fourier space, so that the algorithm efficiency is greatly improved, and the quick and effective tracking can be realized. The deep learning algorithm is represented by GOTURN, MDNet and SimFC, deep semantic features of the target are obtained through training of a large number of samples, tracking of severely deformed and shielded targets can be better achieved, and real-time tracking is difficult to achieve due to large calculation amount.
For satellite video, the resolution of each frame of image can reach 3840 × 2160 or more, which is several tens times or even one hundred times of the size of each frame of image of common video. Wherein dynamic objects such as vehicles, airplanes and the like generally occupy 20 x 20 or less pixels. Compared with the whole image, the method occupies too few pixels, meanwhile, most of targets are similar to the background, and the resolution of the targets is low, so that most of mainstream tracking algorithms cannot continuously and effectively track the targets in the satellite video.
Disclosure of Invention
Aiming at the problems, the method provides a satellite video small target real-time tracking method combining a spatial confidence map and trajectory estimation.
The invention provides a satellite video small target real-time tracking method combining a spatial confidence map and trajectory estimation, which comprises the following steps:
step 1, initializing a tracking filter and parameters of a Kalman filter, wherein the initialization of the tracking filter comprises feature extraction of a tracking search area and construction of a Gaussian response map, a spatial confidence map and the tracking filter;
step 2, the tracked target position is used for regressing a Kalman filter to predict a new position of the target;
step 3, performing Fourier transform on the extracted features of the next frame of image, utilizing the constructed tracking filter, solving the maximum response value of the result through image convolution to serve as a new position of the target, and extracting the features by taking the new position as the center to update the tracking filter;
and 4, circularly transmitting the next frame of image, judging whether the set prediction condition is met or not, executing the step 2 only if the condition is met, taking the result of the step 2 as a new position, and executing the step 2 and the step 3 if the condition is not met, and taking the result of the step 3 as a new position.
Moreover, said step 1 comprises the sub-steps of,
step 1.1, firstly determining the size of a template of a tracking search area, calculating a Gaussian form response graph of the area, transforming a central maximum value to an upper left corner maximum value through cyclic shift, then carrying out Fourier transform on a result, and marking the transformed result as a mark
Figure BDA0001838137650000021
Step 1.2, calculating the space confidence of the size of the template, and marking as m;
step 1.3, extracting image gray level characteristics in the large and small areas of the template, and addingHann window, Fourier transform, labeled
Figure BDA0001838137650000022
Step 1.4, constructing a Lagrange expression, solving the Lagrange expression to obtain a tracking filter, and marking the tracking filter as h;
lagrange's expression is:
Figure BDA0001838137650000023
hc-hmeither as 0 (type two)
hmAs m can
Wherein, the letter
Figure BDA0001838137650000031
A Lagrange multiplier representing a complex number,
Figure BDA0001838137650000032
to represent
Figure BDA0001838137650000033
H in the upper right corner of the letter represents the conjugate transpose matrix, and the variable HmIn order to increase the spatial confidence of the tracking filter,
Figure BDA0001838137650000034
is hmThe fourier transform of (a) the signal,
Figure BDA0001838137650000035
is a variable hcIs preferably set to the expression parameter, and λ is ═ μ/100.0, which is adjusted in the recursive solution process according to the current value. (ii) a
The final tracking filter calculation formula is obtained by solving the partial derivative of each variable of the formula and utilizing the basic property of the diagonal matrix,
Figure BDA0001838137650000036
Figure BDA0001838137650000037
wherein the content of the first and second substances,
Figure BDA0001838137650000038
to represent
Figure BDA0001838137650000039
The conjugate matrix of (a) is determined,
Figure BDA00018381376500000310
to represent
Figure BDA00018381376500000311
D represents hmThe dimension of the matrix of (a) is,
Figure BDA00018381376500000312
representing an inverse fourier transform;
step 1.5, initializing a transfer matrix A, a measurement matrix H, a system noise variance matrix Q and a measurement noise variance matrix R of a Kalman filter, and taking the initial position of a tracking target as an initial measurement value z (0) and an initial state value x (0);
furthermore, said step 2 comprises the sub-steps of,
step 2.1, predicting the position of the next frame according to a Kalman filtering theory;
and 2.2, updating the Kalman gain by taking the position of the current frame as a measured value according to a Kalman filtering theory, and calculating a new state value and a predicted value by using the obtained Kalman gain.
Furthermore, said step 3 comprises the sub-steps of,
step 3.1, transmitting a target enclosing frame of the previous frame, calculating the gray scale feature of the image template size of the frame, carrying out Fourier transform on the feature to a frequency domain, carrying out image convolution on the feature of the frequency domain and a tracking filter obtained from the previous frame, carrying out inverse Fourier transform to obtain a response graph, calculating the maximum value of the response graph, and taking the position of the maximum value as a new position of the target;
3.2, constructing a spatial confidence map by using the image information of the frame and the new position, constructing a new tracking filter, and updating the tracking filter;
the process of constructing the new tracking filter is consistent with the steps 1.2, 1.3 and 1.4, and the obtained newly constructed tracking filter is marked as hnew；
Is provided with hkFor the tracking filter of the current k-th frame, hk-1The tracking filter for the k-th frame of the previous frame, the fusion function is,
hk＝(1-θ)×hk-1+θ×hnew(type six)
Where θ is the fusion coefficient.
Furthermore, said step 4 comprises the sub-steps of,
step 4.1, circularly transmitting the next frame of image, judging whether the number of frames already carried out is greater than 5 frames or not, and simultaneously judging whether the number of current frames is a multiple of 3 or not; if yes, the step 4.2 is carried out, otherwise, the step 4.3 is carried out;
step 4.2, when the condition is met, only the prediction and the update of the Kalman filter are executed, and the predicted position is taken as a new target position;
and 4.3, if the condition is not met, firstly executing tracking of a tracking filter to track the position as a new target position, and then executing updating of the Kalman filter.
Compared with the prior art, the invention has the following beneficial effects:
the method fully considers the characteristics of few pixels of the target relative to the background, no obvious shape feature, high similarity of color and the background and the like in the satellite video, trains a tracking filter with space confidence by adopting a strategy of combining tracking and trajectory estimation, simultaneously regresses a Kalman filter capable of estimating the trajectory, and combines two filters by setting a certain prediction condition, thereby ensuring the effectiveness of a tracking result and having real-time tracking speed. The invention embodies the feasibility and effectiveness of realizing the real-time tracking of the satellite video small target by combining the spatial confidence map and the trajectory estimation.
Drawings
Fig. 1 is a system configuration diagram of an embodiment of the present invention.
Detailed Description
The technical solution of the present invention is described in detail below with reference to the accompanying drawings and examples.
The method fully considers the characteristics of fewer pixels occupied by the target, high similarity with the background and the like, increases a spatial confidence map when training the classifier, and eliminates the interference of similar targets or backgrounds in adjacent areas. Meanwhile, the training of a multi-scale template of the classifier is reduced by almost invariable target scales in the satellite video, and the tracking performance is greatly improved. Aiming at the problem that the target and the background are integrated and cannot be tracked, a Kalman filter is trained to realize the track estimation of the target, and the continuous and effective tracking of the target is guaranteed.
Firstly, initializing a tracking filter and a Kalman filter, mainly comprising extracting gray features of a training area, expanding a certain threshold value by using the center of a target surrounding frame to generate an ideal Gaussian form response image and a space confidence image, and solving a target function in a frequency domain through Fourier transformation to obtain a discriminant tracking filter. And initializing a transfer matrix, a measurement matrix and each noise variance matrix of the Kalman filter, and taking the target initial position as a measurement value.
And continuously regressing a Kalman filter by utilizing the tracked target position, and simultaneously predicting a new position of the target, wherein the method mainly comprises the steps of predicting the position of the current frame based on a Kalman algorithm according to the target position of the previous frame, and updating a Kalman gain by using the predicted position or the position tracked by the tracking filter.
And then tracking the newly-transmitted image by using the tracking filter constructed in the first part and updating the tracking filter, mainly comprising the steps of extracting gray features of the newly-transmitted image according to the size of the calculated template, carrying out Fourier transform, carrying out image convolution on a frequency domain and the tracking filter constructed in the previous step, obtaining a response graph after inverse Fourier transform, and searching the position of the maximum value in the response graph as a new target position. And meanwhile, a new filter is constructed by using the newly extracted features and the new spatial confidence map and is fused with the previous filter according to certain parameters, so that the filter is updated.
And finally, circularly executing the incoming video image and judging whether the set prediction condition is met. Only executing a Kalman filter to predict the position according to the conditions, obtaining a new Kalman gain by using a predicted position regression filter according to an update equation of Kalman, and taking the predicted position as a new target position; and if the condition is not met, the tracking filter is executed to track the target, a new position of the target is obtained, the tracking filter is updated at the same time, and the tracking position is regressed to the Kalman filter to update the Kalman gain. The trajectory estimation of the target can be realized through the position prediction of the Kalman filter, the problem that the target cannot be tracked when the target is consistent with the background is solved, and the target can be effectively tracked by constructing the tracking filter with the spatial confidence coefficient. By combining the two filters, the track estimation assists in tracking to guarantee continuous and effective tracking of the target.
The embodiment provides a satellite video small target real-time tracking method combining a spatial confidence map and trajectory estimation, which is divided into four parts, wherein the first part is to construct an initial tracking filter and a Kalman filter. The second part is to predict and update the new position of the target using a Kalman filter. The third section simultaneously updates the filter with the tracking filter for the incoming image tracking. And the fourth step is to perform tracking circularly, judge whether the prediction condition is met, perform target position prediction if the condition is met, and perform target tracking if the condition is not met.
The specific implementation flow of the embodiment comprises the following steps:
step 1: initializing a tracking filter and parameters of a Kalman filter, wherein the initialization of the tracking filter comprises the steps of extracting the characteristics of a tracking search area, and constructing a Gaussian response map, a spatial confidence map and the filter;
the step 1 specifically comprises the following steps:
step 1.1: firstly, a template for tracking a search area is determined, in the embodiment, an initial frame image and a target of a satellite video are inputThe initial bounding box (i.e. the target bounding box given in the initial frame image of the satellite video) should be included in the template, and preferably the size of the template is set to be 4 times of the size value of the target initial bounding box, and the gaussian shape response map of the region is calculated by taking the center of the target initial bounding box as the center of the template. Circularly shifting the obtained response diagram to convert the former central position to the upper left corner position, then carrying out Fourier transform on the result, and marking the converted result as a mark
Figure BDA0001838137650000051
The function for creating the two-dimensional Gaussian form when the Gaussian form response graph of the region is calculated is as follows:
Figure BDA0001838137650000052
where e is an exponential operator, w is the width of the template, b is the height of the template, i is the row coordinate, j is the column coordinate, and σ is a parameter for adjusting the radius of the gaussian shape, and the preferred embodiment is 1.0, yi,jIs the value corresponding to the row and column coordinates (i, j) in the response map.
Step 1.2: calculating a spatial confidence map of the size of the template, wherein the spatial confidence map sets a weight value for each pixel of the template in the step 1.1, all areas corresponding to the initial target are set to be a maximum value of 1.0, all other areas are set to be 0.0, and the spatial confidence map is marked as m;
step 1.3: extracting image gray level features in the large and small areas of the template, adding Hann window, performing Fourier transform, and marking the obtained feature transform result as
Figure BDA0001838137650000067
Generally speaking, the histogram feature of the directional gradient and the color feature have stronger expression on a target, but because the target in the satellite video only occupies dozens to hundreds of pixels relative to the surrounding environment, no obvious shape information exists, the difference between the color and the background is not large, the histogram feature of the directional gradient and the color feature are compressed, and the gray feature can better express the target feature.
Step 1.4: constructing a Lagrange expression, solving the expression to obtain a tracking filter, and marking the tracking filter as h;
to avoid multi-peak responses in the filter, the filter is constrained by adding a spatial confidence map. The function for solving the ridge regression minimum value in general has no closed solution any more, and the variable h is introducedcAnd constructing a Lagrange expression and obtaining a closed solution again.
Lagrange's expression is:
Figure BDA0001838137650000061
hc-hm＝0 (3)
hm＝m⊙h (4)
wherein, the letter
Figure BDA0001838137650000062
A Lagrange multiplier representing a complex number,
Figure BDA0001838137650000063
to represent
Figure BDA0001838137650000064
H in the upper right corner of the letter represents the conjugate transpose matrix, and the variable HmIn order to increase the spatial confidence of the tracking filter,
Figure BDA0001838137650000065
is hmThe fourier transform of (a) the signal,
Figure BDA0001838137650000066
is a variable hcThe fourier transform of (d). Mu is an expression parameter, the initial value is preferably set to be 5.0, and the parameter can be updated in the recursive solving process, and the preferred updating mode is that the value mu is taken according to the parameter beta and the parameter of the last recursioni-1Updating the current parameter value mui＝β×μi-1Where i is the number of recursions and β is preferably 3.0 by default. λ is an expression parameter, and λ is μ/100.0, and is adjusted according to the current μ value in the recursive solving process. And (3) obtaining a final tracking filter calculation formula by solving partial derivatives of each variable of the formula (2) and utilizing basic properties of a diagonal matrix:
Figure BDA0001838137650000071
Figure BDA0001838137650000072
wherein the content of the first and second substances,
Figure BDA0001838137650000073
to represent
Figure BDA0001838137650000074
The conjugate matrix of (a) is determined,
Figure BDA0001838137650000075
to represent
Figure BDA0001838137650000076
D represents hmThe dimension of the matrix of (a) is,
Figure BDA0001838137650000077
representing the inverse fourier transform.
In the implementation, the number of regression solutions may be preset by those skilled in the art, and in the embodiment, it is preferable to set the tracking filter h to be solved by four recursions, so as to reduce the loss to a lower range.
By increasing the limitation of the space confidence coefficient to the tracking filter, the region in the target enclosing frame is guaranteed to have the highest response value, and the interference of an object with similar characteristics to the target is eliminated.
Step 1.5: initializing a transfer matrix A, a measurement matrix H, a system noise variance matrix Q and a measurement noise variance matrix R of a Kalman filter to track the initial position of a target (namely the central position of a target initial surrounding frame) as an initial measurement value z (0) and an initial state value x (0);
preferably, the transfer matrix a, the measurement matrix H, the system noise variance matrix Q, and the measurement noise variance matrix R of the Kalman filter may be initialized to be an identity matrix of 4 × 4.
Step 2: predicting a new position of the target by using the tracked target position and a regression Kalman filter;
the step 2 specifically comprises the following steps:
step 2.1: predicting the position of the next frame according to a Kalman filtering theory;
the equation used to predict position is:
x′(k)＝A×x(k-1) (7)
P′(k)＝A×P(k-1)×AT+1×Q (8)
wherein x '(k) represents a k-th frame prediction value of the state value, x (k-1) represents a state value of a k-1-th frame, P' (k) represents a k-th frame prediction value of a minimum mean square error, and P (k-1) represents a minimum mean square error of the k-1-th frame.
Step 2.2: according to the Kalman filtering theory, the position of the current frame is used as the measured value z (k) of the kth frame to update the Kalman gain;
by solving the least squares estimate, the kalman gain K is obtained, the function used being:
K(k)＝P′(k)×HT×(H×P′(k)×HT+1×R)-1 (9)
x(k)＝x′(k)+K(k)×(z(k)-H×x′(k)) (10)
P(k)＝(I-K(k)×H)P′(k-1) (11)
wherein K (k) represents the k frame Kalman gain, HTDenotes the transpose of the measurement matrix H, (H.times.P' (k). times.HT+1× R)-1For H × P' (k) × HT+1 xr inverse matrix, x (k) represents the state value of the kth frame, I represents the identity matrix, and p (k) represents the minimum mean square error of the kth frame.
The state value x (k) and the minimum mean square error p (k) of the kth frame are calculated according to equations (10) and (11) using the kalman gain obtained by equation (9).
And step 3: performing Fourier transform on the extracted features of the next frame of image, utilizing the constructed tracking filter, solving the maximum response value of the result through image convolution to serve as a new position of a target, and extracting features by taking the new position as a center to update the tracking filter;
the step 3 specifically comprises the following steps:
step 3.1: transmitting the target enclosing frame of the previous frame, calculating the gray scale feature of the image template size of the frame, performing Fourier transform on the feature to a frequency domain, performing image convolution on the feature of the frequency domain and a tracking filter obtained from the previous frame, performing inverse Fourier transform to obtain a response graph, obtaining the maximum value of the response graph, and taking the position of the maximum value as a new position of the target;
because the shooting distance of the satellite video is far, the target does not have large scale change, so that the scale transformation and scale estimation are not carried out on the target any more, the template size calculated during initialization is used as the template size for extracting the features later, and the center of the transmitted target bounding box is used as the center.
The characteristic matrix in the frequency domain and the tracking filter in the frequency domain can reduce the calculation amount of multiplication of two matrixes to the calculation amount of multiplication of diagonal elements by utilizing the property of the diagonal matrix, thereby greatly accelerating the calculation speed. And transforming the complex matrix of the frequency domain into a real-time matrix of the space domain by an inverse Fourier transform. The result obtained by the image convolution should be a response graph with Gaussian distribution, and the point with the maximum value in the response graph is searched, and the position of the point is used as the new position of the target
Step 3.2: constructing a spatial confidence map by using the image information of the frame and the new position, constructing a new tracking filter, and updating the tracking filter;
the process of constructing the new filter is the same as the steps 1.2, 1.3 and 1.4, and the newly constructed filter is marked as hnew。
Is provided with hkFilter for current k frame, hk-1For the filter of the k frame of the previous frame, the filter of the previous frame and the new filter are fused in a certain proportion to ensure that the previous frame can be utilizedThe characteristic information can also process the new change condition of the current frame. The function used is:
hk＝(1-θ)×hk-1+θ×hnew (12)
wherein, the right h is the tracking filter obtained in step 1.4, the left h is the updated tracking filter, theta is the fusion coefficient preferably 0.02, hnewIs a newly constructed filter.
And 4, step 4: and (4) circularly transmitting the next frame of image, judging whether the set prediction condition is met, executing step 2 only if the condition is met, and taking the result of step 2 as a new position, and executing step 2 and step 3 if the condition is not met, and taking the result of step 3 as a new position.
The step 4 specifically comprises the following steps:
step 4.1: circularly transmitting the next frame of image, judging whether the number of frames already processed is greater than 5 frames or not, and judging whether the current frame number is a multiple of 3 or not; if yes, the step 4.2 is carried out, otherwise, the step 4.3 is carried out;
the determination of whether the frame number is greater than 5 frames is mainly to ensure that the Kalman filter has performed sufficient regression, and the predicted position is closer to the true position. Whether the frame number is a multiple of 3 or not is to predict the target track instead of tracking the target track at a certain frequency so as to solve the problem that the target and the background are completely fused together and the target cannot be distinguished. Due to the characteristic of satellite video imaging, the situation that the target is completely consistent with the background is far greater than that of general ground video and unmanned aerial vehicle video.
Step 4.2: when the condition is met, only prediction and updating of the Kalman filter are executed, and the predicted position is used as a new target position;
step 2.1 is performed with the predicted position as the new center of the target position, step 2.2 is performed with the position as the measured value to regress to the Kalman filter, and then step 4 is continued for the next frame until the end of the video.
Step 4.3: if the condition is not met, firstly executing tracking by a tracking filter to track the position as a new target position, and then executing updating of a Kalman filter;
and (3) tracking the target by the tracking filter by executing the steps 3.1 and 3.2, and taking the obtained position as a new position of the target. Step 2.2 is performed to regress the Kalman filter with the position as a measure, and then step 4 is continued for the next frame until the end of the video.
In specific implementation, the above processes can be automatically operated by adopting a computer software technology. According to the software design habit, the prediction condition judgment setting of the step 4 can also be executed before the step 3, and the effect is equivalent.
Although the present invention has been described in detail and illustrated with reference to the embodiments, the present invention and the applicable embodiments are not limited thereto, and those skilled in the art can make various modifications according to the principle of the present invention and can also apply a part of the method of the present invention to other systems. Thus, modifications made in accordance with the principles of the present invention should be understood to fall within the scope of the present invention.

Claims (5)

1. A satellite video small target real-time tracking method combining a spatial confidence map and trajectory estimation is characterized by comprising the following steps:
step 1, initializing a tracking filter and parameters of a Kalman filter, wherein the initialization of the tracking filter comprises feature extraction of a tracking search area and construction of a Gaussian response map, a spatial confidence map and the tracking filter;
step 2, the tracked target position is used for regressing a Kalman filter to predict a new position of the target;
step 3, performing Fourier transform on the extracted features of the next frame of image, utilizing the constructed tracking filter, solving the maximum response value of the result through image convolution to serve as a new position of the target, and extracting the features by taking the new position as the center to update the tracking filter;
and 4, circularly transmitting the next frame of image, judging whether the set prediction condition is met or not, executing the step 2 only when the condition is met, taking the result of the step 2 as a new position, and not executing the step 2 and the step 3, and taking the result of the step 3 as a new position.
2. The satellite video small-target real-time tracking method combining the spatial confidence map and the trajectory estimation, as claimed in claim 1, wherein: said step 1 comprises the sub-steps of,
step 1.1, firstly determining the size of a template of a tracking search area, calculating a Gaussian form response graph of the area, transforming a central maximum value to an upper left corner maximum value through cyclic shift, then carrying out Fourier transform on a result, and marking the transformed result as a mark
Figure FDA0002975620840000013
Step 1.2, calculating the space confidence of the size of the template, and marking as m;
step 1.3, extracting image gray features in the size region of the template, adding a Hann window, performing Fourier transform, and marking as
Figure FDA0002975620840000012
Step 1.4, constructing a Lagrange expression, solving the Lagrange expression to obtain a tracking filter, and marking the tracking filter as h;
lagrange's expression is:
Figure FDA0002975620840000011
hc-hmeither as 0 (type two)
hmAs m can
Wherein, the letter
Figure FDA0002975620840000021
A Lagrange multiplier representing a complex number,
Figure FDA0002975620840000022
to represent
Figure FDA0002975620840000023
H in the upper right corner of the letter represents the conjugate transpose matrix, and the variable HmIn order to increase the spatial confidence of the tracking filter,
Figure FDA0002975620840000024
is hmThe fourier transform of (a) the signal,
Figure FDA0002975620840000025
is a variable hcThe Fourier transform of (1), mu and lambda are expression parameters, and the lambda is mu/100.0 and is adjusted according to the current value in the recursive solving process;
the final tracking filter calculation formula is obtained by solving the partial derivative of each variable of the formula and utilizing the basic property of the diagonal matrix,
Figure FDA0002975620840000026
Figure FDA0002975620840000027
wherein the content of the first and second substances,
Figure FDA0002975620840000028
to represent
Figure FDA00029756208400000212
The conjugate matrix of (a) is determined,
Figure FDA0002975620840000029
to represent
Figure FDA00029756208400000210
D represents hmThe dimension of the matrix of (a) is,
Figure FDA00029756208400000211
representing an inverse fourier transform;
step 1.5, initializing a transfer matrix A, a measurement matrix H, a system noise variance matrix Q and a measurement noise variance matrix R of a Kalman filter, and taking the initial position of a tracking target as an initial measurement value z (0) and an initial state value x (0).
3. The satellite video small-target real-time tracking method combining the spatial confidence map and the trajectory estimation, as claimed in claim 1, wherein: said step 2 comprises the sub-steps of,
step 2.1, predicting the position of the next frame according to a Kalman filtering theory;
and 2.2, updating the Kalman gain by taking the position of the current frame as a measured value according to a Kalman filtering theory, and calculating a new state value and a predicted value by using the obtained Kalman gain.
4. The satellite video small-target real-time tracking method combining the spatial confidence map and the trajectory estimation as claimed in claim 2, wherein: said step 3 comprises the sub-steps of,
step 3.1, transmitting a target enclosing frame of the previous frame, calculating the gray scale feature of the size of an image template of the current frame, carrying out Fourier transform on the feature to a frequency domain, carrying out image convolution on the feature of the frequency domain and a tracking filter obtained from the previous frame, carrying out inverse Fourier transform to obtain a response graph, calculating the maximum value of the response graph, and taking the position of the maximum value as a new position of the target;
3.2, constructing a spatial confidence map by using the image information of the current frame and the new position, constructing a new tracking filter, and updating the tracking filter;
the process of constructing the new tracking filter is consistent with the steps 1.2, 1.3 and 1.4, and the obtained newly constructed tracking filter is marked as hnew；
Is provided with hkFor the tracking filter of the current k-th frame, hk-1The tracking filter for the k-1 frame of the previous frame, the fusion function is,
hk＝(1-θ)×hk-1+θ×hnew(type six)
Where θ is the fusion coefficient.
5. The satellite video small target real-time tracking method combining the spatial confidence map and the trajectory estimation according to claim 1, 2, 3 or 4, characterized in that: said step 4 comprises the sub-steps of,
step 4.1, circularly transmitting the next frame of image, judging whether the number of frames already carried out is greater than 5 frames or not, and simultaneously judging whether the number of current frames is a multiple of 3 or not; if yes, the step 4.2 is carried out, otherwise, the step 4.3 is carried out;
step 4.2, when the condition is met, only the prediction and the update of the Kalman filter are executed, and the predicted position is taken as a new target position;
and 4.3, if the condition is not met, firstly executing tracking of a tracking filter to track the position as a new target position, and then executing updating of the Kalman filter.
CN201811235772.8A 2018-10-23 2018-10-23 Satellite video small target real-time tracking method Active CN109448023B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811235772.8A CN109448023B (en) 2018-10-23 2018-10-23 Satellite video small target real-time tracking method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811235772.8A CN109448023B (en) 2018-10-23 2018-10-23 Satellite video small target real-time tracking method

Publications (2)

Publication Number Publication Date
CN109448023A CN109448023A (en) 2019-03-08
CN109448023B true CN109448023B (en) 2021-05-18

Family

ID=65548232

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811235772.8A Active CN109448023B (en) 2018-10-23 2018-10-23 Satellite video small target real-time tracking method

Country Status (1)

Country Link
CN (1) CN109448023B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110046435B (en) * 2019-04-19 2023-06-09 湖南银河电气有限公司 Integrated digital filtering harmonic wave display method and system
CN110580713A (en) * 2019-08-30 2019-12-17 武汉大学 Satellite video target tracking method based on full convolution twin network and track prediction
CN111161304B (en) * 2019-12-16 2023-06-30 北京空间机电研究所 Remote sensing video target track tracking method for rapid background estimation
CN111652906B (en) * 2020-05-11 2021-04-20 中国科学院空间应用工程与技术中心 Adaptive tracking method, device and equipment for satellite video ground dynamic target rotation
CN111652910B (en) * 2020-05-22 2023-04-11 重庆理工大学 Target tracking algorithm based on object space relationship

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104537692A (en) * 2014-12-30 2015-04-22 中国人民解放军国防科学技术大学 Key point stabilization tracking method based on time-space contextual information assisting
CN105631895A (en) * 2015-12-18 2016-06-01 重庆大学 Temporal-spatial context video target tracking method combining particle filtering
CN106097383A (en) * 2016-05-30 2016-11-09 海信集团有限公司 A kind of method for tracking target for occlusion issue and equipment
JP2018077807A (en) * 2016-11-11 2018-05-17 Kddi株式会社 Device, program and method for tracing body while taking multiple candidates into consideration at change point

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104537692A (en) * 2014-12-30 2015-04-22 中国人民解放军国防科学技术大学 Key point stabilization tracking method based on time-space contextual information assisting
CN105631895A (en) * 2015-12-18 2016-06-01 重庆大学 Temporal-spatial context video target tracking method combining particle filtering
CN106097383A (en) * 2016-05-30 2016-11-09 海信集团有限公司 A kind of method for tracking target for occlusion issue and equipment
JP2018077807A (en) * 2016-11-11 2018-05-17 Kddi株式会社 Device, program and method for tracing body while taking multiple candidates into consideration at change point

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
《Object Tracking in Satellite Videos Based on a Multi-Frame Optical Flow Tracker》;Bo Du,et al;《arXiv.org》;20180430;第1-11页 *
《卫星遥感图像中的小目标形状描述算法》;张守娟，等;《计算机工程》;20111231;第37卷(第24期);第213-216页 *
《基于方向可靠性的互补跟踪算法》;宋日成，等;《光学学报》;20181010;第38卷(第10期);第1-9页 *
《基于置信图特性的改进时空上下文目标跟踪》;张雷，等;《计算机工程》;20160831;第42卷(第8期);第277-272页 *

Also Published As

Publication number Publication date
CN109448023A (en) 2019-03-08

Similar Documents

Publication Publication Date Title
CN109448023B (en) Satellite video small target real-time tracking method
CN106780620B (en) Table tennis motion trail identification, positioning and tracking system and method
CN109102522B (en) Target tracking method and device
CN106845374B (en) Pedestrian detection method and detection device based on deep learning
CN108776975B (en) Visual tracking method based on semi-supervised feature and filter joint learning
CN108109162B (en) Multi-scale target tracking method using self-adaptive feature fusion
CN108288062B (en) Target tracking method based on kernel correlation filtering
CN110490907B (en) Moving target tracking method based on multi-target feature and improved correlation filter
CN110991272A (en) Multi-target vehicle track identification method based on video tracking
CN108446634B (en) Aircraft continuous tracking method based on combination of video analysis and positioning information
CN111311647B (en) Global-local and Kalman filtering-based target tracking method and device
CN108171715B (en) Image segmentation method and device
CN105160310A (en) 3D (three-dimensional) convolutional neural network based human body behavior recognition method
CN106780542A (en) A kind of machine fish tracking of the Camshift based on embedded Kalman filter
CN108876820B (en) Moving target tracking method under shielding condition based on mean shift
CN107590821B (en) Target tracking method and system based on track optimization
CN111191627A (en) Method for improving accuracy of dynamic gesture motion recognition under multiple viewpoints
CN109323697B (en) Method for rapidly converging particles during starting of indoor robot at any point
CN110827262B (en) Weak and small target detection method based on continuous limited frame infrared image
CN113379789B (en) Moving target tracking method in complex environment
CN111080674A (en) Multi-target ISAR key point extraction method based on Gaussian mixture model
CN110706253B (en) Target tracking method, system and device based on apparent feature and depth feature
CN107705295B (en) Image difference detection method based on robust principal component analysis method
CN117011381A (en) Real-time surgical instrument pose estimation method and system based on deep learning and stereoscopic vision
CN109064498A (en) Method for tracking target based on Meanshift, Kalman filtering and images match

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant