CN110660077A - Multi-scale target tracking method fusing multiple features - Google Patents
Multi-scale target tracking method fusing multiple features Download PDFInfo
- Publication number
- CN110660077A CN110660077A CN201910861204.7A CN201910861204A CN110660077A CN 110660077 A CN110660077 A CN 110660077A CN 201910861204 A CN201910861204 A CN 201910861204A CN 110660077 A CN110660077 A CN 110660077A
- Authority
- CN
- China
- Prior art keywords
- target
- scale
- filter
- tracking
- tracking method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 230000004044 response Effects 0.000 claims abstract description 20
- 238000001914 filtration Methods 0.000 claims abstract description 6
- 230000008569 process Effects 0.000 claims abstract description 4
- 238000012360 testing method Methods 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 claims description 8
- 238000012549 training Methods 0.000 claims description 8
- 230000004927 fusion Effects 0.000 claims description 7
- 238000001514 detection method Methods 0.000 abstract description 7
- 238000009825 accumulation Methods 0.000 abstract description 2
- 230000000903 blocking effect Effects 0.000 abstract 1
- 230000008859 change Effects 0.000 description 8
- 238000005286 illumination Methods 0.000 description 5
- 230000007547 defect Effects 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/207—Analysis of motion for motion estimation over a hierarchy of resolutions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a multi-scale target tracking method fusing multiple characteristics, and belongs to the technical field of computer vision tracking. Firstly, extracting HOG characteristics and Color (CN) characteristics from a target, and fusing the two characteristics by using a linear weighting method to predict a target position; then determining the optimal scale of the target by using the HOG characteristics of the target under multiple scales; and finally, designing a re-detection module by using an image blocking and sample one-by-one testing method, starting the re-detection module when the target cannot be stably tracked due to shielding, finding out a target central point by using a maximum response value after re-detection, and simultaneously introducing a model automatic updating strategy to update the position of the target, thereby avoiding the tracking drift phenomenon. The method can effectively process the situation of tracking drift caused by error accumulation of the shielding filter in the traditional related filtering tracking method, can effectively track the target under the complex situation, and improves the tracking robustness.
Description
Technical Field
The invention relates to the technical field of computer vision target tracking, in particular to a multi-scale target tracking method fusing multiple characteristics.
Background
Target tracking is one of the key research directions in computer vision, and comprises machine learning, event detection, signal processing, video monitoring, automatic driving, statistics and other related knowledge. Although the target tracking has been advanced greatly in these years, the related filtering tracking method still faces a great challenge due to factors such as occlusion, fast motion, blur, illumination and deformation of the tracked target.
The existing target tracking method based on multiple characteristics has the defect that the occlusion, deformation and out-of-plane rotation cannot be processed. When the target is blocked and the tracking fails, a target re-detection module is not provided, the method for searching the optimal scale estimation corresponding to the maximum output response value and the optimal central point position of the target is optimized on the basis of the tracker of the existing method, the thought of image block re-detection is provided, the image blocks after being partitioned are subjected to relevant filtering one by one, the position of the maximum response value is found, and the target central point position of the current frame is updated. In addition, a model updating strategy is introduced, the model is normally updated when the target normally tracks, when the target starts to be shielded, the model is updated slowly or even stopped to be updated, and the error accumulation of the model is prevented.
Disclosure of Invention
In order to make up for the defects of the prior art, the invention provides a multi-scale target tracking method fusing a plurality of characteristics from three aspects of classifier training, model updating and adaptive scale estimation.
To achieve the above purpose, four parts are designed: classifier training, position estimation, adaptive scale estimation and model updating.
The method of each part is as follows:
and training the classifier, namely training a position filter for each layer of neural network by establishing a minimum cost function to obtain an optimal correlation filter:
in the formula (1), flIs a d-dimensional feature vector, d represents the dimension of the selected feature, l ∈ {1, 2.. d }. Each channel has a corresponding filter hlThe operation of convolution is represented, the superscript l represents a certain dimension of the characteristic, g represents the output of an ideal Gaussian function, l is a regularization coefficient, lambda has the function of eliminating the influence of zero-frequency components in the frequency spectrum on one hand, and the overfitting of a filter is avoided on the other hand, and the formula (1) can be solved in a frequency domain to obtain a formula (2):
where capital letters are the frequency domain form of the corresponding variable, and the upper dash line indicates the conjugate complex number of the variable.
The position estimate is used to predict a target position, and is specifically configured to:
after the filter training is finished, the position of the target of the next frame is estimated, and after the HOG characteristic and the CN characteristic of the test sample are extracted, the spatial response of the corresponding characteristic target position is respectively obtained:
wherein
Representing the inverse fourier transform and Z representing the frequency domain description of the candidate samples input in the new frame. And adopting a fixed weight fusion strategy to perform response graph fusion of the HOG characteristic and the CN characteristic, wherein the maximum response value is the predicted target position. The fusion formula is:
y＝(1-r)yhog+rycolor (4)
the scale estimation work is as follows:
when the target is subjected to scale estimation, the optimal scale of the target is found by using a scale filter on the position of the predicted target by using the HOG characteristics of the target image. The target scale estimation process is similar to the target position predicted by the position filter, and the specific operation is as follows:
assuming that the size of an input sample is P multiplied by R, the target optimal scale is determined by the maximum value of the scale response obtained by convolving the input sample with a one-dimensional scale filter, and the selection principle for evaluating the target scale sample is as follows:
in the formula (5), a is a step factor; p, R are width and height of the previous frame of the target, respectively; and S is the number of scales.
The filter updating mode is as follows:
for the tracking task, the target is always in the change, and the position and scale filter needs to be updated to adapt to the appearance and scale change of the target. To solve the problem of calculating a linear equation of dimension d × d for each pixel point in the image, the numerator in equation (2) can be paired
And denominator BtAnd respectively updating:
the position filter and the scale filter are both updated according to equation (6). In order to avoid the phenomenon that tracking drift occurs due to the influence of conditions such as shielding, rotation and illumination change on model updating training, the method introduces a model updating strategy to control the speed of model updating, and aims to reduce the accumulated error of a target model. Setting a threshold value TrWhen responding to the maximum value y>TrIf so, normally updating the model of the current frame (the tth frame), namely the formula (6); otherwise, the target area is considered to have uncertainty and is not updated.
Compared with the existing mainstream correlation filtering tracking method, the method has the advantages that:
1. the HOG characteristic has better adaptability to small deformation, illumination change and the like of the target, but when the target is greatly deformed or shielded, the HOG characteristic cannot adapt to the change of the target, so that the target is lost; the CN feature is a global feature based on pixel points, can effectively solve the problems of target deformation and scale change, but cannot adapt to illumination change. Therefore, the defects of the method can be well complemented by using a mode of fusing HOG characteristics and CN characteristics, so that the tracking can adapt to various conditions, and the accuracy and the robustness of the target tracking method are improved.
2. The improved method based on image block re-detection is provided, the maximum response searching method is optimized, meanwhile, a model automatic updating strategy is introduced, the problem of target shielding can be effectively solved, and robust target tracking can be achieved under complex scenes such as illumination change, shielding, motion blurring and rotation.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
In order to more clearly understand the technical features, objects, and effects of the present invention, embodiments of the present invention will now be described with reference to the accompanying drawings.
FIG. 1 is a flow chart of the method of the present invention:
the first step is as follows: and acquiring initial position information and scale information of the target.
The second step is that: and extracting the HOG characteristic and the CN characteristic of the target according to the initial information acquired in the first step.
Training a position filter by establishing a minimum cost function to obtain an optimal correlation filter:
in the formula (1), flIs a d-dimensional feature vector, d represents the dimension of the selected feature, l ∈ {1, 2.. d }. Each channel has a corresponding filter hlThe operation of convolution is represented, the superscript l represents a certain dimension of the characteristic, g represents an ideal Gaussian function output, and λ is a regularization coefficient, the function of λ is to eliminate the influence of zero-frequency components in the frequency spectrum on one hand, and on the other hand, to avoid filter overfitting, the formula (1) can be solved in the frequency domain to obtain the formula (2):
where capital letters are the frequency domain form of the corresponding variable, and the upper dash line indicates the conjugate complex number of the variable.
The third step: and respectively establishing related filtering models according to the second step, extracting HOG characteristics and CN characteristics of the test sample, and respectively obtaining the spatial response of the corresponding characteristic target position:
wherein
Representing the inverse fourier transform and Z representing the frequency domain description of the candidate samples input in the new frame.
And adopting a fixed weight fusion strategy to perform response graph fusion of the HOG characteristic and the CN characteristic, wherein the maximum response value is the predicted target position. The fusion formula is:
y＝(1-r)yhog+rycolor (10)
the fourth step: and predicting the target position according to the filter response of the third step. The maximum response value y is the predicted target position.
The fifth step: and according to the target position information predicted in the fourth step, on the basis, HOG characteristics are extracted from the target, and the optimal scale of the target is found by using a scale filter.
The size of an input sample is P multiplied by R, the target optimal scale is determined by the maximum scale response value obtained by convolution of the input sample and a one-dimensional scale filter, and the selection principle for evaluating the target scale sample is as follows:
in the formula (5), a is a step factor; p, R are width and height of the previous frame of the target, respectively; and S is the number of scales.
And a sixth step: based on the predicted target response value, a threshold value T can be set for this purposerWhen responding to the maximum value y>TrIf so, normally updating the model of the current frame (the tth frame), namely the formula (6); otherwise, the target area is considered to have uncertainty and is not updated.
The position filter and the scale filter are both updated according to equation (6).
The above are only preferred embodiments of the invention and, of course, are not intended to limit the scope of the invention. Accordingly, equivalent changes made in the claims of the invention are still within the scope of the invention.
Claims (4)
1. A multi-scale target tracking method fusing multiple features is characterized in that the multi-scale target tracking method fusing multiple features specifically comprises the following steps:
the first step is as follows: acquiring initial position information and scale information of a target;
the second step is that: extracting HOG characteristics and CN characteristics of a target according to the initial information acquired in the first step;
the third step: according to the HOG characteristics and CN characteristics extracted in the second step, a linear weighting method is utilized and a fixed coefficient fusion mode is adopted to obtain related filtering response;
the fourth step: predicting the target position according to the filter response of the third step;
the fifth step: according to the target position information predicted in the fourth step, on the basis, HOG characteristics are extracted from the target, and the optimal scale of the target is found by using a scale filter;
and a sixth step: finding the optimal predicted position of the target, retraining the model at the new position, introducing a model updating strategy, and controlling the model updating speed of the whole tracking process, and circulating the steps until the last frame of the image sequence.
2. The multi-scale target tracking method fusing multiple features according to claim 1, characterized in that: the HOG features and CN features extracted in the second step are obtained by constructing an optimal correlation filter to train a position filter for each feature, wherein the optimal correlation filter is obtained by a minimum cost function, and the formula is as follows:
in the formula (1), flIs a d-dimensional feature vector, d represents the dimension of the selected feature, l ∈ {1, 2.. d }. Each channel has a corresponding filter hlThe x represents convolution operation, the superscript l represents a certain dimension of the feature, g represents ideal Gaussian function output, and lambda is a regularization coefficient;
equation (1) can be solved in the frequency domain to obtain equation (2):
where capital letters are the frequency domain form of the corresponding variable, and the upper dash line indicates the conjugate complex number of the variable. After the filter training is finished, the position of the target of the next frame is estimated, the HOG characteristic and the CN characteristic of the test sample are extracted, and the spatial response of the corresponding characteristic target position is respectively obtained:
whereinRepresenting the inverse fourier transform and Z representing the frequency domain description of the candidate samples input in the new frame. The maximum value of the filter response is the target position, and the target scale estimation process is similar to the target position predicted by the position filter.
3. The multi-scale target tracking method fusing multiple features according to claim 1, characterized in that: in the sixth step, the model is retrained, and a model updating strategy is introduced to match molecules in the formula (2)
And denominator BtAnd respectively updating:
eta is the learning rate, the numeric area is more than or equal to 0 and less than 0.1, and the larger the value is, the larger the occupied weight is.
4. The multi-scale target tracking method fusing multiple features according to claim 2, characterized in that: the maximum value of the target response is the predicted target position, and a threshold value T can be set for the predicted target positionrWhen responding to the maximum value y>TrIf so, normally updating the model of the current frame (the tth frame), namely the formula (4); otherwise, the target area is considered to have uncertainty and is not updated.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910861204.7A CN110660077A (en) | 2019-09-12 | 2019-09-12 | Multi-scale target tracking method fusing multiple features |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910861204.7A CN110660077A (en) | 2019-09-12 | 2019-09-12 | Multi-scale target tracking method fusing multiple features |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110660077A true CN110660077A (en) | 2020-01-07 |
Family
ID=69037302
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910861204.7A Pending CN110660077A (en) | 2019-09-12 | 2019-09-12 | Multi-scale target tracking method fusing multiple features |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110660077A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111652903A (en) * | 2020-05-22 | 2020-09-11 | 重庆理工大学 | Pedestrian target tracking method based on convolution correlation network in automatic driving scene |
| CN112767437A (en) * | 2020-12-30 | 2021-05-07 | 大连海事大学 | Water surface unmanned ship tracking method, system and storage medium based on KCF self-adaptive multi-feature fusion filtering |
| CN112785622A (en) * | 2020-12-30 | 2021-05-11 | 大连海事大学 | Long-time tracking method and device for unmanned ship on water surface and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107644430A (en) * | 2017-07-27 | 2018-01-30 | 孙战里 | Target following based on self-adaptive features fusion |
| CN109285179A (en) * | 2018-07-26 | 2019-01-29 | 昆明理工大学 | A kind of motion target tracking method based on multi-feature fusion |
-
2019
- 2019-09-12 CN CN201910861204.7A patent/CN110660077A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107644430A (en) * | 2017-07-27 | 2018-01-30 | 孙战里 | Target following based on self-adaptive features fusion |
| CN109285179A (en) * | 2018-07-26 | 2019-01-29 | 昆明理工大学 | A kind of motion target tracking method based on multi-feature fusion |
Non-Patent Citations (2)
| Title |
|---|
| 曾梦媛 等，: "融合多层卷积特征自适应更新的目标跟踪算法", 《激光与光电子学进展》 * |
| 谢柳 等，: "基于相关滤波融合多特征的运动目标跟踪方法", 《数据采集与处理》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111652903A (en) * | 2020-05-22 | 2020-09-11 | 重庆理工大学 | Pedestrian target tracking method based on convolution correlation network in automatic driving scene |
| CN111652903B (en) * | 2020-05-22 | 2023-09-08 | 重庆理工大学 | Pedestrian target tracking method based on convolution association network in automatic driving scene |
| CN112767437A (en) * | 2020-12-30 | 2021-05-07 | 大连海事大学 | Water surface unmanned ship tracking method, system and storage medium based on KCF self-adaptive multi-feature fusion filtering |
| CN112785622A (en) * | 2020-12-30 | 2021-05-11 | 大连海事大学 | Long-time tracking method and device for unmanned ship on water surface and storage medium |
| CN112785622B (en) * | 2020-12-30 | 2024-04-05 | 大连海事大学 | Method and device for tracking unmanned captain on water surface and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109285179B (en) | Moving target tracking method based on multi-feature fusion | |
| CN108549839B (en) | Adaptive feature fusion multi-scale correlation filtering visual tracking method | |
| CN107689052B (en) | Visual target tracking method based on multi-model fusion and structured depth features | |
| CN110175649B (en) | Rapid multi-scale estimation target tracking method for re-detection | |
| CN107369166B (en) | Target tracking method and system based on multi-resolution neural network | |
| CN109741366B (en) | Related filtering target tracking method fusing multilayer convolution characteristics | |
| CN108198209B (en) | People tracking method under the condition of shielding and scale change | |
| CN110533691B (en) | Target tracking method, device and storage medium based on multiple classifiers | |
| CN108961308B (en) | Residual error depth characteristic target tracking method for drift detection | |
| CN110660077A (en) | Multi-scale target tracking method fusing multiple features | |
| CN111008996B (en) | Target tracking method through hierarchical feature response fusion | |
| CN110796679B (en) | Target tracking method for aerial image | |
| CN111260738A (en) | Multi-scale target tracking method based on relevant filtering and self-adaptive feature fusion | |
| CN111582349B (en) | Improved target tracking algorithm based on YOLOv3 and kernel correlation filtering | |
| CN110660080A (en) | Multi-scale target tracking method based on learning rate adjustment and fusion of multilayer convolution features | |
| CN110555870B (en) | DCF tracking confidence evaluation and classifier updating method based on neural network | |
| CN110889863B (en) | Target tracking method based on target perception correlation filtering | |
| CN106803265A (en) | Multi-object tracking method based on optical flow method and Kalman filtering | |
| CN107590427B (en) | Method for detecting abnormal events of surveillance video based on space-time interest point noise reduction | |
| CN111008991B (en) | Background-aware related filtering target tracking method | |
| CN111340842B (en) | Correlation filtering target tracking method based on joint model | |
| CN108364305B (en) | Vehicle-mounted camera video target tracking method based on improved DSST | |
| CN112085765A (en) | Video target tracking method combining particle filtering and metric learning | |
| CN115471525A (en) | Target tracking method and system based on fusion of twin network and Kalman filtering | |
| CN110827327B (en) | Fusion-based long-term target tracking method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200107 |
|
| WD01 | Invention patent application deemed withdrawn after publication |