CN102930558B

CN102930558B - Real-time tracking method for infrared image target with multi-feature fusion

Info

Publication number: CN102930558B
Application number: CN201210397686.3A
Authority: CN
Inventors: 白俊奇; 赵春光; 王寿峰; 翟尚礼; 汪洋
Original assignee: CETC 28 Research Institute
Current assignee: CETC 28 Research Institute
Priority date: 2012-10-18
Filing date: 2012-10-18
Publication date: 2015-04-01
Anticipated expiration: 2032-10-18
Also published as: CN102930558A

Abstract

The invention discloses a real-time tracking method for an infrared image target with multi-feature fusion. The real-time tracking method comprises the following steps of: initializing a target tracking point position; initializing a target model; calculating a target candidate model; calculating a coefficient of unified feature Bhattacharyya and weight coefficients between feature; calculating a new tracking position of a current frame target; estimating the coefficient of the unified feature Bhattacharyya at the new position; and comparing the two coefficients of the unified feature Bhattacharyya and outputting results. The real-time tracking method can be used for adaptively calculating the weight coefficients between multiple features, enhances the robustness of target tracking, ensures the stability of target tracking, solves the problem of tracking point drifting caused by unstable single feature, and effectively improves the accuracy of target tracking.

Description

Multi-feature fusion infrared image target real-time tracking method

Technical Field

The invention designs an infrared image target tracking method with multi-feature fusion, and particularly relates to an infrared image target tracking method suitable for hardware to realize in real time.

Background

In recent years, with the development of integrated circuit processes and infrared materials, the infrared imaging technology has made great progress and is widely applied to the fields of national defense construction and national economy. However, the infrared image has a relatively low signal-to-noise ratio compared to the visible image, and therefore provides limited information when performing infrared image target detection and tracking. Due to the fact that target features in the infrared image are not obvious, the problems of large background clutter and the like exist, and accurate tracking of the infrared image target becomes more difficult.

Currently, target tracking algorithms are classified into two categories, model-based tracking methods and appearance-based tracking methods. Compared with a model tracking method, the appearance tracking method avoids a complex process of establishing the model and has wider engineering practical value. The mean shift tracking algorithm is widely applied to target tracking due to the characteristics of simplicity, robustness and good real-time performance. The mean shift is a non-parameter density calculation method, and a distribution mode most similar to the distribution of the sample is searched through multiple iterations. The comeniciu et al propose a mean shift target tracking algorithm by finding the maximum of the similarity of the target color histogram and the candidate target color histogram. Chu et al use a Kalman filter to predict the initial iterative position of Mean Shift, but when the target is severely occluded, there is some deviation due to inaccuracy in the target location point found by the Mean Shift algorithm. Collins et al propose an adaptive tracking method that can select easily identifiable color features, where the candidate color feature set contains 49 sets of features calculated from R, G, B-valued linear combinations. Therefore, the existing target tracking algorithm has the following disadvantages: (1) the classical target tracking algorithm adopts a single characteristic to describe a target, and has poor anti-jamming capability; (2) most of the existing multi-feature target tracking algorithms only utilize the current frame to calculate the weight coefficient among features, and when the target is subjected to complex change, the tracking algorithms have poor robustness; (3) under the conditions of non-rigid deformation, local shielding and overlapping of a target, the tracking precision of most of the existing tracking algorithms is reduced, and even the target is lost; (4) most of the existing tracking algorithms greatly increase the complexity of the algorithms while improving the target tracking precision, and are not easy to realize by hardware in real time.

Disclosure of Invention

The purpose of the invention is as follows: the invention aims to solve the technical problem of providing a real-time tracking method of an infrared image target with multi-feature fusion aiming at the defects of the prior art.

In order to solve the technical problem, the invention discloses a real-time tracking method of an infrared image target with multi-feature fusion, which comprises the following steps:

(1) initializing target tracking point location y₀. The initial tracking point is manually designated;

(2) initializing the target model to initiate tracking point y₀Establishing a target gray model q for the center₁And a target LBP texture model q₂(ii) a (local binary pattern, LBP) local binary patterns.

(3) Calculating a target candidate model according to the position y of the tracking point of the target₀Calculating a candidate target gray model p₁(y₀) And candidate target LBP texture model p₂(y₀)；

(4) Coefficient ρ is determined by using the gray-scale feature Bhattacharyya (Bhattacharyya, see Visual C + + digital image processing, page 466, author: Xiancheng, first edition 2008, electronics industry Press)₁And the Bhattacharyya coefficient rho of the LBP texture feature₂And a weight coefficient alpha of the gradation characteristic₁And the weighting coefficient alpha of the LBP texture feature₂Calculating the position y₀And (3) processing the combined characteristic Bhattacharyya coefficient rho, wherein the expression is as follows:

ρ＝α₁·ρ₁+α₂·ρ₂；

(5) calculating the new position y of the target of the current frame₁；

(6) Utilizing gray level feature Bhattacharyya coefficient rho'₁And LBP texture feature Bhattacharyya coefficient ρ'₂And a weight coefficient of a gray-scale feature of α'₁And weight coefficient of LBP texture feature of alpha'₂Calculating the position y₁The coefficient rho' of the Bhattacharyya of the joint characteristic is expressed as follows;

ρ′＝α′₁·ρ′₁+α′₂·ρ′₂，

(7) when rho'<At the time of the rho,otherwise y₁Keeping the same;

(8) if (y)₀-y₁)|<Stop the calculation, otherwise, will y₁Assign y to₀And step (3) is executed, wherein the error constant coefficient is.

In step (2), the target gray model q₁Comprises the following steps:

target LBP texture model q₂Comprises the following steps:

wherein q is_uProbability density of each level, q, of a target gray model gray feature_vFor each level of probability density, m, of the LBP texture feature₁Maximum quantization scale range, m, for the gray features of the target gray model₂And u represents a gray quantization level and v represents a texture quantization level, which is the maximum quantization level range of the LBP texture characteristics.

In step (3), candidate target gray model p₁Comprises the following steps:

candidate target LBP texture model p₂Comprises the following steps:

p_uprobability density of each level, p, of a target gray model gray feature_vThe level probability density, m, of the target gray model gray feature and the LBP texture feature₁Maximum quantization scale range, m, for the gray features of the target gray model₂And u represents a gray quantization level and v represents a texture quantization level, which is the maximum quantization level range of the LBP texture characteristics.

Weighting coefficient alpha of gray characteristic in step (4)₁And the weighting coefficient alpha of the LBP texture feature₂Weight coefficient α 'of gray feature in step (6)'₁And weight coefficient of LBP texture feature of alpha'₂Updating in an iterative mode, wherein the calculation formulas are respectively as follows:

α₁＝(1-λ)·α_1,old+λ·α_1,cur，

α₂＝(1-λ)·α_2,old+λ·α_2,cur，

α'₁＝(1-λ)·α'_1,old+λ·α'_1,cur，

α'₂＝(1-λ)·α'_2,old+λ·α'_2,cur，

wherein alpha is_1,oldAnd alpha_2,oldThe weight coefficients, alpha, of the gray feature and LBP texture feature of the previous frame in the step (4) are respectively_1,curAnd alpha_2,curRespectively are the weight coefficients, alpha ', of the gray feature and the LBP texture feature of the current frame in the step (4)'_1,oldAnd alpha'_2,oldRespectively are the weight coefficients, alpha ', of the gray feature and the LBP texture feature of the previous frame in the step (6)'_1,curAnd alpha'_2,curAnd (4) weighting coefficients of the gray feature and the LBP texture feature of the current frame in the step (6), wherein lambda is a proportionality coefficient. And lambda is more than or equal to 0 and less than or equal to 1, the convergence speed of the weight coefficient is determined, the larger the lambda value is, the faster the convergence speed is, the stronger the tracking mobility is, the smaller the lambda value is, the slower the convergence speed is, and the better the tracking stability is.

Weighting coefficient alpha of gray feature of current frame in step (4)_1,curAnd the weighting coefficient alpha of the LBP texture feature_2,curAnd (5) weighting coefficient alpha 'of the gray level feature of the current frame in the step (6)'_1,curAnd weight coefficient of LBP texture feature of alpha'_2,curThe calculation formula is as follows:

where ρ is₁Is the Bhattacharyya coefficient, rho, of the gray feature in step (4)₂Is the Bhattacharyya coefficient, rho 'of the LBP texture feature in the step (4)'₁Is the Bhattacharyya coefficient, rho 'of the gray feature in the step (6)'₂Is the Bhattacharyya coefficient of the LBP texture feature in step (6).

Ash characteristic Bhattacharyya coefficient rho 'in step (6)'₁And Bhattacharyya coefficient ρ 'of LBP texture feature'₂And a weight coefficient of a gray-scale feature of α'₁And weight coefficient of LBP texture feature of alpha'₂Obtained by the following formula:

p'_uis position y₁Of the target grayscale model grayscale feature of (d'_vIs position y₁Processing each level of probability density of LBP texture characteristics;

wherein, alpha'_1,oldIs the weight coefficient of the gray feature of the previous frame, alpha'_2,oldIs the weighting coefficient of the last frame LBP texture feature, and is the scaling coefficient. And lambda is more than or equal to 0 and less than or equal to 1, the convergence speed of the weight coefficient is determined, the larger the lambda value is, the faster the convergence speed is, the stronger the tracking mobility is, the smaller the lambda value is, the slower the convergence speed is, and the better the tracking stability is.

In the multi-feature fusion infrared image target real-time tracking method, an Epanechnikov (Epanechnikov) kernel function is used for calculating a gray feature probability histogram and an LBP texture feature probability histogram.

Compared with the prior art, the invention has the following remarkable advantages: (1) according to the feature significance and similarity of the target and the background, the weight coefficient among multiple features is calculated in a self-adaptive mode, and the target tracking robustness is enhanced; (2) the weight coefficient among multiple features is updated in an iterative mode, so that the target tracking stability is ensured; (3) the infrared image target is tracked by utilizing a multi-feature fusion target tracking method, so that the problem of tracking point drift caused by instability of single feature is solved, and the target tracking precision is effectively improved; (4) the multi-feature fusion target tracking method provided by the invention has the advantages of no high-order operation and complex structure, small algorithm operation amount and easiness in hardware real-time realization.

Drawings

The above and other advantages of the present invention will become more apparent from the following detailed description of the invention when taken in conjunction with the accompanying drawings.

FIG. 1 is a flow chart of the present invention.

Fig. 2a to 2d are conventional single-feature (gray scale) infrared image target tracking results.

Fig. 3a to 3d are the target tracking results of the infrared image with multi-feature fusion according to the present invention.

Detailed Description

In the multi-feature fusion infrared image target real-time tracking method, the characteristics of the infrared image target are described by utilizing the gray characteristic and the LBP texture characteristic.

Eight neighborhood LBP texture feature expression LBP_8,1As follows:

<math> <mrow> <mi>s</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfenced open='{' close=''> <mtable> <mtr> <mtd> <mn>1</mn> <mo>,</mo> </mtd> <mtd> <mi>x</mi> <mo>&GreaterEqual;</mo> <mn>0</mn> </mtd> </mtr> <mtr> <mtd> <mn>0</mn> <mo>,</mo> </mtd> <mtd> <mi>x</mi> <mo><</mo> <mn>0</mn> </mtd> </mtr> </mtable> </mfenced> </mrow> </math>

wherein, g_cIs the current point, g_nIs the surrounding neighborhood point, n is 0.. 7.

According to the infrared image target real-time tracking method based on multi-feature fusion, the similarity between a target model and a target candidate model is described by using a Bhattacharyya coefficient.

Coefficient rho of Bhattacharyya_BhaThe expression is as follows:

where p is the target candidate model and q is the target model.

In the multi-feature fusion infrared image target real-time tracking method, the weight coefficient alpha of the gray feature₁And the weighting coefficient alpha of the LBP texture feature₂Updating in an iterative mode, wherein the expression is as follows:

α₁＝(1-λ)·α_1,old+λ·α_1,cur

α₂＝(1-λ)·α_2,old+λ·α_2,cur

wherein alpha is_1,oldAnd alpha_2,oldThe weight coefficients of the gray feature of the previous frame and the LBP texture feature are respectively alpha_1,curAnd alpha_2,curIs the weighting coefficient of the gray feature of the current frame and the LBP texture feature, and is the scale coefficient.

In the multi-feature fusion infrared image target real-time tracking method, the weight coefficient alpha of the gray features of the current frame_1,curAnd the weighting coefficient alpha of the LBP texture feature_2,curThe expressions are respectively as follows:

where ρ is₁Is the Bhattacharyya coefficient, rho, of the gray scale feature₂Is the Bhattacharyya coefficient of the LBP texture feature.

Example 1

Referring to fig. 1, the following describes, by way of example, a real-time tracking method for infrared image targets with multi-feature fusion according to the present invention. The number of pixels of the infrared image is 320 × 240, and the frame rate is 25 HZ. The imaging of the thermal infrared imager is transmitted to a special image processing board with a DSP + FPGA framework through optical fibers, the tracking of the infrared image target with multi-feature fusion is realized in a DSP processor, the requirement of real-time processing is met, and the specific implementation steps are as follows:

(1) initializing target tracking point location y₀The initial tracking point is manually specified.

An initial target tracking point position (i, j) is manually specified, i is 80, j is 100 (shown in fig. 2), and an einzekov (Epanechnikov) kernel function bandwidth h is set to 10.

(2) Initializing target model, and establishing target gray scale model q according to gray scale characteristics₁Calculating LBP texture characteristics of the target, and establishing a target texture model q by combining the LBP texture characteristics₂；

An image I in a range of a bandwidth h of 10 centered on an initial tracking point position (80,100) is calculated_LBPThe expression of LBP texture feature of (1) is as follows:

eight neighborhood LBP texture feature expression LBP_8,1As follows:

<math> <mrow> <mi>s</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfenced open='{' close=''> <mtable> <mtr> <mtd> <mn>1</mn> <mo>,</mo> </mtd> <mtd> <mi>x</mi> <mo>&GreaterEqual;</mo> <mn>0</mn> </mtd> </mtr> <mtr> <mtd> <mn>0</mn> <mo>,</mo> </mtd> <mtd> <mi>x</mi> <mo><</mo> <mn>0</mn> </mtd> </mtr> </mtable> </mfenced> <mo>,</mo> </mrow> </math>

wherein, g_cIs the current target point, c-k 2 x 320+ k1, g_nIs g_cAnd (4) surrounding neighborhood points, wherein n is 0.

Target grayscale model q₁Comprises the following steps:

target LBP texture model q₂Comprises the following steps:

wherein q is_uAnd q is_vRespectively representing the probability density of each level of the gray level characteristic and the LBP texture characteristic of the target model, m₁255 and m₂255 represent the quantization levels of the target model grayscale feature and the LBP texture feature, respectively, function b₁Is located at x_iTo a grey scale feature index, function b₂Is located at x_iThe mapping of the pixel to the LBP texture feature index of (1) is a Delta function, C is a normalization coefficient, μ ═ 1.. 255, and v ═ 1.. 255.

(3) And calculating the target candidate model. According to the position y of the tracking point₀Calculating a candidate target gray model p₁(y₀) And a target texture candidate model p₂(y₀)；

Candidate target grayscale model p₁Comprises the following steps:

candidate target LBP texture model p₂Comprises the following steps:

p_uand p_vRespectively representing the probability density of each level of the gray level characteristic and the LBP texture characteristic of the target model, m₁255 and m₂255 represent the quantization levels of the target model grayscale feature and the LBP texture feature, respectively, function b₁Is located at xⁱTo a grey scale feature index, function b₂Is located at xⁱThe mapping of the pixel to the LBP texture feature index of (1) is a Delta function, C is a normalization coefficient, μ ═ 1.. 255, and v ═ 1.. 255.

(4) Respectively calculating Bhattacharyya coefficients rho of the gray feature and the LBP texture feature₁、ρ₂And a weight coefficient alpha₁、α₂Using rho₁、α₁、ρ₂、α₂Calculating the position y₀The combined feature Bhattacharyya coefficient ρ;

the joint feature Bhattacharyya coefficient ρ is described as:

ρ＝α₁·ρ₁+α₂·ρ₂，

gray scale feature weight coefficient alpha₁LBP texture feature alpha₂The update expression is as follows:

wherein alpha is_1,oldAnd alpha_2,oldIs the weight coefficient of the previous frame, α_1,curAnd alpha_2,curIs the current frame weight coefficient and λ is the scale coefficient.

(5) Calculating new target tracking position y of current frame₁；

Wherein,_n' is the number of candidate target pixel points, h is the kernel function bandwidth, q_u、q_v、p_u、p_v、α₁、α₂、x_i、b₁(·)、b₂The meaning of (. cndot.) is the same as defined in steps (2), (3) and (4).

(6) Bhattacharyya coefficient rho 'utilizing gray level feature and LBP texture feature'₁、ρ'₂And a weight coefficient of α'₁、α'₂Calculating the position y₁The coefficient rho' of the Bhattacharyya of the joint characteristic is expressed as follows;

ρ′＝α′₁·ρ′₁+α′₂·ρ′₂，

grayscale characteristic weight coefficient alpha'₁LBP texture feature alpha'₂The update expression is as follows:

wherein, alpha'_1,oldAnd alpha'_2,oldIs the last frame weight coefficient and λ is the scale coefficient.

(7) When rho'<At the time of the rho,otherwise y₁Keeping the same;

(8) if abs (y)₀-y₁)<0.01 then stop, otherwise, y₀←y₁And (5) executing the step (3).

Fig. 2 is a conventional technology, and fig. 3 is a target tracking result of an infrared image obtained according to the present embodiment using only single feature (grayscale feature) and multi-feature fusion, and grayscale colors inevitably appear because the infrared image is used. Fig. 2a, 2b, 2c, and 2d show images of the 20 th frame, the 80 th frame, the 140 th frame, and the 200 th frame, respectively, and fig. 3a, 3b, 3c, and 3d show images of the 20 th frame, the 80 th frame, the 140 th frame, and the 200 th frame, respectively. Comparing fig. 2 and fig. 3, it was found that: only using single characteristic to track the target can cause unstable tracking process and poor tracking precision, such as random swing of a tracking wave gate in fig. 2; the tracking precision can be effectively improved by using a multi-feature fusion tracking method, for example, a tracking wave gate in fig. 3 is always near a target centroid.

The present invention provides a real-time tracking method for infrared image targets with multi-feature fusion, and a plurality of methods and approaches for implementing the technical solution are provided, the above description is only a preferred embodiment of the present invention, it should be noted that, for those skilled in the art, a plurality of improvements and modifications may be made without departing from the principle of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention. All the components not specified in the present embodiment can be realized by the prior art.

Claims

1. A real-time tracking method of an infrared image target with multi-feature fusion is characterized by comprising the following steps:

(1) giving the position y of the initial tracking point of the target₀；

(2) Initializing the target model to initiate tracking point y₀Establishing a target gray model q for the center₁And a target LBP texture model q₂；

(3) Calculating a target candidate model according to the position y of the tracking point of the target₀Calculating a candidate target gray model p₁(y₀) And candidatesTarget LBP texture model p₂(y₀)；

(4) Utilizing the coefficient rho of the gray characteristic Bhattacharyya₁And the Bhattacharyya coefficient rho of the LBP texture feature₂And a weight coefficient alpha of the gradation characteristic₁And the weighting coefficient alpha of the LBP texture feature₂Calculating the position y₀And (3) processing the combined characteristic Bhattacharyya coefficient rho, wherein the expression is as follows:

ρ＝α₁·ρ₁+α₂·ρ₂；

(5) calculating the new position y of the target of the current frame₁；

ρ′＝α′₁·ρ′₁+α′₂·ρ′₂，

(7) when p' < p,otherwise y₁Keeping the same;

(8) if (y)₀-y₁) If l <, stop calculating, otherwise, divide y₁Assign y to₀And executing step (3), wherein the error constant coefficient is obtained;

α₁＝(1-λ)·α_1,old+λ·α_1,cur，

α₂＝(1-λ)·α_2,old+λ·α_2,cur，

α'₁＝(1-λ)·α'_1,old+λ·α'_1,cur，

α'₂＝(1-λ)·α'_2,old+λ·α'_2,cur，

wherein alpha is_1,oldAnd alpha_2,oldThe weight coefficients, alpha, of the gray feature and LBP texture feature of the previous frame in the step (4) are respectively_1,curAnd alpha_2,curRespectively are the weight coefficients, alpha ', of the gray feature and the LBP texture feature of the current frame in the step (4)'_1,oldAnd alpha'_2,oldRespectively are the weight coefficients, alpha ', of the gray feature and the LBP texture feature of the previous frame in the step (6)'_1,curAnd alpha'_2,curAnd (4) weighting coefficients of the gray feature and the LBP texture feature of the current frame in the step (6), wherein lambda is a proportionality coefficient.

2. The method for tracking the infrared image target with the fusion of the multiple features as claimed in claim 1, wherein in the step (2), the target gray model q is obtained₁Comprises the following steps:

target LBP texture model q₂Comprises the following steps:

3. The method for tracking the infrared image target with the fusion of the multiple features as claimed in claim 1, wherein in the step (3), the candidate target gray model p is₁Comprises the following steps:

candidate target LBP texture model p₂Comprises the following steps:

4. The method for tracking the infrared image target in real time through multi-feature fusion according to claim 1, wherein the weighting coefficient α of the gray features of the current frame in the step (4) is_1,curAnd the weighting coefficient alpha of the LBP texture feature_2,curAnd (5) weighting coefficient alpha 'of the gray level feature of the current frame in the step (6)'_1,curAnd weight coefficient of LBP texture feature of alpha'_2,curThe calculation formula is as follows:

5. The method for tracking the infrared image target in real time through multi-feature fusion according to claim 4, wherein in the step (6), the gray feature Bhattacharyya coefficient rho'₁And Bhattacharyya coefficient ρ 'of LBP texture feature'₂And a weight coefficient of a gray-scale feature of α'₁And weight coefficient of LBP texture feature of alpha'₂Obtained by the following formula:

p'_uis position y₁Of the target grayscale model grayscale feature of (d'_vIs position y₁At each level of probability density of the LBP texture features.