CN104200495A - Multi-target tracking method in video surveillance - Google Patents

Abstract

The invention discloses a video target tracking method capable of integrating ASIFT features and particle filter, and belongs to the technical field of video information processing and mode recognition. The video target tracking method comprises the following steps: the adjacent frame difference method is utilized to obtain moving objects in a video sequence; according to the area corresponding to an acquired complete target, a tracking target model is established; ASIFT feature vectors of the target model are established; the particle filter technology is adopted to predict a candidate area target; ASIFT feature vectors of a candidate target model are established; the feature vectors of the tracking targets are matched with the feature vectors of the candidate target; the RANSAC algorithm is adopted to reject wrong matching; the target model is renewed, so as to realize target tracking. The video target tracking method provided by the invention can accurately and quickly track the targets under the condition that brightness changes and is shielded. Therefore, the multi-target tracking method in the video surveillance has relatively good real-time performance and robustness.

Description

Multi-object tracking method in a kind of video monitoring

Technical field

The invention belongs to video information process and mode identification technology, specifically the multi-object tracking method in a kind of video monitoring.

Background technology

Target following is machine vision, artificial intelligence and area of pattern recognition basis always.Target following can be widely used in the industries such as navigator fix, military guidance, security monitoring.

Target following is to utilize known target position information and goal succession to find interested moving target on one section of sequence image.Have for the method for tracking target in video monitoring at present multiple, as the tracking based on particle filter, the tracking based on Mean Shift, the method for tracking target based on Kalman filtering etc.But in the time that target is blocked with surrounding environment existence interference (as noise, illumination), easily there is the phenomenons such as lose objects, tracking window depart from these traditional methods, causes following the tracks of unsuccessfully.

Utilize the method for video image being carried out to multiple target followings based on unique point to there is higher robustness, for example, based on SIFT (Scale Invariant Feature Transform, SIFT) the target following technology of feature, can, in the situation that rotation, change of scale and luminance transformation appear in target, still can carry out stable identification to target.But this technology does not possess higher anti-affinity,, for the larger target of deformation, easily there is track rejection in also Shortcomings in object matching precision.Moreover there is too defect in the method in real-time.

Summary of the invention

For above deficiency of the prior art, the object of the present invention is to provide the multi-object tracking method in a kind of video monitoring, with affine-yardstick invariant features conversion (Affine-SIFT, ASIFT) feature is described object module, then utilize particle filter method to search for moving target, finally by improved ASIFT matching algorithm, characteristic matching is carried out in target area, carry out object module renewal, realize target is followed the tracks of.Under illumination variation environment and target generation circumstance of occlusion, improve accuracy, robustness and the real-time of following the tracks of.

Multi-object tracking method in a kind of video monitoring of the present invention, detects moving target by neighbor frame difference method, and the moving target detecting is set up to tracking target model, and the ASIFT proper vector of establishing target model; Adopt particle filter predicting candidate regional aim, and set up the ASIFT proper vector of candidate target model, tracking target proper vector is mated with candidate region target feature vector, adopt RANSAC algorithm to reject erroneous matching, upgrade object module, realize target is followed the tracks of, and comprises the following steps:

Steps A: read video image initial frame, adopt neighbor frame difference method to detect the moving target in video sequence;

Read video image initial frame, the image respective pixel values of adjacent two frames in video carried out to difference calculating,

D _k(x,y)＝|f _k(x,y)-f _k+1(x,y)|

$D_k=\left\{\begin{array}{cc}0& D_k<T_0\\ 1& D_k\&GreaterEqual;T_0\end{array}\right.$

Wherein, f _k(x, y) is the image of present frame, f _k+1(x, y) is the next frame image that current frame image is adjacent; D _kbe the poor absolute values of two two field pictures, D _k=1 is motion target area; Wherein T ₀=0.7;

Step B: build tracking target model affine-yardstick invariant features conversion (Affine-SIFT, ASIFT) proper vector A; Concrete steps are:

Step B1, carries out affine transformation parameter to motion target area, in motion target area, angle of latitude θ is adopted to Geometric Sequence sampling: 1, a, a ²... a ⁿ, a>1, wherein n=5; To longitude angle carry out equal difference sampling: 0, b/t ... kb/t, wherein b=72 °, t=|1/cos θ |, k is kb/t<180 ° last integer of satisfying condition;

Step B2, carries out affined transformation to motion target area, utilizes the sequential parameter obtaining to calculate: $I^{\&prime;}(\mathrm{\&phi;},t)=\left[\begin{array}{cc}\cos \mathrm{\&phi;}& -\sin \mathrm{\&phi;}\\ \sin \mathrm{\&phi;}& \cos \mathrm{\&phi;}\end{array}\right]\&CenterDot;\left[\begin{array}{cc}t& 0\\ 0& 1\end{array}\right]\&CenterDot;I$

Wherein, I is motion target area, and I ' is the motion target area after affined transformation;

Step B3, carries out SIFT feature point detection to the motion target area after affine analog converting;

Step B4, the unique point of motion target area is carried out to vector description, build the ASIFT proper vectors of 128 dimensions;

Step B5, adopt major component component analysis method (PCA) to carry out space dimensionality reduction and obtain proper vector A to ASIFT proper vector;

Step C, reads next frame image;

Step D, adopts particle filter method to the image prediction candidate region target reading in step C, and builds the ASIFT proper vector B of candidate target model; Concrete steps are:

D1, to motion target area, M particle sample of random selection from one group of probability sample of former frame in t moment;

D2, carries out probability redistribution to the M newly a collecting particle;

D3, carrys out the weighted value of compute histograms according to RGB histogram to M particle, then M particle position is weighted to average computation according to weight, obtains the candidate region of tracking target;

D4, the ASIFT proper vector that builds candidate region obtains proper vector B;

Step e: motion target area proper vector A is mated with the ASIFT proper vector B of candidate region;

Step F: adopt random sample consistance RANSAC method to reject erroneous matching;

Step G: upgrade object module, return to step C, realize target is followed the tracks of.

Further: in described step B5, adopt major component component analysis method (PCA) to carry out space dimensionality reduction to ASIFT proper vector, concrete steps are:

B51, each ASIFT unique point of obtaining is described as respectively the vector of one 128 dimension, and using unique point as sample, writing out sample matrix is [x ₁, x ₂..., x _n] ^t, wherein n is unique point number, x _irepresent 128 dimensional feature vectors of i unique point;

B52, the averaged feature vector of n sample of calculating

B53, calculates the poor of the proper vector of all sample points and feature average vector, obtains difference value vector $d_i=x_i-\stackrel{\&OverBar;}{x},i=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;,n;$

B54, builds covariance matrix wherein Q=[d ₁, d ₂..., d _n];

B55, asks 128 eigenvalue λ i and 128 proper vector ei of covariance matrix;

B56, arranges λ by 128 eigenwerts obtaining by order from big to small ₁>=λ ₂>=...>=λ ₁₂₈with characteristic of correspondence vector (e ₁, e ₂... e ₁₂₈);

B57, chooses the direction of a corresponding m maximal eigenvector as major component;

B58, the matrix R of a 128*t of structure, its each row are made up of t proper vector;

B59, presses y 128 original dimension ASIFT feature descriptors _i=x _i* R projection, calculates the ASIFT feature descriptor y of 36 dimensions ₁, y ₂..., y _n, wherein, x _ifor the vector representation of the ASIFT unique point in original object region, y _ifor the vector representation of ASIFT unique point in target area after dimensionality reduction.

Further, in described step e, when the ASIFT proper vector of motion target area proper vector and candidate region is carried out to matching operation, adopt the approximate KNN searching method based on KD-Tree.

Beneficial effect of the present invention:

(1) adopt ASIFT feature matching method compare SIFT, SURF feature matching method, under the impact of target occlusion and environmental factor, more unique point can be detected, more stable in the time of target following, not lose objects easily.

(2) adopt PCA technology to reduce dimension processing to ASIFT proper vector, the vector of 128 dimensions is represented with 32 dimensional vectors, reduced calculated amount, more meet the real-time of target following.

(3) adopt the approximate KNN way of search based on KD-Tree to replace overall nearest neighbor search to mate with candidate region target feature vector tracking target proper vector, improved the search efficiency of matching characteristic point, reduced calculating consuming time.

(4) ASIFT feature matching method and particle filter after improving are merged, the region occurring at next frame by particle filter technology target of prediction model, has avoided ASIFT to mate whole two field picture, has improved degree of accuracy.

Compared with existing scheme, method of the present invention tracking target quickly and accurately under brightness variation, circumstance of occlusion, has good real-time and robustness.

Brief description of the drawings

Fig. 1 is the multi-object tracking method process flow diagram in a kind of video monitoring of the present invention;

Embodiment

In conjunction with Fig. 1, the multi-object tracking method in a kind of video monitoring, detects moving target by neighbor frame difference method, and the moving target detecting is set up to tracking target model, and the ASIFT proper vector of establishing target model; Adopt particle filter predicting candidate regional aim, and set up the ASIFT proper vector of candidate target model, tracking target proper vector is mated with candidate region target feature vector, adopt RANSAC algorithm to reject erroneous matching, upgrade object module, realize target is followed the tracks of, and comprises the following steps:

Steps A: read video image initial frame, adopt neighbor frame difference method to detect the moving target in video sequence; The video that the video image reading collects for monitoring camera.

Read video image initial frame, the image respective pixel values of adjacent two frames in video carried out to difference calculating,

D _k(x,y)＝|f _k(x,y)-f _k+1(x,y)|

$D_k=\left\{\begin{array}{cc}0& D_k<T_0\\ 1& D_k\&GreaterEqual;T_0\end{array}\right.$

Wherein, f _k(x, y) is the image of present frame, x, and y represents respectively horizontal ordinate and the ordinate of pixel, f _k+1(x, y) is the next frame image that current frame image is adjacent; D _kbe the poor absolute values of two two field pictures, represent moving region, D _k=1 is motion target area; Wherein T ₀for binaryzation threshold values, binaryzation threshold values T of the present invention ₀=0.7, according to different requirements, also can take other value;

Calculate through above, in figure, pixel value only has 0 and 1 two kind, and the pixel region that value is 1 is target corresponding region, and by this mode, the motion target area in video sequence can be out divided.

Step B: build tracking target model affine-yardstick invariant features conversion (Affine-SIFT, ASIFT)

Proper vector A; Concrete steps are:

Step B1, carries out affine transformation parameter to motion target area, in motion target area, angle of latitude θ is adopted to Geometric Sequence sampling: 1, a, a ²... a ⁿ, a>1, wherein n=5; To longitude angle carry out equal difference sampling: 0, b/t ... kb/t, wherein b=72 °, t=|1/cos θ |, k is kb/t<180 ° last integer of satisfying condition;

Wherein, parameter θ and the angle of latitude of representative shooting camera optical axis and the longitude angle of camera optical axis respectively.Generally can there is affine deformation to a certain degree in target area, mainly caused by the conversion of camera light direction of principal axis, and optical axis direction conversion depend on parameter θ and before affine simulation is carried out in target area, need parameter θ and carry out resampling.

The parameter θ that motion target area is obtained is as shown in table 1.

Table 1

To the parameter of motion target area sampling interval be set as: and sample range be [0,180 °].In the time of t=1, parameter concrete sampled value is: 0,72 °, and 144 °.

Step B2, carries out affined transformation to motion target area, utilizes the sequential parameter obtaining to calculate: $I^{\&prime;}(\mathrm{\&phi;},t)=\left[\begin{array}{cc}\cos \mathrm{\&phi;}& -\sin \mathrm{\&phi;}\\ \sin \mathrm{\&phi;}& \cos \mathrm{\&phi;}\end{array}\right]\&CenterDot;\left[\begin{array}{cc}t& 0\\ 0& 1\end{array}\right]\&CenterDot;I$

Wherein, I is motion target area, and I ' is the motion target area after affined transformation;

Step B3, carries out SIFT feature point detection to the motion target area after affine analog converting;

Step B4, the unique point of motion target area is carried out to vector description, build the ASIFT proper vectors of 128 dimensions;

Step B5, adopt major component component analysis method (PCA) to carry out space dimensionality reduction and obtain proper vector A to ASIFT proper vector;

Step C, reads next frame image;

Step D, adopts particle filter method to the image prediction candidate region target reading in step C, and builds the ASIFT proper vector B of candidate target model; Concrete steps are:

D1, to motion target area, M particle sample of random selection from one group of probability sample of former frame in t moment;

D2, carries out probability redistribution to the M newly a collecting particle;

If the movement velocity of t-1 moment tracking target is:

$\mathrm{vec}\stackrel{\&OverBar;}{x}=\frac{\mathrm{\&Delta;}\stackrel{\&OverBar;}{x}}{\mathrm{vecunitperpixel}}$

$\mathrm{vec}\stackrel{\&OverBar;}{y}=\frac{\mathrm{\&Delta;}\stackrel{\&OverBar;}{y}}{\mathrm{vecunitperpixel}}$

with represent respectively the position skew of t-1 moment motion target area, vecuniteperpixel represents the motor unit of each pixel.

Can obtain the reposition of t each particle of moment by formula below:

$x_t^i=x_t^i+r_t^i\&times;\mathrm{vec}\stackrel{\&OverBar;}{x}\&times;\mathrm{vecunitperpixel}+r_t^i\&times;H_{t-1}^i$

$y_t^i=y_t^i+r_t^i\&times;\mathrm{vec}\stackrel{\&OverBar;}{x}\&times;\mathrm{vecunitperpixel}+r_t^i\&times;W_{t-1}^i$

Wherein, for Gaussian number, for particle is high, for particle wide.

D3, carrys out the weighted value of compute histograms according to RGB histogram to M particle, then M particle position is weighted to average computation according to weight, obtains the candidate region of tracking target;

Computing formula is as follows:

${\stackrel{\&OverBar;}{x}}_t=f\underset{i=1}{\overset{M}{\mathrm{\&Sigma;}}}{W}^i\&times;x_t^i$

${\stackrel{\&OverBar;}{y}}_t=f\underset{i=1}{\overset{M}{\mathrm{\&Sigma;}}}{W}^i\&times;y_t^i$

Wherein, f is normalization coefficient, w ⁱfor the weight of each particle.

Calculate behind tracking target estimated position, with t-1 moment initial position 3x3 pixel rectangular extent around, form 10 searching positions, search therein a new position, making with the quadratic sum (SSD) of previous frame t-1 moment target area gray scale difference is minimum, the reposition with this new position as moving target.

S(x,y)＝(∫∫ _w|(J(X)-I(X)|) (11)

$(x_t,y_t)=\min (S(x,y),S(\stackrel{\&OverBar;}{x},\stackrel{\&OverBar;}{y})),(x_m+1,y_m+1)\&GreaterEqual;x,y\&GreaterEqual;(x_m-1,y_m-1)$

Wherein, S represents the brightness of this position and the luminance difference of template; X, y is for being illustrated in x _m, y _mcentered by reposition.J, I represent respectively the luminance function of the two width images in t-1 and t moment.

Variable M=150 in step D1-D3.

D4, the ASIFT proper vector that builds candidate region obtains proper vector B;

By the method for step B, build equally the ASIFT proper vector of candidate target model, and adopt major component component analysis technology (PCA) to carry out space dimensionality reduction, the ASIFT unique point of final candidate target region also adopts 36 dimensional vectors to represent.

Step e: motion target area proper vector A is mated with the ASIFT proper vector B of candidate region;

Step F: adopt random sample consistance RANSAC method to reject erroneous matching;

Step G: upgrade object module, return to step C, realize target is followed the tracks of.

Further: in described step B5, adopt major component component analysis method (PCA) to carry out space dimensionality reduction to ASIFT proper vector, concrete steps are:

B51, each ASIFT unique point of obtaining is described as respectively the vector of one 128 dimension, and using unique point as sample, writing out sample matrix is [x ₁, x ₂..., x _n] ^t, wherein n is unique point number, x _irepresent 128 dimensional feature vectors of i unique point;

B52, the averaged feature vector of n sample of calculating

B53, calculates the poor of the proper vector of all sample points and feature average vector, obtains difference value vector $d_i=x_i-\stackrel{\&OverBar;}{x},i=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;,n;$

B54, builds covariance matrix wherein Q=[d ₁, d ₂..., d _n];

B55, asks 128 eigenvalue λ of covariance matrix _iwith 128 proper vector e _i;

B56, arranges λ by 128 eigenwerts obtaining by order from big to small ₁>=λ ₂>=...>=λ ₁₂₈with characteristic of correspondence vector (e ₁, e ₂... e ₁₂₈);

B57, chooses the direction of a corresponding m maximal eigenvector as major component;

B58, the matrix R of a 128*t of structure, its each row are made up of t proper vector;

B59, presses y 128 original dimension ASIFT feature descriptors _i=x _i* R projection, calculates the ASIFT feature descriptor y of 36 dimensions ₁, y ₂..., y _n, wherein, x _ifor the vector representation of the ASIFT unique point in original object region, y _ifor the vector representation of ASIFT unique point in target area after dimensionality reduction.

Wherein, x _ifor the vector representation of the ASIFT unique point in original object region.Y _ifor the vector representation of ASIFT unique point in target area after dimensionality reduction.

Further, in described step e, when the ASIFT proper vector of motion target area proper vector and candidate region is carried out to matching operation, adopt the approximate KNN searching method based on KD-Tree.

Calculation procedure is:

(1) set up KD-Tree according to ASIFT unique point, specific implementation step is as follows

A, determine the value in split territory;

Put at x by calculated characteristics, the data variance in y dimension, gets the dimension of variance yields maximum as the value in split territory;

B, determine Node-data territory;

According to the value in the split territory obtaining, characteristic point data is sorted in this dimension, obtain Node-data numeric field data point according to the intermediate value of data, like this, just determine cutting apart of this node of super face;

C, determine left and right subspace;

Cut apart super face whole space is divided into two parts, the point of cutting apart the super face left side is left subspace, and the point of cutting apart super face the right is right subspace.

D, then can obtain one-level child node according to He You subspace, left subspace, more respectively by space and data set further segmentation again, until only comprise a data point in space.

(2) by binary tree search, retrieval in KD-Tree with the approximate point of query point apart from neighbour;

(3) according to contrasting with adjacent other unique points, find with query point Euclidean distance nearest front two

Individual unique point;

(4) nearest Euclidean distance is removed near Euclidean distance in proper order, if this value is less than certain proportion threshold value γ, accept this pair of match point, Feature Points Matching success, otherwise, mate unsuccessful.

$\frac{d_1}{d_2}<\mathrm{\&gamma;}$

Wherein d ₁be two Euclidean distances that unique point to be matched is nearest; d ₂be two unique points to be matched time near Euclidean distances.Threshold gamma=0.8 is set in the present invention.。

Judgement, whether tracking target proper vector mates successful with candidate region target feature vector, if success performs step five.Otherwise, return to execution step (3).

Embodiments of the invention are interpreted as being only not used in and limiting the scope of the invention for the present invention is described.After having read the content of record of the present invention, technician can make various changes or modifications the present invention, and these equivalences change and modification falls into the scope of the claims in the present invention equally.

Claims (3)

1. the multi-object tracking method in video monitoring, detects moving target by neighbor frame difference method, and the moving target detecting is set up to tracking target model, and the ASIFT proper vector of establishing target model; Adopt particle filter predicting candidate regional aim, and set up the ASIFT proper vector of candidate target model, tracking target proper vector is mated with candidate region target feature vector, adopt RANSAC algorithm to reject erroneous matching, upgrade object module, realize target is followed the tracks of, and comprises the following steps:

Steps A: read video image initial frame, adopt neighbor frame difference method to detect the moving target in video sequence;

Read video image initial frame, the image respective pixel values of adjacent two frames in video carried out to difference calculating,

D _k(x,y)＝|f _k(x,y)-f _k+1(x,y)|

$D_k=\left\{\begin{array}{cc}0& D_k<T_0\\ 1& D_k\&GreaterEqual;T_0\end{array}\right.$

Wherein, f _k(x, y) is the image of present frame, f _k+1(x, y) is the next frame image that current frame image is adjacent; D _kbe the poor absolute values of two two field pictures, D _k=1 is motion target area; Wherein T ₀=0.7;

Step B: build tracking target model affine-yardstick invariant features conversion (Affine-SIFT, ASIFT) proper vector A; Concrete steps are:

Step B1, carries out affine transformation parameter to motion target area, in motion target area, angle of latitude θ is adopted to Geometric Sequence sampling: 1, a, a ²... a ⁿ, a>1, wherein n=5; To longitude angle carry out equal difference sampling: 0, b/t ... kb/t, wherein b=72 °, t=|1/cos θ |, k is kb/t<180 ° last integer of satisfying condition;

Step B2, carries out affined transformation to motion target area, utilizes the sequential parameter obtaining to calculate: $I^{\&prime;}(\mathrm{\&phi;},t)=\left[\begin{array}{cc}\cos \mathrm{\&phi;}& -\sin \mathrm{\&phi;}\\ \sin \mathrm{\&phi;}& \cos \mathrm{\&phi;}\end{array}\right]\&CenterDot;\left[\begin{array}{cc}t& 0\\ 0& 1\end{array}\right]\&CenterDot;I$

Wherein, I is motion target area, and I ' is the motion target area after affined transformation;

Step B3, carries out SIFT feature point detection to the motion target area after affine analog converting;

Step B4, the unique point of motion target area is carried out to vector description, build the ASIFT proper vectors of 128 dimensions;

Step B5, adopt major component component analysis method (PCA) to carry out space dimensionality reduction and obtain proper vector A to ASIFT proper vector;

Step C, reads next frame image;

Step D, adopts particle filter method to the image prediction candidate region target reading in step C, and builds the ASIFT proper vector B of candidate target model; Concrete steps are:

D1, to motion target area, M particle sample of random selection from one group of probability sample of former frame in t moment;

D2, carries out probability redistribution to the M newly a collecting particle;

D3, carrys out the weighted value of compute histograms according to RGB histogram to M particle, then M particle position is weighted to average computation according to weight, obtains the candidate region of tracking target;

D4, the ASIFT proper vector that builds candidate region obtains proper vector B;

Step e: motion target area proper vector A is mated with the ASIFT proper vector B of candidate region;

Step F: adopt random sample consistance RANSAC method to reject erroneous matching;

Step G: upgrade object module, return to step C, realize target is followed the tracks of.

2. the multi-object tracking method in video monitoring according to claim 1, is characterized in that:

In step B5, adopt major component component analysis method (PCA) to carry out space dimensionality reduction to ASIFT proper vector, concrete steps are:

B51, each ASIFT unique point of obtaining is described as respectively the vector of one 128 dimension, and using unique point as sample, writing out sample matrix is [x ₁, x ₂..., x _n] ^t, wherein n is unique point number, x _irepresent 128 dimensional feature vectors of i unique point;

B52, the averaged feature vector of n sample of calculating

B53, calculates the poor of the proper vector of all sample points and feature average vector, obtains difference value vector $d_i=x_i-\stackrel{\&OverBar;}{x},i=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;,n;$

B54, builds covariance matrix wherein Q=[d ₁, d ₂..., d _n];

B55, asks 128 eigenvalue λ of covariance matrix _iwith 128 proper vector e _i;

B56, arranges λ by 128 eigenwerts obtaining by order from big to small ₁>=λ ₂>=...>=λ ₁₂₈with characteristic of correspondence vector (e ₁, e ₂... e ₁₂₈);

B57, chooses the direction of a corresponding m maximal eigenvector as major component;

B58, the matrix R of a 128*t of structure, its each row are made up of t proper vector;

B59, presses y 128 original dimension ASIFT feature descriptors _i=x _i* R projection, calculates the ASIFT feature descriptor y of 36 dimensions ₁, y ₂..., y _n, wherein, x _ifor the vector representation of the ASIFT unique point in original object region, y _ifor the vector representation of ASIFT unique point in target area after dimensionality reduction.

3. the multi-object tracking method in video monitoring according to claim 1, it is characterized in that: in step e, when the ASIFT proper vector of motion target area proper vector and candidate region is carried out to matching operation, adopt the approximate KNN searching method based on KD-Tree.