CN105354542A - Method for detecting abnormal video event in crowded scene - Google Patents

Abstract

The present invention relates to a method for detecting an abnormal video event in a crowded scene. The method specifically comprises: in a training phase, firstly dividing a video into blocks, extracting optical flow information of the blocks, using the extracted optical flow information to represent local features of the blocks, using the local features of the blocks as elements to construct a map, reducing dimensions through Laplace feature mapping, classifying the local features by using an adaptive clustering method, acquiring class centers as code words, and finally forming a codebook by using the code words; and in a test phase, firstly extracting local features of blocks of a video, then calculating a feature distance similarity between the local features and the codebook, and detecting an event in conjunction with previous 8-neighborhood information. According to the method, a normal-event model can be established only by clustering elements sets comprised of optical flow features, and the neighborhood information of historical moments further increases the accuracy of detecting an abnormal event.

Description

A kind of video accident detection method under crowd scene

Technical field

The invention belongs to Intelligent Video Surveillance Technology field, relate to a kind of video accident detection method under crowd scene.

Background technology

Video accident detection refers to event in energy automatic analysis video monitoring scene, if there is anomalous event just can send alerting signal at once, thus improves response and the rescue efficiency of relevant departments.Such as, on shopping mall by bike, skidding and vehicle pass-through, on square crowd fear, these anomalous events such as to trample and can be detected timely and report to the police.It has a wide range of applications in field of video monitoring.

Current existing accident detection method can be roughly divided into two classes: 1) by following the tracks of destination object, analyzing its movement locus, and then judging it; 2) not needing to follow the tracks of destination object, by setting up normal event model, analysis being made to light stream, Texture eigenvalue.First kind method, by following the tracks of destination object, obtain direction of motion with the movement velocity of target and according to destination object feature and size ratio, these class methods obtain good application for when only there is minority moving target in scene, but under crowded environment, due to overlapped between target, be difficult to follow the tracks of destination object, therefore the detection perform of these class methods is not good enough.The present invention adopts Equations of The Second Kind method.

Summary of the invention

The object of this invention is to provide a kind of video accident detection method under crowd scene, to improve accident detection rate.

For solving the problem, technical scheme provided by the invention is as follows:

Step (1) feature extraction, specific as follows:

Each frame in video is divided into non-overlapping copies and size is the block of N × N, and gets M frame continuously, obtain the stereo block that size is N × N × M, therefore the video of M frame length is made up of several stereo block, and each stereo block is called atom.The resolution sizes of setting video frame is W × H, and each frame obtains the quantity of block wherein represent and round downwards.The movable information of t loci block of locations represents with a histogram 1≤loci≤f_block, loci is integer, wherein h _i, 1≤i≤4, be loci block of locations according to light stream direction by the light stream amplitude sum on 4 directions of 90 degree of interval quantizing acquisitions.Current time is t, and in conjunction with the histogram information of (M-1)/2 frame before t and rear (M-1)/2 frame, the atom on t loci position is expressed as m gets odd number, 1≤loci≤f_block.For one section of video, be divided into P segment in units of M frame, the atomic quantity obtained is P × f_block, by the atom set of these these videos of atomic building.

Step (2) feature learning, specific as follows:

2-1. adopts Laplacian Eigenmap method that atom set is mapped in lower dimensional space, then carries out cluster to it.First to atom collection design of graphics G=(V, E), vertex set V represents each atom, the limit E of Weight represents the similarity between each atom, in figure, between i-th atom and a jth atom, the weight on limit calculates by formula (1), 1≤i≤P × f_block, 1≤j≤P × f_block:

$w_{ij}=\exp \left(\frac{-(1-d_f^{i,j})}{{\mathrm{\σ}}_i{\mathrm{\σ}}_j}\right)\exp \left(\frac{-d_s^{i.j}}{{\mathrm{\σ}}_s}\right)---\left(1\right)$

In the Section 1 of formula (1) the right mathematic(al) representation be:

$d_f^{i,j}={\mathrm{\α}}^{i,j}\×d_f(x_i,x_j)---\left(2\right)$

COS distance wherein <x _i, x _j> represents and asks x _iwith x _jinner product, definition $d_f(x_i,x_j)=\frac{\underset{k=0}{\overset{4M}{\&Sigma;}}\min \&lsqb;x_i\left(k\right),x_j\left(k\right)\&rsqb;}{\max \&lsqb;\underset{k=0}{\overset{4M-1}{\&Sigma;}}{x}_i\left(k\right),\underset{k=0}{\overset{4M-1}{\&Sigma;}}{x}_j\left(k\right)\&rsqb;};{\mathrm{\&sigma;}}_r=\sqrt{(1-d_f^{r,G})},$ form when being exactly i=r, j=G in formula (2), σ _rscale factor, wherein x _gx _rg Neighbor Points, the distance metric of neighbour adopts Euclidean distance, r=i or j.

In the Section 2 of formula (1) the right, represent the Euclidean distance on i-th atom and a jth atomic space; σ _sfor the space scale factor.After having built figure, by means of figure, spectral clustering is carried out to atom.In graph theory, the problem of cluster changes the problem that figure cuts into.Its principle is that the limit weight maximization in subgraph and the limit weight between each subgraph minimize.Cut-off limit weight sum is minimum, even if the minimization of object function shown in formula (3).

$E\left(Y\right)=\underset{ij}{\&Sigma;}{w}_{ij}\left|\right|y_i-y_j|{|}^2---\left(3\right)$

Wherein w _ijprovided by formula (1); y _iand y _jx respectively _iand x _jbe mapped to the coordinate vector on object space, Y is by vectorial y _icomposition, 1≤i≤P × f_block.The problem equivalent of the minimization of object function is in solving optimum Y:

Y _opt=argmin (YLY) s.t.Y ^tin DY=1 (4) formula (4), Laplacian Matrix L=D-W.D is diagonal matrix, element value d on its diagonal line _ii=Σ _jw _ij; W is by w _ijform.Calculate L relative to the generalized eigenvalue of D and proper vector, choose l minimum non-zero eigenwert and characteristic of correspondence is vectorial.The l a tried to achieve proper vector is become the characteristic vector space of (P × f_block) × l, and wherein every a line represents the coordinate of atom at l dimension space.Finally according to the l dimension space coordinate of each atom, adopt the adaptive clustering scheme (LihiZelnik-Manor of LihiZelnik-Manor, Self-TuningSpectralClustering.InProceedingsofthe18thAnnu alConferenceonNeuralInformationProcessingSystems2004) at l dimension space, cluster is carried out to atom, the class number of self-adaptation determination cluster.Obtain Num0 class after cluster, each Lei Lei center represents the local feature of a class event, and class center calculates by formula (5):

${\mathrm{\&mu;}}_k=\frac{1}{N_k}\underset{i=1}{\overset{N_k}{\&Sigma;}}{x}_i---\left(5\right)$

N _krepresent the atom number belonging to kth class event.Class center is as code word, and on the loci of position, all possible code word forms code book.

The process that 2-2. sets up code book in loci block of locations is as follows:

A () sets up initial codebook, feature learning training data being carried out step 2-1 obtains Num class event, Num=Num0.W is calculated by formula (6) _{k, loci}if, w _{k, loci}>0, then add to kLei Lei center in the code book on the loci of position as code word, preserves the training data of kth class local feature, k={1,2 simultaneously ... ... ..Num0};

$w_{k,loci}=\frac{n_{k,loci}}{{\mathrm{\&Sigma;}}_k{n}_{k,loci}},k=\{1,2,\mathrm{...........}Num0\}---\left(6\right)$

Wherein n _{k, loci}represent the number of times that on the loci of position, kth class event occurs, 1≤loci≤f_block.

B () inputs new training data it is compared with the code book in loci block of locations, if meet characteristic distance similarity time, th is the threshold value of setting; Will join in a new set U, otherwise handle to join on the loci of position in the training data of most Similarity Class local feature, then recalculate the class center of this kind of local feature, upgrade code word in code book, wherein it is the similarity of the characteristic distance calculated by formula (7);

$d_{max}\left(x_{loci}^{new}\right)=\underset{k}{max}\left(d_f\right(x_{loci}^{new},c_{loci}^k\left)\right)---\left(7\right)$

Wherein represent a kth code word in the code book on the loci of position, 1≤k≤Num.

C (), when gathering the data bulk in U and not reaching Q, returns step (b); When gathering the data bulk in U and reaching Q, again carry out step 2-1 operation to set U, be clustered into Num1 class, renewal Num0 is Num0=Num1, if w _{k, loci}>0, then just add to kth class event class center in the code book on the loci of position as code word, preserves the training data of kth class local feature, k={1,2 simultaneously ... ... ..Num0}; Empty set U, classification sum Num is updated to Num=Num+Num1, judges the training data of all inputs whether train complete, if not complete, return step (b).

Step (3) video accident detection, specifically:

By test data the code book of the loci block of locations set up with the training stage compares, 1≤loci≤f_block, if meet time, then tentatively think loci block of locations there is anomalous event to occur, otherwise does not occur;

Step (4) space-time aftertreatment, specifically:

In t, if being initially judged to of loci block of locations has anomalous event to occur, then consider the 8-neighborhood of loci block of locations in the t-1 moment, if it is abnormal to have at least two positions to exist in neighborhood, then sentencing in loci block of locations has anomalous event to occur, otherwise does not have anomalous event to occur.

The atom set that the present invention only need be formed Optical-flow Feature carries out spectral clustering and can set up normal event model.Space-time aftertreatment considers the neighborhood information of the historical juncture of judgement position, further increases the accuracy of accident detection.

Accompanying drawing explanation

Fig. 1 is accident detection model training procedure chart;

Fig. 2 is accident detection model inspection procedure chart.

Embodiment

Below in conjunction with accompanying drawing and embodiment, the present invention is described in detail.

Respectively as shown in Figure 1 and Figure 2, concrete steps are as follows for abnormality detection model training of the present invention and testing process:

Step (1) feature extraction, specifically:

Each frame in video is divided into non-overlapping copies and size is the block of N × N, and gets M frame continuously, obtain the stereo block that size is N × N × M, therefore the video of M frame length can be made up of several stereo block, and each stereo block is called atom, gets N=10, M=3.The resolution sizes of setting video frame is 320 × 240, and each frame obtains the quantity f_block=768 of block.The movable information of t loci block of locations represents with a histogram 1≤loci≤f_block, loci is integer, wherein h _i, 1≤i≤4, be loci block of locations according to light stream direction by the light stream amplitude sum on 4 directions of 90 degree of interval quantizing acquisitions.Current time is t, and in conjunction with front 1 frame of t and the histogram information of rear 1 frame, the atom on t loci position can be expressed as get P=10, the atomic quantity obtained is P × f_block=7680, by the atom set of these these videos of atomic building.

Step (2) feature learning, specific as follows:

2-1. adopts Laplacian Eigenmap method that atom set is mapped in lower dimensional space, then carries out cluster to it.First to atom collection design of graphics G=(V, E), vertex set V represents each atom, the limit E of Weight represents the similarity between each atom, in figure, between i-th atom and a jth atom, the weight on limit calculates by formula (1), 1≤i≤P × f_block, 1≤j≤P × f_block.Get G=7, σ _s=0.4.

After having built figure, by means of figure, spectral clustering is carried out to atom.Make the minimization of object function shown in formula (3).The problem equivalent of the minimization of object function is in the optimum solving formula (4).Calculate L relative to the generalized eigenvalue of D and proper vector, choose l minimum non-zero eigenwert and characteristic of correspondence is vectorial, get l=10.The l a tried to achieve proper vector is become the characteristic vector space of (P × f_block) × l, and wherein every a line represents the coordinate of atom at l dimension space.Finally according to the l dimension space coordinate of each atom, the adaptive clustering scheme of LihiZelnik-Manor is adopted to carry out cluster at l dimension space to atom, the class number of self-adaptation determination cluster.Obtain Num0 class after cluster, each Lei Lei center represents the local feature of a class event, and class center calculates by formula (5).

The process that 2-2. sets up code book on the loci of position is as follows, 1≤loci≤f_block:

A () sets up initial codebook, feature learning training data being carried out step 2-1 obtains Num class event, Num=Num0.W is calculated by formula (6) _{k, loci}if, w _{k, loci}>0, then add to kLei Lei center in the code book on the loci of position as code word, preserves the training data of kth class local feature, k={1,2 simultaneously ... ... ..Num0};

$w_{k,loci}=\frac{n_{k,loci}}{{\mathrm{\&Sigma;}}_k{n}_{k,loci}},k=\{1,2,\mathrm{...........}Num0\}---\left(6\right)$

Wherein n _{k, loci}represent the number of times that on the loci of position, kth class event occurs, 1≤loci≤f_block.

B () inputs new training data it is compared with the code book in loci block of locations, if meet characteristic distance similarity time, th is the threshold value of setting, gets th=0.85, will join in a new set U, otherwise handle to join on the loci of position in the training data of most Similarity Class local feature, then recalculate the class center of this kind of local feature, upgrade code word in code book, wherein it is the similarity of the characteristic distance calculated by formula (7);

$d_{max}\left(x_{loci}^{new}\right)=\underset{k}{max}\left(d_f\right(x_{loci}^{new},c_{loci}^k\left)\right)---\left(7\right)$

Wherein represent a kth code word in the code book on the loci of position, 1≤k≤Num.

C (), when gathering the data bulk in U and not reaching Q, returns step (b); When gathering the data bulk in U and reaching Q, again carry out step 2-1 operation to set U, be clustered into Num1 class, renewal Num0 is Num0=Num1, if w _{k, loci}>0, then just add to kth class event class center in the code book on the loci of position as code word, preserves the training data of kth class local feature, k={1,2 simultaneously ... ... ..Num0}; Empty set U, classification sum Num is updated to Num=Num+Num1, judges the training data of all inputs whether train complete, if not complete, return step (b).

Step (3) video accident detection, specifically:

By test data the code book of the loci block of locations set up with the training stage compares, 1≤loci≤f_block, if meet time, then tentatively think loci block of locations there is anomalous event to occur, otherwise does not occur;

Step (4) space-time aftertreatment, specifically:

In t, if being initially judged to of loci block of locations has anomalous event to occur, then consider the 8-neighborhood of loci block of locations in the t-1 moment, if it is abnormal to have at least two positions to exist in neighborhood, then sentencing in loci block of locations has anomalous event to occur, otherwise does not have anomalous event to occur.

Claims (1)

1. a video accident detection method under crowd scene, is characterized in that comprising the steps:

Step (1) feature extraction, specific as follows:

Each frame in video is divided into non-overlapping copies and size is the block of N × N, and gets M frame continuously, obtain the stereo block that size is N × N × M, therefore the video of M frame length is made up of several stereo block, and each stereo block is called atom; The resolution sizes of setting video frame is W × H, and each frame obtains the quantity of block wherein represent and round downwards; The movable information of t loci block of locations represents with a histogram 1≤loci≤f_block, loci is integer, wherein h _i, 1≤i≤4, be loci block of locations according to light stream direction by the light stream amplitude sum on 4 directions of 90 degree of interval quantizing acquisitions; Current time is t, and in conjunction with the histogram information of (M-1)/2 frame before t and rear (M-1)/2 frame, the atom on t loci position is expressed as $x_{loci}=\&lsqb;h_{loci}^{t-(M-1)/2}...h_{loci}^{t+(M-1)/2}\&rsqb;,$ M gets odd number, 1≤loci≤f_block; For one section of video, be divided into P segment in units of M frame, the atomic quantity obtained is P × f_block, by the atom set of these these videos of atomic building;

Step (2) feature learning, specific as follows:

2-1. adopts Laplacian Eigenmap method that atom set is mapped in lower dimensional space, then carries out cluster to it; First to atom collection design of graphics G=(V, E), vertex set V represents each atom, the limit E of Weight represents the similarity between each atom, in figure, between i-th atom and a jth atom, the weight on limit calculates by formula (1), 1≤i≤P × f_block, 1≤j≤P × f_block:

$w_{ij}=\exp \left(\frac{-(1-d_f^{i,j})}{{\mathrm{\&sigma;}}_i{\mathrm{\&sigma;}}_j}\right)\exp \left(\frac{-d_s^{i.j}}{{\mathrm{\&sigma;}}_s}\right)---\left(1\right)$

In the Section 1 of formula (1) the right mathematic(al) representation be:

$d_f^{i,j}={\mathrm{\&alpha;}}^{i,j}\&times;d_f(x_i,x_j)---\left(2\right)$

COS distance wherein <x _i, x _j> represents and asks x _iwith x _jinner product, definition $d_f(x_i,x_j)=\frac{\underset{k=0}{\overset{4M-1}{\&Sigma;}}min\&lsqb;x_i\left(k\right),x_j\left(k\right)\&rsqb;}{\max \&lsqb;\underset{k=0}{\overset{4M-1}{\&Sigma;}}{x}_i\left(k\right),\underset{k=0}{\overset{4M-1}{\&Sigma;}}{x}_j\left(k\right)\&rsqb;};{\mathrm{\&sigma;}}_r=\sqrt{(1-d_f^{r,G})},$ σ _rscale factor, wherein x _gx _rg Neighbor Points, the distance metric of neighbour adopts Euclidean distance, r=i or j;

In the Section 2 of formula (1) the right, represent the Euclidean distance on i-th atom and a jth atomic space; σ _sfor the space scale factor; After having built figure, by means of figure, spectral clustering is carried out to atom; In graph theory, the problem of cluster changes the problem that figure cuts into; Its principle is that the limit weight maximization in subgraph and the limit weight between each subgraph minimize; Cut-off limit weight sum is minimum, even if the minimization of object function shown in formula (3);

$E\left(r\right)=\underset{ij}{\&Sigma;}{w}_{ij}\left|\right|y_i-y_j|{|}^2---\left(3\right)$

Wherein w _ijprovided by formula (1); y _iand y _jx respectively _iand x _jbe mapped to the coordinate vector on object space, Y is by vectorial y _icomposition, 1≤i≤P × f_block; The problem equivalent of the minimization of object function is in solving optimum Y:

Y _opt＝argmin(YLY)s.t.Y ^TDY＝1(4)

In formula (4), Laplacian Matrix L=D-W; D is diagonal matrix, element value d on its diagonal line _ii=∑ _jw _ij; W is by w _ijform; Calculate L relative to the generalized eigenvalue of D and proper vector, choose l minimum non-zero eigenwert and characteristic of correspondence is vectorial; The l a tried to achieve proper vector is become the characteristic vector space of (P × f_block) × l, and wherein every a line represents the coordinate of atom at l dimension space; Finally according to the l dimension space coordinate of each atom, the adaptive clustering scheme of LihiZelnik-Manor is adopted to carry out cluster at l dimension space to atom, obtain Num0 class after cluster, each Lei Lei center represents the local feature of a class event, and class center calculates by formula (5):

${\mathrm{\&mu;}}_k=\frac{1}{N_k}\underset{i=1}{\overset{N_k}{\&Sigma;}}{x}_i---\left(5\right)$

N _krepresent the atom number belonging to kth class event; Class center is as code word, and on the loci of position, all possible code word forms code book;

The process that 2-2. sets up code book in loci block of locations is as follows:

A () sets up initial codebook, feature learning training data being carried out step 2-1 obtains Num class event, Num=Num0; W is calculated by formula (6) _{k, loci}if, w _{k, loci}>0, then add to kLei Lei center in the code book on the loci of position as code word, preserves the training data of kth class local feature, k={1,2 simultaneously ... ... ..Num0};

$w_{k,loci}=\frac{n_{k,loci}}{{\mathrm{\&Sigma;}}_k{n}_{k,loci}},k=\{1,2,.........\&CenterDot;\&CenterDot;Num0\}---\left(6\right)$

Wherein n _{k, loci}represent the number of times that on the loci of position, kth class event occurs, 1≤loci≤f_block;

B () inputs new training data it is compared with the code book in loci block of locations, if meet characteristic distance similarity time, th is the threshold value of setting; Will join in a new set U, otherwise handle to join on the loci of position in the training data of most Similarity Class local feature, then recalculate the class center of this kind of local feature, upgrade code word in code book, wherein it is the similarity of the characteristic distance calculated by formula (7);

$d_{max}\left(x_{loci}^{new}\right)=\underset{k}{max}\left(d_f\right(x_{loci}^{new},c_{loci}^k\left)\right)---\left(7\right)$

Wherein represent a kth code word in the code book on the loci of position, 1≤k≤Num;

C (), when gathering the data bulk in U and not reaching Q, returns step (b); When gathering the data bulk in U and reaching Q, again carry out step 2-1 operation to set U, be clustered into Num1 class, renewal Num0 is Num0=Num1, if w _{k, loci}>0, then just add to kth class event class center in the code book on the loci of position as code word, preserves the training data of kth class local feature, k={1,2 simultaneously ... ... ..Num0}; Empty set U, classification sum Num is updated to Num=Num+Num1, judges the training data of all inputs whether train complete, if not complete, return step (b);

Step (3) video accident detection, specifically:

By test data the code book of the loci block of locations set up with the training stage compares, 1≤loci≤f_block, if meet time, then tentatively think loci block of locations there is anomalous event to occur, otherwise does not occur;

Step (4) space-time aftertreatment, specifically:

In t, if being initially judged to of loci block of locations has anomalous event to occur, then consider the 8-neighborhood of loci block of locations in the t-1 moment, if it is abnormal to have at least two positions to exist in neighborhood, then sentencing in loci block of locations has anomalous event to occur, otherwise does not have anomalous event to occur.