CN109064484B - Crowd movement behavior identification method based on fusion of subgroup component division and momentum characteristics - Google Patents

Crowd movement behavior identification method based on fusion of subgroup component division and momentum characteristics Download PDF

Info

Publication number
CN109064484B
CN109064484B CN201810236397.2A CN201810236397A CN109064484B CN 109064484 B CN109064484 B CN 109064484B CN 201810236397 A CN201810236397 A CN 201810236397A CN 109064484 B CN109064484 B CN 109064484B
Authority
CN
China
Prior art keywords
point
sub
points
motion
feature tracking
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810236397.2A
Other languages
Chinese (zh)
Other versions
CN109064484A (en
Inventor
陈志�
陈璐
岳文静
周传
刘玲
龚凯
掌静
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Posts and Telecommunications
Original Assignee
Nanjing University of Posts and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Posts and Telecommunications filed Critical Nanjing University of Posts and Telecommunications
Priority to CN201810236397.2A priority Critical patent/CN109064484B/en
Publication of CN109064484A publication Critical patent/CN109064484A/en
Application granted granted Critical
Publication of CN109064484B publication Critical patent/CN109064484B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20164—Salient point detection; Corner detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a crowd movement behavior recognition method based on the fusion of subgroup division and momentum characteristics, which comprises the steps of firstly, acquiring the space-time information of a moving target in a video image frame by utilizing an angular point tracking and background modeling method, dividing spatially adjacent crowds into a plurality of subgroups by utilizing the spatial region information of the group distribution in a foreground, and further dividing the subgroups through the movement correlation within a period of time to obtain the subgroups with movement consistency; secondly, extracting three momentum characteristics of the crowd movement for fusion on the basis of sub-population segmentation; and finally, training by taking the fused features and the pixel features of the video frame as the input of a differential cyclic convolution neural network, marking the training video segments into different description words by adopting an artificial marking method, and adjusting the result of the differential cyclic convolution neural network by using marked data to obtain a good training result, so that the motion behaviors of people can be effectively recognized, and a good effect is achieved.

Description

Crowd movement behavior identification method based on fusion of subgroup component division and momentum characteristics
Technical Field
The invention relates to a crowd movement behavior recognition method based on the fusion of subgroup division and momentum characteristics. Momentum feature extraction is carried out on the basis of subgroups, three-dimensional momentum feature video data extracted are input to a differential cycle convolution neural network for training and are converted into crowd behavior labels, the goal of crowd motion behavior identification is achieved, and the method belongs to the field of image processing, video detection and artificial intelligence cross technology application.
Background
The purpose of crowd motion behavior recognition is to divide dense crowds into subgroups through motion trail and foreground extraction from a sequence image, and crowd motion behavior recognition is carried out on the basis of the subgroups. The recognition of the activities at the group level increasingly becomes a hot problem in the field of computer vision, and has wide application in the aspects of intelligent video monitoring, public safety, sports competition and the like. The algorithm for identifying the motion behavior of the crowd in the video image frame mainly comprises a Harris corner detection algorithm, a Gaussian mixture background modeling method and a momentum feature fusion method.
(1) Harris corner detection algorithm: the algorithm uses a fixed window to slide on an image in any direction, the two conditions before and after sliding are compared, the gray level change degree of pixels in the window is large, and when the sliding in any direction exists, the corner point exists in the window. The corner points can effectively reduce the data volume of the information while keeping the important characteristics of the image graph, so that the content of the information is high, the calculation speed is effectively improved, the reliable matching of the image is facilitated, and the real-time processing becomes possible.
(2) The modeling method of the Gaussian mixture background comprises the following steps: the basic idea is to compare an input image with a background model and judge abnormal conditions which do not conform to the background model according to information such as difference values, so as to distinguish foreground pixels from background pixels. The method uses a gray level histogram to represent the distribution of gray level values in an image, and uses the statistical result to segment the image by assuming that the distribution of pixel gray level values in an image sequence obeys a normal distribution function.
(3) The momentum characteristic fusion method comprises the following steps: the method starts from the layer surface of the crowd subgroups, takes the crowd aggregated groups as a research target, and constructs momentum characteristics taking the subgroups as a unit. All sub-groups in a scene are detected through a maximum expectation optimization algorithm, the momentum characteristics such as the collective property, stability and conflict of the groups are extracted aiming at the detected groups, the momentum characteristics facing the sub-groups are formed, the group momentum characteristics can improve the dependence of the individual momentum characteristics on the scene, the independent momentum characteristics of the scene are constructed by group units, and the robustness and the expandability of the crowd movement behavior analysis are improved.
Disclosure of Invention
The invention aims to provide a crowd movement behavior recognition method based on the fusion of subgroup component division and momentum characteristics, solves the problems of division error caused by crowd overlapping in micro-segmentation and neglect of crowd movement details caused by overlarge segmentation granularity in macro-segmentation in a crowd dense scene, and provides a crowd movement momentum characteristic fusion training model based on the problem, so that the movement behaviors of crowds can be effectively recognized.
The invention discloses a crowd movement behavior identification method based on the fusion of subgroup component division and momentum characteristics, which comprises the following steps of:
step 1): the method comprises the steps that a user inputs continuous videos, the videos are divided into continuous video frames, each single pedestrian of each video frame serves as a characteristic tracking point P, and the motion information of the point P is represented by a four-dimensional vector P (P)x,Py,Pv,Pd) Is said Px、PySpatial coordinates, P, representing characteristic tracking pointsvIndicates the magnitude of the displacement of the point, PdIndicates the direction of movement of the point, PdHas a value of
Figure GDA0003307300630000021
The point set of all the characteristic tracking points of the image frame is marked as OI＝{P1,P2,P3,P4}；
Step 2): the movement characteristics of the sub-population are determined by the momentum characteristics, and three different momentum characteristics are defined on the basis of the characteristic tracking points in the sub-population and the sub-population: the motion direction consistency, the space stability and the crowd friction conflict are improved; each sub-population contains H feature tracking points, namely Ck＝(P1,P2,...,PH)；
Step 3): calculating the average value of the description factors in 5 continuous frames, and constructing a vector by using the three average values
Figure GDA0003307300630000022
Omega (C), rho (C)), jointly form a three-channel image, form data of 224 x 3 dimensions, input into a differential recurrent neural network DRCNN for training, and convert into 4096-dimensional feature vectors, wherein the differential recurrent neural network connects a VGG-16 (Visual Geometry Group-16) model and a 3-layer stacked long-short term memory recurrent neural network LSTM into an end-to-end model, and finally converts the feature vectors into crowd behavior labels by adopting an output function, and the training video segments are labeled as different description vocabularies according to the difference of behavior occurrence main bodies, behavior occurrence places and behaviors by adopting an artificial labeling method, and the result of the differential recurrent neural network is adjusted by using the labeled data to realize the recognition of the crowd motion behaviors.
Wherein the content of the first and second substances,
the step 1) is specifically as follows:
step 1.1): obtaining position information of feature tracking points in continuous video frames through a Harris corner detection algorithm to obtain foreground features of a target group, wherein the Harris corner detection algorithm is to slide a fixed window on an image in any direction, compare two conditions before and after sliding, and judge the gray level change degree of pixels in the window, when the gray level change degree is large when the sliding in any direction exists, the corner points exist in the window, the positions of all the feature tracking points in the continuous video frames are connected in series to obtain the motion trail T of each feature tracking point, and the motion trail set of all the feature tracking points is TI＝{T1,T2,T3,T4}；
Step 1.2): and (3) performing foreground extraction by using a Gaussian mixture background modeling method, and for the gray value of a pixel in the current video frame, when the difference value of the mean value of the S-th Gaussian distribution in the Gaussian mixture background meets the formula: i (x, y, t) -mus t|＜λ·σs tIs considered to beMatching successfully, i.e. the pixel is background, wherein I (x, y, t) represents the pixel value of the pixel (x, y) at time t, μst represents the mean gray value at the time of the S-th Gaussian distribution t, λ represents the multiple coefficient of the standard deviation, σst represents the variance of the gray value of the S-th Gaussian distribution at the t moment, the space size of a target group and the distance relation between the target group and surrounding groups are obtained through foreground extraction, the space size is called as foreground plaque, and a plaque set is marked as BI＝{B1,B2,B3,...,BkDividing spatially adjacent individuals through spatial relation change;
step 1.3): dividing the dense group by using the divided two kinds of space-time information;
the step 1.3) is specifically as follows:
step 1.3.1): set O of feature tracking pointsIDivided into subsets containing a number of points, denoted OI＝{CI,FI}, said CI＝{C1,C2,...,CKThe image frames are divided into point sets with motion consistency, and form a sub-population of people division, wherein the point set F is a point setIIf the feature tracking point P is located in the contour range of the patch, dividing the point into the points of the patch, otherwise, rejecting the point;
step 1.3.2): attribute P of feature tracking point PdRepresenting the motion relation of the feature tracking points between frames, calculating the cosine value of the included angle between the displacement vector value of the feature tracking point P and the X axis, obtaining a direction included angle theta, dividing the direction included angle 0-2 pi into 12 equal parts, and respectively marking each interval as Di(i ═ 1,2,3., 12) is PdAssignment, wherein the specific division method formula is as follows:
Figure GDA0003307300630000031
dividing the characteristic tracking points with similar motion trends into a sub-group through the patch range division and the constraint of the motion direction;
step 1.3.3): correcting abnormal points, and calculating attribute value P of each adjacent point in K adjacent points of the feature tracking point PdFrequency of occurrence, the maximum frequency of occurrence being marked by a value DiCharacteristic tracking point PdIs marked as DjFor all feature tracking points, when i +1 equals j or i-1 equals j, P of P is addeddThe value is corrected to Di(ii) a Removing abnormal points: calculating the number I of points with the same motion trend in L adjacent points of the characteristic tracking point P, setting a critical value M, wherein M is less than or equal to L, and when I is<M, consider point P as an outlier, from subgroup CkAnd (5) removing.
The step 2) is specifically as follows:
step 2.1): extracting the motion direction consistency characteristics: calculating the coordinate and displacement of each feature tracking point in the sub-population, averaging to obtain the centroid coordinate and average displacement of each sub-population in several continuous frames, and calculating the overall motion trend vector of each sub-population
Figure GDA0003307300630000041
According to the formula:
Figure GDA0003307300630000042
calculating the velocity correlation of the overall motion trend vector of each sub-population and the vector, wherein
Figure GDA0003307300630000043
A motion trend vector representing each sub-population,
Figure GDA0003307300630000044
a motion trend vector representing the k-th adjacent point, N represents the number of divided sub-groups, and the motion trend vector represents the k-th adjacent point
Figure GDA00033073006300000411
A larger value indicates a higher velocity correlation;
step 2.2): extracting space stability momentum characteristics: space stabilizationSex means that each feature tracking point keeps a stable neighbor in a certain time range and keeps a specific topological structure in a certain time; defining each feature tracking point P in the sub-populationiThe stability at time t is given by the formula:
Figure GDA0003307300630000045
obtaining where N denotes the number of divided sub-populations, PiRepresents the ith feature tracking point, | N1(Pi)\N1(Pi)TI represents the average number of points which maintain a stable adjacent relation with adjacent points of one feature tracking point in a certain sub-group from 1 to T, and K represents the number of the most adjacent points; the stability of the distance between each feature tracking point in the sub-population and the adjacent point in the neighborhood thereof can be represented by the following formula:
Figure GDA0003307300630000046
wherein N represents the number of divided sub-populations, PiThe ith feature tracking point is represented as,
Figure GDA0003307300630000047
representing the average distance between the feature tracking point and its k adjacent points, adding the above two stabilities to form the overall stability of the subgroup, using ω (C)k) Represents;
step 2.3): extracting the characteristics of crowd friction and conflict momentum: the formula for calculating the conflict is as follows:
Figure GDA0003307300630000048
wherein N represents the number of divided sub-populations, PiThe ith feature tracking point is represented as,
Figure GDA0003307300630000049
a motion trend vector representing each sub-population,
Figure GDA00033073006300000410
represents the motion trend vector, avrg (N), of the k-th neighboring pointother(Pi) ) represents a sub-populationThe adjacency of the feature tracking point in (1) includes the average values of the feature tracking points in other sub-populations, and α and β are weight coefficients.
In the step 1.2), the multiple coefficient lambda of the standard deviation is 2.5.
Has the advantages that: the crowd movement behavior identification method based on the fusion of subgroup component division and momentum characteristics, provided by the invention, has the following specific beneficial effects:
1. the invention analyzes the subgroup division of the crowd under the crowd dense scene from the view of the subgroup, not only can solve the difficulty of separating moving individuals from the dense crowd, but also can obtain the ignored internal characteristics when the crowd is regarded as a whole as a research object.
2. The invention provides an algorithm based on space-time constraint, which takes the temporal motion relation and the spatial proximity among individuals in a motion group as the basis of group division, and the two conditions are constrained mutually to divide the group into sub-groups with motion consistency. The algorithm can be suitable for monitoring video streams with different population densities and observation visual angles, and is simple to implement and high in running speed.
3. The invention trains the abstracted three-dimensional video data by utilizing the differential recursive convolutional neural network, separates the steps of feature extraction and parameter learning, and is suitable for group videos with various resolutions and moving scenes.
Drawings
Fig. 1 is a flow chart of a subgroup division algorithm.
FIG. 2 is a flow chart of a momentum feature fusion algorithm.
Detailed Description
Some embodiments of the invention are described in more detail below with reference to the accompanying drawings.
The user enters 15 seconds of continuous video and divides it into one image frame every 0.5 seconds, i.e. DI＝{D1,D2,D3,...,D30}，DtFor the image frame at the time t, the people with motion consistency are divided into subgroups in the crowd dense scene, and the algorithm flow chart is shown in fig. 1.
Taking the first 15 framesVideo image frame, each single pedestrian is used as a tracking characteristic tracking point, and a four-dimensional vector P is used as (P)x,Py,Pv,Pd) Representing motion information of a characteristic tracking point P, where Px,PySpatial coordinates, P, representing characteristic tracking pointsvIndicates the magnitude of the displacement of the point, PdRepresents the direction of movement of the point, and has a value of
Figure GDA0003307300630000051
Coordinate position information of feature tracking points in 15 video frames is obtained through a Harris corner detection algorithm, the coordinate positions of all the feature tracking points in continuous video frames are connected in series to obtain a motion trail T of each feature tracking point, and the motion trails of all the feature tracking points are integrated into TI＝{T1,T2,T3,T4Each track comprises a set T of a plurality of characteristic tracking pointsi＝{P1,P2,P3,...,PkAnd the continuous motion track represents the motion relationship of the population within a period of time.
And (3) performing foreground extraction by using a Gaussian mixture background modeling method, and if the difference value between the gray value of the pixel in the current video frame and the mean value of the Kth Gaussian distribution in the Gaussian mixture background meets the formula:
Figure GDA0003307300630000063
the matching is considered successful, i.e. the pixel is taken as background, where λ is a multiple coefficient of the standard deviation and λ takes a value of 2.5. The space size of a target group and the distance relation between the target group and surrounding groups are obtained through foreground extraction, the space size and the distance relation are called foreground plaques, and a plaque set is recorded as BI＝{B1,B2,B3,...,BkAnd (4) dividing spatially adjacent individuals through spatial relation change.
And dividing the dense group by using the divided two kinds of space-time information. Set O of feature tracking pointsIDivided into subsets containing a number of points, denoted OI＝{CI,FIIn which C isI＝{C1,C2,...,CKThe image frames are divided into point sets with motion consistency, and form a sub-population of people division, wherein the point set F is a point setIIt is the point that is culled. And (3) representing the space information of the patch by using a rectangular frame, dividing a rectangular area by using a coordinate value of a patch boundary pixel position, acquiring coordinates of right lower corners and left upper corners of the rectangle, if the coordinate position of the feature tracking point P is in the outline range of the patch, dividing the point into the points of the patch, and if not, rejecting the points. Attribute P of feature tracking point PdRepresenting the motion relationship of the feature tracking points from frame to frame. Firstly, calculating the displacement vector of the characteristic tracking point P
Figure GDA0003307300630000061
And the cosine value of the included angle between the X axis and the X axis so as to obtain the direction included angle theta. Dividing the direction included angle of 0-2 pi into 12 equal parts, and respectively marking each interval as Di(i-1, 2,3 …,12) is PdAssignment, wherein the specific division method comprises the following steps:
Figure GDA0003307300630000062
and dividing the characteristic tracking points with similar motion trends into a sub-population through the restriction of the range division and the motion direction of the plaque.
And correcting and eliminating abnormal points. First, the outlier is corrected, and the attribute value P of each of the K adjacent points of the feature tracking point P is calculateddThe mark value with the largest frequency of occurrence is DiCharacteristic tracking point PdIs marked as DjFor all feature tracking points, if i +1 ═ j or i-1 ═ j, then P of P will be assigneddThe value is corrected to Di(ii) a And secondly, removing the abnormal points, and calculating the number I of points with the same motion trend in the L adjacent points of the characteristic tracking point P. Setting a critical value M (M is less than or equal to L), wherein M is 5, and if I is<M, then the P point is considered as an abnormal point, and the sub-population CkAnd (5) removing.
Setting the movement characteristic momentum characteristic decision of the sub-population, and deciding the movement characteristic momentum characteristic decision based on the sub-population and the characteristic tracking points in the sub-populationThree different momentum characteristics are defined: uniformity of motion direction, spatial stability, and crowd friction conflict. Suppose that each sub-population contains H feature tracking points, namely Ck＝(P1,P2,...,PH)。
Extracting the motion direction consistency characteristics: calculating the coordinate and displacement of each feature tracking point in the sub-population, averaging to obtain the centroid coordinate and average displacement of each sub-population in several continuous frames, and calculating the overall motion trend vector of each sub-population
Figure GDA0003307300630000071
According to the formula:
Figure GDA0003307300630000072
calculating the velocity correlation of the overall motion trend vector of each sub-population and the vector, wherein
Figure GDA0003307300630000073
A motion trend vector representing each sub-population,
Figure GDA0003307300630000074
a motion trend vector representing the k-th adjacent point, N represents the number of divided sub-groups, and the motion trend vector represents the k-th adjacent point
Figure GDA0003307300630000075
A larger value indicates a higher velocity correlation.
Extracting space stability momentum characteristics: the spatial stability refers to that each characteristic tracking point keeps a stable neighbor in a certain time range and keeps a specific topological structure in a certain time; defining each feature tracking point P in the sub-populationiAt time t, the region composed of K most adjacent points is Nt(Pi) Stability can be represented by the formula:
Figure GDA0003307300630000076
obtaining where N denotes the number of divided sub-populations, PiRepresents the ith feature tracking point, | N1(Pi)\N1(Pi)TI represents the average number of points which maintain a stable adjacent relation with adjacent points of a feature point in a certain sub-population from 1 to T, and K represents the number of the most adjacent points; the stability of the distance between each feature tracking point in the sub-population and the adjacent point in the neighborhood thereof can be represented by the following formula:
Figure GDA0003307300630000077
wherein N represents the number of divided sub-populations, PiThe ith feature tracking point is represented as,
Figure GDA0003307300630000078
representing the average distance between the feature tracking point and its k adjacent points, adding the above two stabilities to form the overall stability of the subgroup, using ω (C)k) And (4) showing.
Extracting group friction conflict momentum characteristics: the formula for calculating the conflict is as follows: the formula for calculating the conflict is as follows:
Figure GDA0003307300630000079
wherein N represents the number of divided sub-populations, PiRepresenting the ith feature tracking point, wherein
Figure GDA00033073006300000710
A motion trend vector representing each sub-population,
Figure GDA00033073006300000711
represents the motion trend vector, avrg (N), of the k-th neighboring pointother(Pi) A) represents the average of the feature tracking points in other sub-populations, including the adjacency of feature tracking points in one sub-population, and α and β are weighting coefficients.
Calculating the average value of the description factors in the first 5 frames, and constructing a vector by using the three average values
Figure GDA0003307300630000081
Similar to RGB values of pixel points, the RGB values form a three-channel image together, and the three-channel image forms dimensions of 224 multiplied by 3The data is input into a differential cyclic convolution neural network (DRCNN) for training and is converted into 4096 characteristic vectors. The method comprises the steps that a differential cyclic convolution neural network VGG-16 (Visual Geometry Group-16) model and a 3-layer stacked Long Short-Term Memory recurrent neural network (LSTM) are connected to an end-to-end model, training accuracy is improved, feature vectors are converted into crowd behavior labels by adopting an output function, training video segments are marked as different description vocabularies according to different behavior occurrence main bodies, behavior occurrence places and behaviors per se by adopting an artificial marking method, and the results of the differential cyclic convolution neural network are adjusted by using marked data, so that the recognition of the crowd movement behaviors is realized.

Claims (4)

1. A crowd movement behavior identification method based on the fusion of subgroup component division and momentum characteristics is characterized by comprising the following steps:
step 1): the method comprises the steps that a user inputs continuous videos, the videos are divided into continuous video frames, each single pedestrian of each video frame serves as a characteristic tracking point P, and the motion information of the point P is represented by a four-dimensional vector P (P)x,Py,Pv,Pd) Is said Px、PySpatial coordinates, P, representing characteristic tracking pointsvIndicates the magnitude of the displacement of the point, PdIndicates the direction of movement of the point, PdHas a value of
Figure FDA0003307300620000011
The point set of all the characteristic tracking points of the image frame is marked as OI＝{P1,P2,P3,P4}；
Step 2): the movement characteristics of the sub-population are determined by the momentum characteristics, and three different momentum characteristics are defined on the basis of the characteristic tracking points in the sub-population and the sub-population: the motion direction consistency, the space stability and the crowd friction conflict are improved; each sub-population contains H feature tracking points, namely Ck＝(P1,P2,...,PH)；
Step 3): computing descriptions within successive 5 framesAverage value of the factor, constructing a vector using three average values: (
Figure FDA0003307300620000013
Omega (C), rho (C)), jointly form a three-channel image, form data of 224 x 3 dimensions, input into a differential recursive convolutional neural network DRCNN for training, convert into 4096-dimensional characteristic vectors, the said differential recursive convolutional neural network connects VGG-16 model and 3 layers of stacked long and short term memory recursive neural network LSTM to the end-to-end model, finally adopt the output function to convert the characteristic vectors into crowd behavior labels, adopt the method of artificial marking to train the video segment according to behavior occurrence subject, behavior occurrence place, behavior difference, mark as different description vocabulary, use the data with mark to adjust the result of the differential recursive neural network, realize the recognition of crowd movement behavior;
the step 1) is specifically as follows:
step 1.1): obtaining position information of feature tracking points in continuous video frames through a Harris corner detection algorithm to obtain foreground features of a target group, wherein the Harris corner detection algorithm is to slide a fixed window on an image in any direction, compare two conditions before and after sliding, and judge the gray level change degree of pixels in the window, when the gray level change degree is large when the sliding in any direction exists, the corner points exist in the window, the positions of all the feature tracking points in the continuous video frames are connected in series to obtain the motion trail T of each feature tracking point, and the motion trail set of all the feature tracking points is TI＝{T1,T2,T3,T4}；
Step 1.2): and (3) performing foreground extraction by using a Gaussian mixture background modeling method, and for the gray value of a pixel in the current video frame, when the difference value of the mean value of the S-th Gaussian distribution in the Gaussian mixture background meets the formula:
Figure FDA0003307300620000012
the matching is considered to be successful, i.e. the pixel is taken as background, wherein I (x, y, t) represents that the pixel point (x, y) is at the time of tThe value of the pixel of (a) is,
Figure FDA0003307300620000021
represents the mean gray value at the time t of the S-th gaussian distribution, lambda represents the coefficient of the multiple of the standard deviation,
Figure FDA0003307300620000022
representing the variance of the gray value of the S-th Gaussian distribution t moment, extracting the space size of the target group and the distance relation between the space size and the surrounding groups through the foreground, and obtaining the foreground plaque, wherein the plaque set is marked as BI＝{B1,B2,B3,...,BkDividing spatially adjacent individuals through spatial relation change;
step 1.3): and dividing the dense group by using the divided two kinds of space-time information.
2. The method for identifying the motion behavior of the crowd based on the fusion of the subgroup component division and the momentum feature according to claim 1, wherein the step 1.3) specifically comprises the following steps:
step 1.3.1): set O of feature tracking pointsIDivided into subsets containing a number of points, denoted OI＝{CI,FI}, said CI＝{C1,C2,...,CKThe image frames are divided into point sets with motion consistency, and form a sub-population of people division, wherein the point set F is a point setIIf the feature tracking point P is located in the contour range of the patch, dividing the point into the points of the patch, otherwise, rejecting the point;
step 1.3.2): attribute P of feature tracking point PdRepresenting the motion relation of the feature tracking points between frames, calculating the cosine value of the included angle between the displacement vector value of the feature tracking point P and the X axis, obtaining a direction included angle theta, dividing the direction included angle 0-2 pi into 12 equal parts, and dividing each interval into 12 equal partsAre respectively marked as Di(i ═ 1,2,3., 12) is PdAssignment, wherein the specific division method formula is as follows:
Figure FDA0003307300620000023
dividing the characteristic tracking points with similar motion trends into a sub-group through the patch range division and the constraint of the motion direction;
step 1.3.3): correcting abnormal points, and calculating attribute value P of each adjacent point in K adjacent points of the feature tracking point PdFrequency of occurrence, the maximum frequency of occurrence being marked by a value DiCharacteristic tracking point PdIs marked as DjFor all feature tracking points, when i +1 equals j or i-1 equals j, P of P is addeddThe value is corrected to Di(ii) a Removing abnormal points: calculating the number I of points with the same motion trend in L adjacent points of the characteristic tracking point P, setting a critical value M, wherein M is less than or equal to L, and when I is<M, consider point P as an outlier, from subgroup CkAnd (5) removing.
3. The method for identifying the motion behavior of the crowd based on the fusion of the subgroup component division and the momentum feature according to claim 1, wherein the step 2) is specifically as follows:
step 2.1): extracting the motion direction consistency characteristics: calculating the coordinate and displacement of each feature tracking point in the sub-population, averaging to obtain the centroid coordinate and average displacement of each sub-population in several continuous frames, and calculating the overall motion trend vector of each sub-population
Figure FDA0003307300620000031
According to the formula:
Figure FDA0003307300620000032
calculating the velocity correlation of the overall motion trend vector of each sub-population and the vector, wherein
Figure FDA0003307300620000033
A motion trend vector representing each sub-population,
Figure FDA0003307300620000034
a motion trend vector representing the k-th adjacent point, N represents the number of divided sub-groups, and the motion trend vector represents the k-th adjacent point
Figure FDA0003307300620000035
A larger value indicates a higher velocity correlation;
step 2.2): extracting space stability momentum characteristics: the spatial stability refers to that each characteristic tracking point keeps a stable neighbor in a certain time range and keeps a specific topological structure in a certain time; defining each feature tracking point P in the sub-populationiThe stability at time t is given by the formula:
Figure FDA0003307300620000036
obtaining where N denotes the number of divided sub-populations, PiRepresents the ith feature tracking point, | N1(Pi)\N1(Pi)TI represents the average number of points which maintain a stable adjacent relation with adjacent points of one feature tracking point in a certain sub-group from 1 to T, and K represents the number of the most adjacent points; the stability of the distance between each feature tracking point in the sub-population and the adjacent point in the neighborhood thereof can be represented by the following formula:
Figure FDA0003307300620000037
wherein N represents the number of divided sub-populations, PiThe ith feature tracking point is represented as,
Figure FDA0003307300620000038
representing the average distance between the feature tracking point and its k adjacent points, adding the above two stabilities to form the overall stability of the subgroup, using ω (C)k) Represents;
step 2.3): extracting the characteristics of crowd friction and conflict momentum: conflict calculating methodThe formula is as follows:
Figure FDA0003307300620000039
wherein N represents the number of divided sub-populations, PiThe ith feature tracking point is represented as,
Figure FDA00033073006200000310
a motion trend vector representing each sub-population,
Figure FDA00033073006200000311
represents the motion trend vector, avrg (N), of the k-th neighboring pointother(Pi) A) represents the average of the feature tracking points in other sub-populations, including the adjacency of feature tracking points in one sub-population, and α and β are weighting coefficients.
4. The method for identifying the motion behaviors of the people based on the fusion of the subgroup component and the momentum characteristics as claimed in claim 1, wherein the standard deviation multiple coefficient lambda in the step 1.2) is 2.5.
CN201810236397.2A 2018-03-21 2018-03-21 Crowd movement behavior identification method based on fusion of subgroup component division and momentum characteristics Active CN109064484B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810236397.2A CN109064484B (en) 2018-03-21 2018-03-21 Crowd movement behavior identification method based on fusion of subgroup component division and momentum characteristics

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810236397.2A CN109064484B (en) 2018-03-21 2018-03-21 Crowd movement behavior identification method based on fusion of subgroup component division and momentum characteristics

Publications (2)

Publication Number Publication Date
CN109064484A CN109064484A (en) 2018-12-21
CN109064484B true CN109064484B (en) 2022-02-08

Family

ID=64819919

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810236397.2A Active CN109064484B (en) 2018-03-21 2018-03-21 Crowd movement behavior identification method based on fusion of subgroup component division and momentum characteristics

Country Status (1)

Country Link
CN (1) CN109064484B (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109829382B (en) * 2018-12-30 2020-04-24 北京宇琪云联科技发展有限公司 Abnormal target early warning tracking system and method based on intelligent behavior characteristic analysis
CN109918996A (en) * 2019-01-17 2019-06-21 平安科技（深圳）有限公司 The illegal action identification method of personnel, system, computer equipment and storage medium
CN110263690A (en) * 2019-06-12 2019-09-20 成都信息工程大学 A kind of group behavior feature extraction based on small group and description method and system
CN110443789B (en) * 2019-08-01 2021-11-26 四川大学华西医院 Method for establishing and using immune fixed electrophoretogram automatic identification model
CN110472604B (en) * 2019-08-20 2021-05-14 中国计量大学 Pedestrian and crowd behavior identification method based on video
CN112850436A (en) * 2019-11-28 2021-05-28 宁波微科光电股份有限公司 Pedestrian trend detection method and system of elevator intelligent light curtain
CN111160150A (en) * 2019-12-16 2020-05-15 盐城吉大智能终端产业研究院有限公司 Video monitoring crowd behavior identification method based on depth residual error neural network convolution
CN111429185B (en) * 2020-03-27 2023-06-02 京东城市(北京)数字科技有限公司 Crowd figure prediction method, device, equipment and storage medium
CN111739651A (en) * 2020-06-16 2020-10-02 南京众智未来人工智能研究院有限公司 Multi-body space detection system and method based on group identification
CN111933298B (en) * 2020-08-14 2024-02-13 医渡云（北京）技术有限公司 Crowd relation determining method and device, electronic equipment and medium
CN113283387B (en) * 2021-06-23 2022-11-11 华东交通大学 Group abnormal behavior detection method and device

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106529465A (en) * 2016-11-07 2017-03-22 燕山大学 Pedestrian cause and effect relation identification method based on momentum kinetic model

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106529465A (en) * 2016-11-07 2017-03-22 燕山大学 Pedestrian cause and effect relation identification method based on momentum kinetic model

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Dynamic Image Networks for Action Recognition;Hakan Bilen等;《CVPR》;20161231;第79-85页 *
结合群组动量特征与卷积神经网络的人群行为分析;成金庚等;《科学技术与工程》;20170531;第17卷(第14期);第3034-3042页 *

Also Published As

Publication number Publication date
CN109064484A (en) 2018-12-21

Similar Documents

Publication Publication Date Title
CN109064484B (en) Crowd movement behavior identification method based on fusion of subgroup component division and momentum characteristics
CN108830252B (en) Convolutional neural network human body action recognition method fusing global space-time characteristics
CN107832672B (en) Pedestrian re-identification method for designing multi-loss function by utilizing attitude information
CN109543695B (en) Population-density population counting method based on multi-scale deep learning
CN106022220B (en) Method for tracking multiple faces of participating athletes in sports video
CN104751491B (en) A kind of crowd&#39;s tracking and people flow rate statistical method and device
CN107016357B (en) Video pedestrian detection method based on time domain convolutional neural network
CN109035293B (en) Method suitable for segmenting remarkable human body example in video image
Brown View independent vehicle/person classification
CN108875588A (en) Across camera pedestrian detection tracking based on deep learning
CN107273905B (en) Target active contour tracking method combined with motion information
CN114220176A (en) Human behavior recognition method based on deep learning
Cai et al. Multi-object detection and tracking by stereo vision
Mukherjee et al. Anovel framework for automatic passenger counting
CN105528794A (en) Moving object detection method based on Gaussian mixture model and superpixel segmentation
CN103164858A (en) Adhered crowd segmenting and tracking methods based on superpixel and graph model
CN107862240B (en) Multi-camera collaborative face tracking method
CN114240997B (en) Intelligent building online trans-camera multi-target tracking method
CN107153824A (en) Across video pedestrian recognition methods again based on figure cluster
CN105930790A (en) Human body behavior recognition method based on kernel sparse coding
CN111353448A (en) Pedestrian multi-target tracking method based on relevance clustering and space-time constraint
CN112989889A (en) Gait recognition method based on posture guidance
Zhu et al. Tracking multiple objects through occlusion with online sampling and position estimation
CN114648557A (en) Multi-target cooperative tracking method based on high-altitude visual angle and ground visual angle
Lejbolle et al. Multimodal neural network for overhead person re-identification

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant