CN113836969A - Abnormal event detection method based on double flows - Google Patents
Abnormal event detection method based on double flows Download PDFInfo
- Publication number
- CN113836969A CN113836969A CN202010579321.7A CN202010579321A CN113836969A CN 113836969 A CN113836969 A CN 113836969A CN 202010579321 A CN202010579321 A CN 202010579321A CN 113836969 A CN113836969 A CN 113836969A
- Authority
- CN
- China
- Prior art keywords
- training
- flow
- example packet
- video
- rgb
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an abnormal event detection method based on double flows, which comprises the following steps: step 1, dividing each training video into equal sections, and then respectively forming a positive example packet and a negative example packet for training; step 2, extracting a Flow diagram and an RGB diagram of the video by using dense Flow; step 3, utilizing ResNet50, ResNet101 and ResNet152 networks to extract the characteristics of the Flow chart and the RGB chart, and taking 32 pictures as a segment each time and taking an average value once in the extraction process; and 4, fusing the obtained features of the Flow diagram and the RGB diagram by using a double-Flow method, transmitting the features into the MLP, performing score evaluation on each section, selecting a potential abnormal sample with the largest score from a positive example packet, selecting a non-abnormal sample with the largest score from a negative example packet, and training model parameters of the MLP by using the two samples. The method repeatedly excavates complementary information of various modes in vision, detects abnormal events based on double flows, and has high detection precision.
Description
Technical Field
The invention belongs to the field of abnormal event detection methods, and particularly relates to an abnormal event detection method based on double flows.
Background
Abnormal event detection is the most challenging and long-standing problem in computer vision. For video surveillance applications, there are several methods that attempt to detect violence or aggression in videos: detecting human violence by using human motion and limb orientation; detecting offensive behavior in surveillance video using video and audio data; detecting violence of a crowd in a video using a violence flow descriptor; violent and non-violent videos are classified based on a behavior heuristic method.
The current common abnormal event detection method is to perform abnormal detection by using a video with a weaker mark, and simultaneously use C3D to extract video features, but the method is trained under weak supervision, specifically, in a section of video, only whether an abnormal event exists is concerned, and the specific abnormal type and the frame in which the abnormal occurs are not concerned, and the detection precision is low.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention aims to provide an abnormal event detection method based on double flows.
The invention discloses an abnormal event detection method based on double flows, which is characterized by comprising the following steps of:
step 1, dividing each training video into equal sections, and then respectively forming a positive example packet and a negative example packet for training;
step 2, extracting a Flow diagram and an RGB diagram of the video by using dense Flow;
step 3, utilizing ResNet50, ResNet101 and ResNet152 networks to extract the characteristics of the Flow chart and the RGB chart, and taking 32 pictures as a segment each time and taking an average value once in the extraction process;
and 4, fusing the obtained features of the Flow diagram and the RGB diagram by using a double-Flow method, transmitting the features into the MLP, performing score evaluation on each section, selecting a potential abnormal sample with the largest score from a positive example packet, selecting a non-abnormal sample with the largest score from a negative example packet, and training model parameters of the MLP by using the two samples.
Further, the specific steps in the stage of training the MLP model are as follows:
step 1, in a model with a plurality of layers in parallel, the structure and weight initialization of each network are the same, the setting of training parameters is also the same, and the training processes are mutually independent;
step 2, setting training parameters: the initial learning rate is set to be 0.0001, the number of images of each training iteration is set to be 60, and the maximum number of training iterations is set to be 20000;
step 3, loading training data: loading and extracting video features of a UCF-crime data set, wherein the total number of the video features is 1945, 1655 video features are used as a training set, and 290 video features are used as a test set;
and 4, adopting a random gradient descent algorithm.
The double-flow-based abnormal event detection method provided by the invention has the following beneficial effects: the ROC curve area obtained by the method is about 0.80, and is greatly improved compared with 0.74 obtained by the conventional method. The detection method is shown to reach state-of-art in accuracy and prove the feasibility of the dual-stream method in the multi-mode direction of the video.
Drawings
FIG. 1 is a general flow chart of the abnormal event detection according to the present invention.
Detailed Description
The following embodiments are merely illustrative of the present invention and are not intended to limit the scope of the present invention. Simple modifications of the invention applying the inventive concept are within the scope of the invention as claimed.
Referring to fig. 1, a dual-stream-based abnormal event detection method according to the present invention is characterized by comprising the following steps:
step 1, dividing each training video into equal sections, and then respectively forming a positive example packet and a negative example packet for training. One example packet corresponds to one label, at least one positive sample is required in one positive example packet, only all negative samples are required in one negative example packet, and a plurality of samples are contained in one packet.
And 2, extracting a Flow graph and an RGB graph of the video by using dense Flow. The dense Flow uses OpenCV compiled by GPU, the extracted optical Flow picture provides RGB data and Flow data for Two-Stream, and meanwhile, the optical Flow is calculated and stored locally.
Step 3, utilizing ResNet50, ResNet101 and ResNet152 networks to extract the characteristics of the Flow chart and the RGB chart, and taking 32 pictures as a segment each time and taking an average value once in the extraction process; the purpose of this operation is to reduce the number of parameters extracted to avoid the model being too large during training.
And 4, fusing the obtained characteristics of the Flow diagram and the RGB diagram by using a double-Flow method, transmitting the characteristics into the MLP, performing score evaluation on each section, selecting a potential abnormal sample with the maximum score from a positive example packet, selecting a non-abnormal sample with the maximum score from a negative example packet, and training model parameters of the MLP by using the two samples, wherein the training effect is that the model outputs high scores to the abnormal samples and low scores to the non-abnormal samples. The dual stream method, as the name suggests, is as if two streams flow respectively and finally converge into one block, wherein the name of one stream is the information of an "RGB" graph, and the name of the other stream is the information of an "optical flow" graph, which are the information change on the X axis and the information change on the Y axis, respectively.
In the preferred embodiment, the specific steps in the stage of training the MLP model are as follows:
step 1, in a model with a plurality of layers in parallel, the structure and weight initialization of each network are the same, the setting of training parameters is also the same, and the training processes are mutually independent;
step 2, setting training parameters: the initial learning rate is set to be 0.0001, the number of images of each training iteration is set to be 60, and the maximum number of training iterations is set to be 20000;
step 3, loading training data, namely loading and extracting the video features of the UCF-crime data set, wherein the total number of the video features is 1945, 1655 video features are used as a training set, and 290 video features are used as a test set;
and 4, adopting a random gradient descent algorithm.
The double-flow-based abnormal event detection method provided by the invention has the following beneficial effects: the ROC curve area obtained by the method is about 0.80, and is greatly improved compared with 0.74 obtained by the conventional method. The detection method is shown to reach state-of-art in accuracy and prove the feasibility of the dual-stream method in the multi-mode direction of the video.
The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The above-described embodiments of the invention are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims, and not by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.
Claims (2)
1. An abnormal event detection method based on double flows is characterized by comprising the following steps:
step 1, dividing each training video into equal sections, and then respectively forming a positive example packet and a negative example packet for training;
step 2, extracting a Flow diagram and an RGB diagram of the video by using dense Flow;
step 3, utilizing ResNet50, ResNet101 and ResNet152 networks to extract the characteristics of the Flow chart and the RGB chart, and taking 32 pictures as a segment each time and taking an average value once in the extraction process;
and 4, fusing the obtained features of the Flow diagram and the RGB diagram by using a double-Flow method, transmitting the features into the MLP, performing score evaluation on each section, selecting a potential abnormal sample with the largest score from a positive example packet, selecting a non-abnormal sample with the largest score from a negative example packet, and training model parameters of the MLP by using the two samples.
2. The dual-flow-based abnormal event detection method according to claim 1, wherein the specific steps of the training MLP model stage are as follows:
step 1, in a model with a plurality of layers in parallel, the structure and weight initialization of each network are the same, the setting of training parameters is also the same, and the training processes are mutually independent;
step 2, setting training parameters: the initial learning rate is set to be 0.0001, the number of images of each training iteration is set to be 60, and the maximum number of training iterations is set to be 20000;
step 3, loading training data: loading and extracting video features of a UCF-crime data set, wherein the total number of the video features is 1945, 1655 video features are used as a training set, and 290 video features are used as a test set;
and 4, adopting a random gradient descent algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010579321.7A CN113836969A (en) | 2020-06-23 | 2020-06-23 | Abnormal event detection method based on double flows |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010579321.7A CN113836969A (en) | 2020-06-23 | 2020-06-23 | Abnormal event detection method based on double flows |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113836969A true CN113836969A (en) | 2021-12-24 |
Family
ID=78963939
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010579321.7A Pending CN113836969A (en) | 2020-06-23 | 2020-06-23 | Abnormal event detection method based on double flows |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113836969A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018218286A1 (en) * | 2017-05-29 | 2018-12-06 | Saltor Pty Ltd | Method and system for abnormality detection |
| CN109615019A (en) * | 2018-12-25 | 2019-04-12 | 吉林大学 | Anomaly detection method based on space-time autocoder |
| CN109977904A (en) * | 2019-04-04 | 2019-07-05 | 成都信息工程大学 | A kind of human motion recognition method of the light-type based on deep learning |
| CN110263666A (en) * | 2019-05-29 | 2019-09-20 | 西安交通大学 | A kind of motion detection method based on asymmetric multithread |
-
2020
- 2020-06-23 CN CN202010579321.7A patent/CN113836969A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018218286A1 (en) * | 2017-05-29 | 2018-12-06 | Saltor Pty Ltd | Method and system for abnormality detection |
| CN109615019A (en) * | 2018-12-25 | 2019-04-12 | 吉林大学 | Anomaly detection method based on space-time autocoder |
| CN109977904A (en) * | 2019-04-04 | 2019-07-05 | 成都信息工程大学 | A kind of human motion recognition method of the light-type based on deep learning |
| CN110263666A (en) * | 2019-05-29 | 2019-09-20 | 西安交通大学 | A kind of motion detection method based on asymmetric multithread |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Sudhakaran et al. | Attention is all we need: Nailing down object-centric attention for egocentric activity recognition | |
| CN108805047B (en) | Living body detection method and device, electronic equipment and computer readable medium | |
| US20190228313A1 (en) | Computer Vision Systems and Methods for Unsupervised Representation Learning by Sorting Sequences | |
| CN108229277A (en) | Gesture identification, control and neural network training method, device and electronic equipment | |
| CN104717457B (en) | A kind of video concentration method and device | |
| EP3566175A1 (en) | Systems and methods for behaviour understanding from trajectories | |
| WO2022188315A1 (en) | Video detection method and apparatus, electronic device, and storage medium | |
| CN110060276A (en) | Object tracking method, tracking process method, corresponding device, electronic equipment | |
| CN111488815A (en) | Basketball game goal event prediction method based on graph convolution network and long-time and short-time memory network | |
| Xu et al. | Violent video classification based on spatial-temporal cues using deep learning | |
| Pu et al. | Audio-guided attention network for weakly supervised violence detection | |
| CN113688387A (en) | Defense method for federal learning poisoning attack based on server and client dual detection | |
| CN109583334A (en) | A kind of action identification method and its system based on space time correlation neural network | |
| CN115601696A (en) | YOLOv 5-based fire early-stage smoke detection method and system | |
| Zhang et al. | Weakly supervised anomaly detection in videos considering the openness of events | |
| CN110147876A (en) | The neural network and its movement motion generation method of view-based access control model characteristic similarity | |
| Wang et al. | Violence video detection by discriminative slow feature analysis | |
| Ju et al. | Trajectory-based Badminton Shots Detection | |
| CN113836969A (en) | Abnormal event detection method based on double flows | |
| CN110956097B (en) | Method and module for extracting occlusion human body, and scene conversion method and device | |
| CN110826459B (en) | Migratable campus violent behavior video identification method based on attitude estimation | |
| CN115546491B (en) | Fall alarm method, system, electronic equipment and storage medium | |
| Uçan et al. | Deepfake and security of video conferences | |
| JP5840186B2 (en) | Emotion search device, method and program | |
| CN116453005A (en) | Video cover extraction method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |