CN112488021A - Monitoring video-based garbage delivery violation detection method and system - Google Patents
Monitoring video-based garbage delivery violation detection method and system Download PDFInfo
- Publication number
- CN112488021A CN112488021A CN202011439546.9A CN202011439546A CN112488021A CN 112488021 A CN112488021 A CN 112488021A CN 202011439546 A CN202011439546 A CN 202011439546A CN 112488021 A CN112488021 A CN 112488021A
- Authority
- CN
- China
- Prior art keywords
- frame
- garbage
- detection
- video
- pedestrian
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000010813 municipal solid waste Substances 0.000 title claims abstract description 254
- 238000001514 detection method Methods 0.000 title claims abstract description 146
- 238000012544 monitoring process Methods 0.000 title claims abstract description 31
- 238000000034 method Methods 0.000 claims abstract description 31
- 230000008569 process Effects 0.000 claims abstract description 15
- 238000013135 deep learning Methods 0.000 claims description 21
- 238000012549 training Methods 0.000 claims description 20
- 238000000605 extraction Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 9
- 230000007423 decrease Effects 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 239000004576 sand Substances 0.000 claims description 6
- 230000006399 behavior Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006698 induction Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000010791 domestic waste Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Molecular Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- Biomedical Technology (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a monitoring video-based garbage delivery violation detection method, which comprises the following steps: s1, receiving monitoring video stream data in a garbage delivery booth and nearby the garbage delivery booth; s2, detecting video stream data based on the detection frame model, and judging whether a garbage bag exists in the garbage delivery booth or not and nearby; s3, if the fact that a garbage bag exists in the t frame of video image is detected, forward detection is started from the t frame, and whether pedestrians and garbage bags are carried in the t-j frame of video image in the forward detection process or not is judged; s4, matching the similarity of the garbage bags carried by the pedestrians in the t-j frame video image with the garbage bags existing in the t frame video image, and judging whether the matched similarity value is larger than a preset threshold value or not; s5, starting to detect the current garbage bag and the pedestrian backwards from the t-j frame of video image until the last frame of the pedestrian video, judging whether the pedestrian carrying the current garbage bag leaves with the garbage bag in the detection process, and if not, obtaining that the pedestrian carrying the garbage bag is delivered illegally.
Description
Technical Field
The invention relates to the technical field of video processing, in particular to a method and a system for detecting garbage delivery violation based on a monitoring video.
Background
Along with the continuous development of domestic scientific and technological process, the urbanization process is accelerated and people's standard of living improves rapidly, the quantity of the domestic waste that produces also increases rapidly among the daily life, a lot of residents do not have the habit of developing the regular delivery rubbish according to the community requirement, can all put rubbish in rubbish input pavilion limit in the non-input time quantum many times, and the sack often can not be tied tightly in the disposal bag of putting on the street many times, rubbish is turned over at the street at will not only can influence the holistic pleasing to the eye of community, the peculiar smell that rubbish distributed out also can lead to the fact the influence to the fact the resident life. No special personnel in the community manages and controls the garbage delivery violation, and the time and the labor are consumed by manual control.
In order to solve the above problem, patent publication No. CN108820647A discloses a method and a system for trash throwing based on image recognition, where the method is used for an intelligent trash can, the intelligent trash can is provided with an image collecting and recognizing device, and the method includes: acquiring images of people entering the induction area to obtain an initial image; identifying characteristic elements in the initial image and judging whether garbage to be thrown exists or not; when garbage to be thrown exists, classifying and comparing the characteristic elements to obtain corresponding garbage classification information; configuring a corresponding throwing port for the garbage to be thrown according to the garbage classification information; and opening the corresponding throwing port. According to the scheme, the image acquisition device acquires images of personnel entering the induction area to obtain initial images, the identification device acquires whether garbage is to be thrown in, the identification module classifies, contrasts and identifies the garbage to be thrown in when the garbage is to be thrown in, the classification information of the garbage is acquired, and the corresponding throwing port is configured for the garbage according to the classification information of the garbage and the throwing port is opened. Although the above patent can acquire the personnel throwing the rubbish through the image acquisition device, the pedestrian who passes by carrying the rubbish bag can not be effectively identified.
Therefore, the technical problems existing in the prior art are improved.
Disclosure of Invention
The invention aims to provide a garbage delivery violation detection method and system based on a monitoring video aiming at the defects of the prior art, and the method and system are used for identifying the pedestrian behaviors in the monitoring video and finding out the pedestrians who violate the garbage delivery based on a convolutional neural network and an image processing technology.
In order to achieve the purpose, the invention adopts the following technical scheme:
a monitoring video-based garbage delivery violation detection method comprises the following steps:
s1, receiving monitoring video stream data collected by a camera in and near a garbage delivery booth;
s2, detecting the received video stream data every preset time based on the detection frame model, and judging whether garbage bags exist in the garbage delivery booth or nearby the garbage delivery booth according to a detection result;
s3, if detecting that a garbage bag exists in the t frame video image in the video stream data, starting forward detection from the t frame, and judging whether pedestrians simultaneously exist in the t-j frame video image and carry the garbage bag in the forward detection, if so, executing the step S4;
s4, matching the similarity of the garbage bags carried by the pedestrians in the t-j frame video image with the garbage bags existing in the t frame video image, judging whether the matched similarity value is larger than a first preset threshold value or not, and if yes, executing the step S5; if not, continue to step S3;
s5, detecting the current garbage bag and the pedestrian carrying the current garbage bag backwards from the t-j frame of video image, stopping detection until the last frame of image of the pedestrian monitoring video, judging whether the pedestrian carrying the current garbage bag leaves with the garbage bag in the video image from the t-j frame of video image to the detection stopping, if so, determining that the pedestrian carrying the current garbage bag passes by, and continuing to execute the step S3; if not, the pedestrian carrying the current garbage bag is judged to be illegal delivery.
Further, the step S3, where a trash bag exists in the t-th frame of video image specifically is that only a trash bag exists in the t-th frame of video image, or the t-th frame of video image includes a trash bag and a pedestrian, and the distance between the trash bag and the pedestrian is greater than a second preset threshold.
Further, the step S2 of detecting the received video stream data at preset intervals based on the detection frame model further includes training the detection frame model, specifically:
A1. intercepting a video frame image from a video data stream, and marking the intercepted video frame image, wherein the marked content is a detection frame of a target object and the category of the target object;
A2. inputting the cut image into a deep learning network, and extracting the feature F by utilizing a shallow structure in the deep learning networkSAnd deep structure extraction features FDAnd extracting the feature F of the shallow structureSInputting the information into an SNet channel of a target detection algorithm to obtain the category information of the garbage bags and/or the category information of pedestrians in the video frame image; extracting deep structures to obtain features FDInputting the information into a GNet channel of a target detection algorithm to obtain garbage bags and/or pedestrians in a video frame image;
A3. initializing parameters of a deep learning network, and setting the maximum iteration number of the deep learning network as k; judging whether the Loss function Loss value continuously decreases in the training process, if so, continuing training until K times of iteration to obtain a final detection frame model; and if the Loss function Loss value tends to be stable in the midway, stopping iteration to obtain a final detection frame model.
Further, in step S2, the detecting, based on the detection frame model, the received video stream data every preset time includes:
B1. inputting received video stream data into a detection frame model for processing, and outputting a garbage bag detection frame and a pedestrian detection frame by the detection frame model;
B2. and judging whether the distance between the output garbage bag detection frame and the pedestrian detection frame is smaller than a third preset threshold value, if so, executing the step S3.
Further, in the step S3, it is determined whether there is a pedestrian and a pedestrian carrying garbage bag in the t-j frame video image at the time of the forward detection, and the intersection ratio of the pedestrian and the pedestrian carrying garbage bag is greater than a fourth preset threshold.
Further, in the step S4, it is determined whether the matched similarity value is greater than a first preset threshold, and if so, it is determined that the trash bag carried by the pedestrian in the t-j frame video image is the same as the trash bag existing in the t-j frame video image.
Further, the step S5 includes detecting the current trash bag and the pedestrian carrying the current trash bag forward from the t-j frame of video image, and stopping the detection until the pedestrian monitors the first frame of video image.
Correspondingly, still provide a garbage delivery violation detection system based on surveillance video, include:
the receiving module is used for receiving monitoring video stream data collected by the camera in the garbage delivery booth and nearby the garbage delivery booth;
the detection module is used for detecting the received video stream data at preset time intervals based on the detection frame model and judging whether garbage bags exist in the garbage delivery booth or nearby the garbage delivery booth according to the detection result;
the first judgment module is used for starting forward detection from a t frame if a garbage bag is detected in a t frame video image in the video stream data, and judging whether a pedestrian exists in a t-j frame video image and carries the garbage bag;
the second judgment module is used for carrying out similarity matching on the garbage bags carried by the pedestrians in the t-j frame video image and the garbage bags existing in the t frame video image, and judging whether the similarity value after matching is larger than a first preset threshold value or not;
and the third judgment module is used for detecting the current garbage bag and the pedestrian carrying the current garbage bag backwards from the t-j frame video image, stopping detection until the pedestrian monitors the last frame image of the video, and judging whether the pedestrian carrying the current garbage bag leaves with the garbage bag in the video images from the t-j frame detection to the detection stopping.
Further, the existence of a garbage bag in the t-th frame of video image in the first judgment module is specifically that only a garbage bag exists in the t-th frame of video image, or the t-th frame of video image comprises a garbage bag and a pedestrian, and the distance between the garbage bag and the pedestrian is greater than a second preset threshold value.
Further, the detecting module, when detecting the received video stream data at preset intervals based on the detection frame model, further includes training the detection frame model, specifically:
the intercepting module is used for intercepting a video frame image from a video data stream and marking the intercepted video frame image, wherein the marked content is a detection frame of a target object and the category of the target object;
an extraction module for inputting the cut images into the deep learning network and extracting the feature F by using the shallow structure in the deep learning networkSAnd deep structure extraction features FDAnd extracting the feature F of the shallow structureSInputting the information into an SNet channel of a target detection algorithm to obtain the category information of the garbage bags and/or the category information of pedestrians in the video frame image; extracting deep structures to obtain features FDInputting the information into a GNet channel of a target detection algorithm to obtain garbage bags and/or pedestrians in a video frame image;
the training module is used for initializing parameters of the deep learning network and setting the maximum iteration number of the deep learning network as k; and judging whether the Loss function Loss value continuously decreases in the training process.
Compared with the prior art, the invention has the following beneficial effects:
1. the invention monitors and manages the placed garbage, thereby obtaining the pedestrian corresponding to the garbage and warning the pedestrian;
2. the invention can also judge whether the pedestrian corresponding to the garbage detected for the first time passes by, namely the pedestrian carrying the garbage but passing by, so that the detection accuracy is further improved, and the pedestrian carrying the garbage to pass by is eliminated;
3. the invention saves the finally confirmed video of the whole process of the pedestrian throwing the garbage illegally, and sends out warning to the pedestrian.
Drawings
FIG. 1 is a flowchart of a monitoring video-based method for detecting spam violations according to an embodiment;
FIG. 2 is a flow chart and a block diagram of a garbage delivery violation detection according to an embodiment;
fig. 3 is a block diagram of a monitoring video-based spam violation detection system according to the second embodiment.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
The invention aims to provide a method and a system for detecting garbage delivery violation based on a monitoring video, aiming at the defects of the prior art.
Example one
The embodiment provides a monitoring video-based method for detecting a spam violation, as shown in fig. 1-2, comprising the following steps:
s11, receiving monitoring video stream data collected by a camera in the garbage delivery booth and near the garbage delivery booth;
s12, detecting the received video stream data every preset time based on the detection frame model, and judging whether garbage bags exist in the garbage delivery booth or nearby the garbage delivery booth according to a detection result;
s13, if a garbage bag is detected in the t frame video image in the video stream data, forward detection is started from the t frame, whether a pedestrian exists in the t-j frame video image and carries the garbage bag or not during forward detection is judged, and if yes, the step S14 is executed;
s14, matching the similarity of the garbage bags carried by the pedestrians in the t-j frame video image with the garbage bags existing in the t frame video image, judging whether the matched similarity value is larger than a first preset threshold value or not, and if yes, executing the step S15; if not, continue to step S13;
s15, detecting the current garbage bag and the pedestrian carrying the current garbage bag backwards from the t-j frame of video image, stopping detection until the last frame of image of the pedestrian monitoring video, judging whether the pedestrian carrying the current garbage bag leaves with the garbage bag in the video image from the t-j frame of video image to the detection stopping, if so, determining that the pedestrian carrying the current garbage bag passes by, and continuing to execute the step S13; if not, the pedestrian carrying the current garbage bag is judged to be illegal delivery.
The invention is applied to detecting illegal behaviors of garbage bags and pedestrians in a non-putting time period or a garbage bin for putting residential garbage on a roadside.
In step S11, the monitoring video stream data L collected by the camera in and near the trash delivery kiosk is received.
The camera is installed at the garbage delivery booth, collects monitoring video stream data in and near the garbage delivery booth, and uploads the collected video stream data to the detection system. The camera can be started for 24 hours or can be started outside a specified garbage throwing time period; if the camera is turned on for 24 hours, the uploaded data only comprise video stream data outside the specified garbage putting time period.
For example, the specified time period for putting garbage is 7 am to 9 am, and 5 pm to 8 pm; then the non-delivery time period is from 9 am to 5 pm, and 8 pm to 7 am, the camera can collect data within 24 hours, but only the video stream data collected from 9 am to 5 pm, and 8 pm to 7 am are uploaded to the detection system.
In step S12, the received video stream data is detected every preset time based on the detection frame model, and whether there is a garbage bag in and near the garbage delivery kiosk is determined according to the detection result.
Detecting the video stream data L once every a period of time (for example, detecting one frame every ten minutes) based on the detection frame model, and judging and identifying whether a garbage bag placed by residents exists in the garbage delivery pavilion and the vicinity thereof (if no pedestrian exists around the garbage bag, the video image comprises the following two types, namely no pedestrian exists in the video image, only the garbage bag exists, or the garbage bag and the pedestrian exist in the video, but the distance between the pedestrian and the garbage bag is larger than a threshold value T1)。
In this embodiment, detecting the received video stream data every preset time based on the detection frame model further includes training a deep learning network model, specifically:
preparing data of detection frames of the garbage bags and the pedestrians: firstly, intercepting a video frame image from a video data stream L, labeling the extracted image, wherein the content of the label is a detection frame of a target object and the category of the target object, namely (c)k,xk,yk,wk,hk) Wherein c isjRepresenting the class of the object in the detection frame, ck1 indicates that the detected frame is a pedestrian, ckIf 0, the detection frame is a garbage bag, xk,ykRepresenting coordinate information of the top left vertex of the detection box, wk,hkRepresenting the width and height information of the detection frame, and dividing the labeled data sample into a training set, a verification set and a test set according to the ratio of 8:1: 1;
and (3) network structure design: the network input is an extracted video frame image, and a shallow structure of ResNet is utilized to extract a feature FSAnd deep structure extraction features FDShallow feature FSThe SNet is used for extracting the class information of the garbage bags and the class information of pedestrians and the deep characteristic FDInput to GNet, whose function is to distinguish trash bags in imagesUnlike a pedestrian, the first frame of video stream data is used to initialize the SNet channel and the GNet channel, the updating method of the two modules is different, for a new frame of image, the interested region is concentrated in the target position of the previous frame, wherein the interested region contains the information of the target object and the information of the background, the information is transmitted by the full convolution network, the SNet module and the GNet module respectively generate a heat map of the foreground, the predicted target position of the new frame of image is based on the heat map, and the selection mechanism is used to determine whether the heat map is finally generated by the SNet module or the GNet module, thereby determining the final target position; wherein, FDFeature maps that represent less detail but more semantic information; fSFeature maps that show more detail but less semantic information.
The network employs a swish activation function,
and (3) training a detection frame model: initializing network parameters, setting the maximum iteration number of the network as k, and inputting the prepared data set into the network for training. If the Loss value continuously decreases, continuing training until K times of iteration to obtain a final detection frame model; and if the Loss value tends to be stable in the midway, stopping iteration to obtain a final detection frame model.
In step S12, the method for detecting the received video stream data every preset time based on the detection frame model specifically includes:
inputting received video stream data into a detection frame model for processing, and outputting a garbage bag detection frame and a pedestrian detection frame by the detection frame model;
wherein the information of the garbage bag detection frame is (0, x)1,y1,w1,h1) The pedestrian detection frame information is (1, x)2,y2,w2,h2) The distance formula between the pedestrian detection frame and the garbage bag detection frame is as follows:
and judging whether the distance d between the output garbage bag detection frame and the pedestrian detection frame is smaller than a third preset threshold value, if so, executing the step S13.
In step S13, if it is detected that there is a trash bag in the t-th frame of video image in the video stream data, the video stream data is detected forward from the t-th frame, and it is determined whether there is a pedestrian and the pedestrian carries the trash bag in the t-j frame of video image during the forward detection, if yes, step S14 is executed.
When the video images of the garbage or the garbage and the pedestrians are detected in the T-th frame or near the garbage delivery booth in the step S12, the forward detection is started along the current T-th frame, and whether the pedestrians and the garbage bags carried by the pedestrians exist in the video images detected forward or not is judged, and the intersection ratio of the pedestrians and the garbage bags carried by the pedestrians is larger than the threshold value T2(ii) a If yes, go to step S14; if not, the process continues to step S12.
The detection is performed from the t-th frame forward, and the detection of whether there are pedestrians and the garbage bags carried by the pedestrians is performed by processing the current video image through the detection frame model obtained in step S12, which is similar to the processing method in step S12 and is not repeated here.
In this embodiment, the pedestrian carrying the trash bag is denoted by h, and the trash bag carried by the pedestrian h is denoted by r.
In step S14, similarity matching is performed between a trash bag carried by a pedestrian in the t-j frame of video image and a trash bag existing in the t frame of video image, and it is determined whether a similarity value after matching is greater than a first preset threshold, if yes, step S15 is executed; if not, the process continues to step S13.
After the detection in step S13, if a pedestrian h carrying a trash bag and a trash bag r are detected in the T-j frame video image, then similarity matching is further performed between the trash bag r carried by the pedestrian h and the trash bag existing in the T-j frame video image, and it is determined whether the similarity value after matching is greater than the preset threshold T or not3If the similarity is greater than the preset threshold T3If the garbage bag r in the t-j frame video image and the garbage bag r in the t frame video image are the same garbage bag, otherwise, repeatingStep S13.
Cutting out a garbage bag surrounding frame detected in the t-j frame and a garbage bag surrounding frame detected in the t frame, wherein an image taken out of the t-j frame is p1The image taken from the t frame is p2. Firstly, p is1And p2Down to 8x8 size; converting the reduced image into 64-level gray scale; comparing the gray scale of each pixel with the average value of 8x8 pixels, recording the average value greater than or equal to 1, and the average value smaller than 0, combining the obtained 64 values to form two 64-bit integers, wherein the two 64-bit integers are p1And p2The features of (1); p is to be1And p2The more the same number of bits, the more p1And p2The higher the similarity.
It should be noted that, if the similarity value after matching is judged to be smaller than the preset threshold T3Step S13 is continuously executed, wherein step S13 is executed to continue forward detection along the t-j frame without restarting detection from the t-th frame.
In step S15, detecting the current trash bag and the pedestrian carrying the current trash bag backward from the t-j frame of video image, stopping the detection until the last frame of image of the pedestrian monitoring video, and determining whether the pedestrian carrying the current trash bag leaves with the trash bag in the video image from the t-j frame of video image to the detection stop, if yes, determining that the pedestrian carrying the current trash bag passes by, and continuing to execute step S13; if not, the pedestrian carrying the current garbage bag is judged to be illegal delivery
In order to judge whether the pedestrian illegally delivers garbage or passes by carrying a garbage bag, detecting and tracking the pedestrian h carrying the garbage bag and the garbage bag r backwards from the t-j frame until the pedestrian leaves the last frame of image of the monitoring video; judging whether the pedestrian h carrying the garbage bag in the video image from the t-j frame starting detection to the stopping detection leaves with the garbage bag r, if the pedestrian h carrying the garbage bag r leaves in the backward detection tracking process, determining that the pedestrian h carries the garbage bag r to pass by, and at the moment, determining that the pedestrian is a passerby, and continuously executing the step S13; if the pedestrian h does not leave with the garbage bag r in the backward detection and tracking process, it is indicated that only the pedestrian h is present in the image when the last frame is detected, and the garbage bag r is absent, in order to more accurately determine the illegal throwing behavior of the pedestrian, the pedestrian h carrying the garbage bag and the garbage bag r need to be detected and tracked forward from the t-j frame until the pedestrian enters the first frame image of the monitoring video, whether the pedestrian h carries the garbage bag r in the first frame image enters the monitoring range is judged, and if not, the step S13 is continued; if yes, the pedestrian h delivers the garbage in violation, the system gives out warning, and videos of the whole process of the pedestrian h delivers the garbage in violation are stored.
Compared with the prior art, the embodiment has the following beneficial effects:
1. the embodiment monitors and manages the placed garbage, so that pedestrians corresponding to the garbage are obtained, and the pedestrians are warned;
2. the embodiment can also judge whether the pedestrian corresponding to the garbage detected for the first time passes by, namely the pedestrian carrying the garbage but passing by, so that the detection accuracy is further improved, and the pedestrian carrying the garbage to pass by is eliminated;
3. according to the embodiment, the system stores the finally confirmed video of the pedestrian who violates the garbage throwing, and gives a warning to the pedestrian.
Example two
The embodiment provides a monitoring video-based system for detecting a spam violation, as shown in fig. 3, including:
the receiving module 11 is used for receiving monitoring video stream data collected by the camera in the garbage delivery booth and near the garbage delivery booth;
the detection module 12 is used for detecting the received video stream data every preset time based on the detection frame model and judging whether a garbage bag exists in the garbage delivery booth or nearby the garbage delivery booth according to the detection result;
the first judging module 13 is configured to, if it is detected that a trash bag exists in a t-th frame video image in the video stream data, start forward detection from the t-th frame, and judge whether a pedestrian exists in a t-j-th frame video image during forward detection and carries the trash bag by the pedestrian;
the second judging module 14 is configured to perform similarity matching between a garbage bag carried by a pedestrian in the t-j frame video image and a garbage bag existing in the t frame video image, and judge whether a similarity value after matching is greater than a first preset threshold value;
and the third judging module 15 is configured to detect the current trash bag and the pedestrian carrying the current trash bag backward from the t-j frame of video image, stop detection until the last frame of image of the pedestrian monitoring video, and judge whether the pedestrian carrying the current trash bag leaves with the trash bag in the video image from the t-j frame of detection to the detection stop.
Further, the existence of a garbage bag in the t-th frame of video image in the first judgment module is specifically that only a garbage bag exists in the t-th frame of video image, or the t-th frame of video image comprises a garbage bag and a pedestrian, and the distance between the garbage bag and the pedestrian is greater than a second preset threshold value.
Further, the detecting module, when detecting the received video stream data at preset intervals based on the detection frame model, further includes training the detection frame model, specifically:
the intercepting module is used for intercepting a video frame image from a video data stream and marking the intercepted video frame image, wherein the marked content is a detection frame of a target object and the category of the target object;
an extraction module for inputting the cut images into the deep learning network and extracting the feature F by using the shallow structure in the deep learning networkSAnd deep structure extraction features FDAnd extracting the feature F of the shallow structureSInputting the information into an SNet channel of a target detection algorithm to obtain the category information of the garbage bags and/or the category information of pedestrians in the video frame image; extracting deep structures to obtain features FDInputting the information into a GNet channel of a target detection algorithm to obtain garbage bags and/or pedestrians in a video frame image;
the training module is used for initializing parameters of the deep learning network and setting the maximum iteration number of the deep learning network as k; and judging whether the Loss function Loss value continuously decreases in the training process.
It should be noted that the system for detecting spam violations based on surveillance videos provided in this embodiment is similar to the embodiment, and will not be described herein again.
Compared with the prior art, the embodiment has the following beneficial effects:
1. the embodiment monitors and manages the placed garbage, so that pedestrians corresponding to the garbage are obtained, and the pedestrians are warned;
2. the embodiment can also judge whether the pedestrian corresponding to the garbage detected for the first time passes by, namely the pedestrian carrying the garbage but passing by, so that the detection accuracy is further improved, and the pedestrian carrying the garbage to pass by is eliminated;
3. according to the embodiment, the system stores the finally confirmed video of the pedestrian who violates the garbage throwing, and gives a warning to the pedestrian.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A monitoring video-based garbage delivery violation detection method is characterized by comprising the following steps:
s1, receiving monitoring video stream data collected by a camera in and near a garbage delivery booth;
s2, detecting the received video stream data every preset time based on the detection frame model, and judging whether garbage bags exist in the garbage delivery booth or nearby the garbage delivery booth according to a detection result;
s3, if detecting that a garbage bag exists in the t frame video image in the video stream data, starting forward detection from the t frame, and judging whether pedestrians simultaneously exist in the t-j frame video image and carry the garbage bag in the forward detection, if so, executing the step S4;
s4, matching the similarity of the garbage bags carried by the pedestrians in the t-j frame video image with the garbage bags existing in the t frame video image, judging whether the matched similarity value is larger than a first preset threshold value or not, and if yes, executing the step S5; if not, continue to step S3;
s5, detecting the current garbage bag and the pedestrian carrying the current garbage bag backwards from the t-j frame of video image, stopping detection until the last frame of image of the pedestrian monitoring video, judging whether the pedestrian carrying the current garbage bag leaves with the garbage bag in the video image from the t-j frame of video image to the detection stopping, if so, determining that the pedestrian carrying the current garbage bag passes by, and continuing to execute the step S3; if not, the pedestrian carrying the current garbage bag is judged to be illegal delivery.
2. The method according to claim 1, wherein the garbage bag in the t-th frame of video image in the step S3 is only a garbage bag in the t-th frame of video image, or the t-th frame of video image includes a garbage bag and a pedestrian, and the distance between the garbage bag and the pedestrian is greater than a second preset threshold.
3. The method according to claim 2, wherein the step S2 of detecting the received video stream data at preset time intervals based on the detection frame model further includes training the detection frame model, specifically:
A1. intercepting a video frame image from a video data stream, and marking the intercepted video frame image, wherein the marked content is a detection frame of a target object and the category of the target object;
A2. inputting the cut-out image into a deep learning network,extracting feature F by using shallow structure in deep learning networkSAnd deep structure extraction features FDAnd extracting the feature F of the shallow structureSInputting the information into an SNet channel of a target detection algorithm to obtain the category information of the garbage bags and/or the category information of pedestrians in the video frame image; extracting deep structures to obtain features FDInputting the information into a GNet channel of a target detection algorithm to obtain garbage bags and/or pedestrians in a video frame image;
A3. initializing parameters of a deep learning network, and setting the maximum iteration number of the deep learning network as k; judging whether the Loss function Loss value continuously decreases in the training process, if so, continuing training until K times of iteration to obtain a final detection frame model; and if the Loss function Loss value tends to be stable in the midway, stopping iteration to obtain a final detection frame model.
4. The method for detecting spam violations based on surveillance videos as claimed in claim 3, wherein the step S2 is performed based on a detection frame model to detect the received video stream data at preset time intervals, specifically:
B1. inputting received video stream data into a detection frame model for processing, and outputting a garbage bag detection frame and a pedestrian detection frame by the detection frame model;
B2. and judging whether the distance between the output garbage bag detection frame and the pedestrian detection frame is smaller than a third preset threshold value, if so, executing the step S3.
5. The method according to claim 1, wherein the step S3 is performed to determine whether there are pedestrians and bags carried by pedestrians in the t-j frame video images during the forward detection, and the intersection ratio of the pedestrians and the bags carried by pedestrians is greater than a fourth preset threshold.
6. The method according to claim 1, wherein in step S4, it is determined whether the matched similarity value is greater than a first preset threshold, and if so, it is determined that the garbage bag carried by the pedestrian in the t-j frame video image is the same as the garbage bag existing in the t-j frame video image.
7. The method for detecting spam delivery violations based on surveillance videos as claimed in claim 5, wherein the step S5 further includes detecting the current spam bag and the pedestrian carrying the current spam bag forward from the t-j frame video image, and stopping the detection until the pedestrian monitors the first frame image of the video.
8. A monitoring video-based spam violation detection system, comprising:
the receiving module is used for receiving monitoring video stream data collected by the camera in the garbage delivery booth and nearby the garbage delivery booth;
the detection module is used for detecting the received video stream data at preset time intervals based on the detection frame model and judging whether garbage bags exist in the garbage delivery booth or nearby the garbage delivery booth according to the detection result;
the first judgment module is used for starting forward detection from a t frame if a garbage bag is detected in a t frame video image in the video stream data, and judging whether a pedestrian exists in a t-j frame video image and carries the garbage bag;
the second judgment module is used for carrying out similarity matching on the garbage bags carried by the pedestrians in the t-j frame video image and the garbage bags existing in the t frame video image, and judging whether the similarity value after matching is larger than a first preset threshold value or not;
and the third judgment module is used for detecting the current garbage bag and the pedestrian carrying the current garbage bag backwards from the t-j frame video image, stopping detection until the pedestrian monitors the last frame image of the video, and judging whether the pedestrian carrying the current garbage bag leaves with the garbage bag in the video images from the t-j frame detection to the detection stopping.
9. The system according to claim 8, wherein the existence of the garbage bag in the t-th frame of video image in the first determination module is specifically that only the garbage bag exists in the t-th frame of video image, or the garbage bag and the pedestrian are included in the t-th frame of video image, and the distance between the garbage bag and the pedestrian is greater than a second preset threshold.
10. The system according to claim 9, wherein the detection of the received video stream data at preset time intervals based on the detection frame model in the detection module further comprises training the detection frame model, specifically:
the intercepting module is used for intercepting a video frame image from a video data stream and marking the intercepted video frame image, wherein the marked content is a detection frame of a target object and the category of the target object;
an extraction module for inputting the cut images into the deep learning network and extracting the feature F by using the shallow structure in the deep learning networkSAnd deep structure extraction features FDAnd extracting the feature F of the shallow structureSInputting the information into an SNet channel of a target detection algorithm to obtain the category information of the garbage bags and/or the category information of pedestrians in the video frame image; extracting deep structures to obtain features FDInputting the information into a GNet channel of a target detection algorithm to obtain garbage bags and/or pedestrians in a video frame image;
the training module is used for initializing parameters of the deep learning network and setting the maximum iteration number of the deep learning network as k; and judging whether the Loss function Loss value continuously decreases in the training process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011439546.9A CN112488021A (en) | 2020-12-10 | 2020-12-10 | Monitoring video-based garbage delivery violation detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011439546.9A CN112488021A (en) | 2020-12-10 | 2020-12-10 | Monitoring video-based garbage delivery violation detection method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112488021A true CN112488021A (en) | 2021-03-12 |
Family
ID=74941585
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011439546.9A Pending CN112488021A (en) | 2020-12-10 | 2020-12-10 | Monitoring video-based garbage delivery violation detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112488021A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113065509A (en) * | 2021-04-20 | 2021-07-02 | 广州铁路职业技术学院（广州铁路机械学校） | Garbage processing method, device, equipment and computer storage medium |
| CN113221804A (en) * | 2021-05-25 | 2021-08-06 | 城云科技（中国）有限公司 | Disordered material detection method and device based on monitoring video and application |
| CN113435419A (en) * | 2021-08-26 | 2021-09-24 | 城云科技（中国）有限公司 | Illegal garbage discarding behavior detection method, device and application |
| CN113657143A (en) * | 2021-06-25 | 2021-11-16 | 中国计量大学 | Garbage classification method based on classification and detection joint judgment |
| CN113705370A (en) * | 2021-08-09 | 2021-11-26 | 百度在线网络技术（北京）有限公司 | Method and device for detecting illegal behavior of live broadcast room, electronic equipment and storage medium |
| CN114715562A (en) * | 2022-03-21 | 2022-07-08 | 盛视科技股份有限公司 | Recognition method for kitchen garbage illegal putting behavior |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105069042A (en) * | 2015-07-23 | 2015-11-18 | 北京航空航天大学 | Content-based data retrieval methods for unmanned aerial vehicle spying images |
| CN107239735A (en) * | 2017-04-24 | 2017-10-10 | 复旦大学 | A kind of biopsy method and system based on video analysis |
| US20190347486A1 (en) * | 2018-05-08 | 2019-11-14 | Electronics And Telecommunications Research Institute | Method and apparatus for detecting a garbage dumping action in real time on video surveillance system |
| CN110990633A (en) * | 2019-12-25 | 2020-04-10 | 浙江丰牛环境科技有限公司 | Garbage classification fixed-point management system and method based on image retrieval and public system |
| CN111178182A (en) * | 2019-12-16 | 2020-05-19 | 深圳奥腾光通***有限公司 | Real-time detection method for garbage loss behavior |
| CN111611970A (en) * | 2020-06-01 | 2020-09-01 | 城云科技（中国）有限公司 | Urban management monitoring video-based disposable garbage behavior detection method |
-
2020
- 2020-12-10 CN CN202011439546.9A patent/CN112488021A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105069042A (en) * | 2015-07-23 | 2015-11-18 | 北京航空航天大学 | Content-based data retrieval methods for unmanned aerial vehicle spying images |
| CN107239735A (en) * | 2017-04-24 | 2017-10-10 | 复旦大学 | A kind of biopsy method and system based on video analysis |
| US20190347486A1 (en) * | 2018-05-08 | 2019-11-14 | Electronics And Telecommunications Research Institute | Method and apparatus for detecting a garbage dumping action in real time on video surveillance system |
| CN111178182A (en) * | 2019-12-16 | 2020-05-19 | 深圳奥腾光通***有限公司 | Real-time detection method for garbage loss behavior |
| CN110990633A (en) * | 2019-12-25 | 2020-04-10 | 浙江丰牛环境科技有限公司 | Garbage classification fixed-point management system and method based on image retrieval and public system |
| CN111611970A (en) * | 2020-06-01 | 2020-09-01 | 城云科技（中国）有限公司 | Urban management monitoring video-based disposable garbage behavior detection method |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113065509A (en) * | 2021-04-20 | 2021-07-02 | 广州铁路职业技术学院（广州铁路机械学校） | Garbage processing method, device, equipment and computer storage medium |
| CN113221804A (en) * | 2021-05-25 | 2021-08-06 | 城云科技（中国）有限公司 | Disordered material detection method and device based on monitoring video and application |
| CN113221804B (en) * | 2021-05-25 | 2023-03-24 | 城云科技（中国）有限公司 | Disordered material detection method and device based on monitoring video and application |
| CN113657143A (en) * | 2021-06-25 | 2021-11-16 | 中国计量大学 | Garbage classification method based on classification and detection joint judgment |
| CN113657143B (en) * | 2021-06-25 | 2023-06-23 | 中国计量大学 | Garbage classification method based on classification and detection combined judgment |
| CN113705370A (en) * | 2021-08-09 | 2021-11-26 | 百度在线网络技术（北京）有限公司 | Method and device for detecting illegal behavior of live broadcast room, electronic equipment and storage medium |
| CN113705370B (en) * | 2021-08-09 | 2023-06-30 | 百度在线网络技术（北京）有限公司 | Method and device for detecting illegal behaviors of live broadcasting room, electronic equipment and storage medium |
| CN113435419A (en) * | 2021-08-26 | 2021-09-24 | 城云科技（中国）有限公司 | Illegal garbage discarding behavior detection method, device and application |
| CN114715562A (en) * | 2022-03-21 | 2022-07-08 | 盛视科技股份有限公司 | Recognition method for kitchen garbage illegal putting behavior |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112488021A (en) | Monitoring video-based garbage delivery violation detection method and system | |
| US9418546B1 (en) | Traffic detection with multiple outputs depending on type of object detected | |
| CN106845890B (en) | Storage monitoring method and device based on video monitoring | |
| US11335086B2 (en) | Methods and electronic devices for automated waste management | |
| CN109829382B (en) | Abnormal target early warning tracking system and method based on intelligent behavior characteristic analysis | |
| CN111611970B (en) | Urban management monitoring video-based random garbage throwing behavior detection method | |
| JP5106356B2 (en) | Image monitoring device | |
| CN111178182A (en) | Real-time detection method for garbage loss behavior | |
| CN106339657B (en) | Crop straw burning monitoring method based on monitor video, device | |
| CN102332092A (en) | Flame detection method based on video analysis | |
| CN109993031A (en) | A kind of animal-drawn vehicle target is driven against traffic regulations behavioral value method, apparatus and camera | |
| Momin et al. | Vehicle detection and attribute based search of vehicles in video surveillance system | |
| CN104463232A (en) | Density crowd counting method based on HOG characteristic and color histogram characteristic | |
| CN104537360A (en) | Method and system for detecting vehicle violation of not giving way | |
| CN111914634A (en) | Complex-scene-interference-resistant automatic manhole cover type detection method and system | |
| CN109993032A (en) | A kind of shared bicycle target identification method, device and camera | |
| CN113657143B (en) | Garbage classification method based on classification and detection combined judgment | |
| Di Mauro et al. | Estimating the occupancy status of parking areas by counting cars and non-empty stalls | |
| Chughtai et al. | Traffic Surveillance System: Robust Multiclass Vehicle Detection and Classification | |
| CN116886874A (en) | Ecological garden security monitoring and early warning data acquisition method and system | |
| Ilayarajaa et al. | Text recognition in moving vehicles using deep learning neural networks | |
| CN103366163A (en) | Human face detection system and method based on incremental learning | |
| Muniruzzaman et al. | Deterministic algorithm for traffic detection in free-flow and congestion using video sensor | |
| Roy et al. | Vehicle number plate recognition and parking system | |
| Saranya et al. | The Proficient ML method for Vehicle Detection and Recognition in Video Sequence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |