CN115169673A - Intelligent campus epidemic risk monitoring and early warning system and method - Google Patents

Intelligent campus epidemic risk monitoring and early warning system and method Download PDF

Info

Publication number
CN115169673A
CN115169673A CN202210765657.1A CN202210765657A CN115169673A CN 115169673 A CN115169673 A CN 115169673A CN 202210765657 A CN202210765657 A CN 202210765657A CN 115169673 A CN115169673 A CN 115169673A
Authority
CN
China
Prior art keywords
pedestrian
submodule
detection
target
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210765657.1A
Other languages
Chinese (zh)
Inventor
欧吉顺
聂庆慧
相铮
龙秀江
张俊
刘路
陈楹颖
苏光磊
徐佳兴
谢菲
林长硕
黄韵玲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yangzhou University
Original Assignee
Yangzhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yangzhou University filed Critical Yangzhou University
Priority to CN202210765657.1A priority Critical patent/CN115169673A/en
Publication of CN115169673A publication Critical patent/CN115169673A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/04Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/06Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063Operations research, analysis or management
    • G06Q10/0635Risk analysis of enterprise or organisation activities
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10Services
    • G06Q50/20Education
    • G06Q50/205Education administration or guidance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G06V10/765Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/40Scenes; Scene-specific elements in video content
    • G06V20/41Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/52Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161Detection; Localisation; Normalisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168Feature extraction; Face representation
    • G06V40/171Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172Classification, e.g. identification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20Movements or behaviour, e.g. gesture recognition

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Human Resources & Organizations (AREA)
  • Strategic Management (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Economics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • Tourism & Hospitality (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Entrepreneurship & Innovation (AREA)
  • General Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Databases & Information Systems (AREA)
  • Marketing (AREA)
  • Medical Informatics (AREA)
  • Educational Technology (AREA)
  • Game Theory and Decision Science (AREA)
  • Quality & Reliability (AREA)
  • Development Economics (AREA)
  • Operations Research (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Primary Health Care (AREA)
  • Psychiatry (AREA)

Abstract

The invention discloses a smart campus epidemic situation risk monitoring and early warning system and a smart campus epidemic situation risk monitoring and early warning method, wherein the system comprises the following steps: including image data acquisition processing module, the identity differentiates the module, the target detection module, track identification module and epidemic situation risk detection module, image data acquisition processing module is used for gathering campus crowd flow condition, carry out the preliminary treatment according to flow data, including data enhancement, preliminary treatment such as data mark and data set division, the identity is differentiateed the module and is used for discerning target identity information based on pedestrian face identification and pedestrian gesture recognition, target detection module is used for carrying out pedestrian target state detection to image data, the track identification module is used for detecting pedestrian's action scope and action mode, epidemic situation risk detection module is used for differentiateing campus flow, study and judge and the early warning to campus crowd flow condition. The campus epidemic situation control system can provide refined track information of people going out in a campus, and realize accurate campus epidemic situation prevention and control by assisting campus epidemic situation flow-regulation data.

Description

Intelligent campus epidemic situation risk monitoring and early warning system and method
Technical Field
The invention belongs to the technical field of data monitoring, and particularly relates to an epidemic situation risk monitoring and early warning system and method for an intelligent campus.
Background
Epidemic outbreaks give rise to significant impact on the national economy and society. The campus is used as a dense place for people and is a key link and an important area for epidemic prevention and control. Compared with other public place crowds, the campus crowds have the typical characteristics of strong social interaction and interactivity, concentrated activity places and high overlapping degree of flow tracks, so that the existing epidemic prevention and control technology product is difficult to play a key role in campus epidemic prevention and control.
Aiming at risk monitoring and prevention and control of new crown epidemic situations, a series of technical exploration is developed by scholars at home and abroad, and abundant research achievements are obtained in some fields, including face recognition, mask wearing monitoring, crowd density monitoring, body temperature automatic measurement and the like. Some scientific and technological innovation enterprises also provide new technical products for power-assisted epidemic situation prevention and control, such as rapid temperature measurement, mobile phone or voice control of elevators, non-contact meal delivery, disinfection robots and the like.
Some internet enterprises also invest in research and development of related epidemic situation monitoring technology products in a dispute, including research and development of intelligent epidemic situation robots based on a voice recognition technology and human body automatic temperature measurement and face recognition systems based on an infrared thermal imaging and cloud face recognition technology. Some scientific and technological innovation enterprises also provide new technical products for power-assisted epidemic situation prevention and control, such as rapid temperature measurement, mobile phone or voice control of elevators, non-contact meal delivery, disinfection robots and the like.
In summary, most of the existing technical products only aim at general public places, and ignore the important application place of campus, so that the existing epidemic situation risk prevention and control technical products are difficult to play a key role in the campus epidemic situation prevention and control process.
Disclosure of Invention
In order to solve the problems, the invention discloses a smart campus epidemic situation risk monitoring and early warning system and method, which aim to detect the crowd distribution in a campus, analyze the travel characteristics of the campus crowd, design various core monitoring functions, provide refined track information of travel personnel in the campus, assist campus epidemic situation flow regulation data to realize precise campus epidemic situation prevention and control and provide accurate epidemic situation risk monitoring and early warning service for campus managers at people, places and moments.
In order to achieve the purpose, the technical scheme of the invention is as follows:
the utility model provides a wisdom campus epidemic situation risk monitoring and early warning system, includes: the system comprises an image data acquisition and processing module, an identity distinguishing module, a target detection module, a track identification module and an epidemic situation risk detection module;
the image data acquisition and processing module is used for acquiring the flow condition of the school crowd and preprocessing the flow data, including data enhancement, data annotation, data set division and the like, and comprises an image data acquisition submodule and an image data preprocessing submodule;
the identity recognition module is used for recognizing the identity information of the target based on the face recognition and the gesture recognition of the pedestrian, and comprises a face recognition submodule and a gesture recognition submodule;
the target detection module is used for detecting the pedestrian target state of the image data and comprises a pedestrian detection sub-module, a pedestrian density detection sub-module and a mask detection sub-module;
the track identification module is used for detecting the action range and the action mode of the pedestrian and comprises a line crossing detection submodule, a distance detection submodule and a speed detection submodule;
the epidemic situation risk detection module is used for acquiring data detected by the identity identification module, the target detection module and the track identification module to identify campus pedestrian flow, studying and judging and early warning the campus crowd flow condition, and comprises a high-risk area identification submodule, an illegal person monitoring submodule, a personnel flow counting submodule and a propagation risk early warning submodule.
Further, the image data acquisition submodule is accessed into the aerial photography equipment and the plurality of cameras through the outside to acquire the campus crowd flow video, and different samples in the acquired image video are detected, tracked and trained to establish an image data set;
the image data preprocessing submodule is used for operating the acquired image video, and comprises the steps of turning, rotating, shearing, splicing and mosaic enhancing the image video, carrying out image morphological operation on the image video, selecting a modeling image video by using an image pyramid technology, carrying out artificial annotation on the image by using an image annotation software tool, determining the size of a prior marking frame by using a clustering technology, acquiring an image data set, and dividing the collected data into a training set, a verification set and a test set according to the characteristics of the data set.
Furthermore, a face recognition sub-module is externally connected into a campus face library, a Cascade Cascade classifier is built based on PCA dimension reduction and LDA feature extraction technologies, face key point features and gray level changes are recognized, a face recognition model is trained according to different face feature recognition, and accurate identity recognition is carried out by matching recognized face facial features with targets in the campus face library;
the posture recognition sub-module is based on image segmentation modeling of Masked R-CNN to segment a pedestrian object from an image video, and captures human posture key position information in a human behavior expression process by using a 2D key point detection algorithm to extract human key representation points and determine target identity information.
Further, the pedestrian detection submodule is used for detecting a pedestrian target in an image video, constructing a multi-target detection model based on a transform framework, extracting the characteristics of the pedestrian target in the video image by using a CNN convolutional neural network through introducing a multi-head self-attention mechanism, performing accurate filtering and nonlinear transformation on the extracted characteristics of the pedestrian target through constructing the transform network, constructing a prediction model through associating characteristic information, a classification label and a detection frame position, and obtaining an optimal target detection model through multiple rounds of iterative training, so that the detection label of the pedestrian target is realized;
the pedestrian detection sub-module carries out target tracking on the marked pedestrian target, constructs a TransTransTransTransTransTransck target tracking model, takes a pedestrian target feature map of the current frame as an input key, takes pedestrian target features of a series of past frames and a series of learnable queries as input queries, and detects a new pedestrian target and outputs a detection boundary frame by constructing the learnable queries, wherein the pedestrian target features of the past frames are generated by past frame detection and used for positioning an object and an output track frame which exist in the current frame, and the output is completed by intersection and matching of the detection frame and the track frame;
the pedestrian density detection submodule clusters the positions of the pedestrian target marks by using a clustering algorithm and visually presents the slow traffic density in a thermodynamic diagram manner;
the mask detection submodule is used for detecting whether school garden pedestrians wear the mask in the image video.
Further, the line crossing detection submodule is used for automatically judging whether a pedestrian crosses a reference line in the image video, and automatically sending out early warning information if the pedestrian crosses the reference line;
the distance detection submodule is used for calculating the distance between every two pedestrians in the image video and judging whether the two pedestrians have potential epidemic situation propagation risks or not;
the speed detection submodule is used for estimating the speed of the target pedestrian in the image video.
Further, the high risk area identification submodule is used for identifying epidemic situation propagation high risk areas with large school garden people flow and high track overlapping degree in the image video;
the illegal person monitoring submodule is used for monitoring the illegal behaviors of the campus persons in the image video in real time;
the personnel flow statistics submodule is used for counting the personnel flow entering and exiting different areas of the campus in the image video;
the propagation risk early warning submodule is used for studying and judging potential propagation risks through image videos and providing quick early warning service for a campus manager, and comprises system early warning, short message early warning and mail early warning, wherein the system early warning carries out risk early warning through an information bullet frame inside the system, the short message early warning automatically triggers an early warning mechanism according to a study and judgment risk threshold value, early warning short messages are timely sent to the campus manager, the mail early warning is connected with an external mail system through the system, an automatic mail sending program is compiled, and mail early warning is carried out.
A smart campus epidemic risk monitoring and early warning method comprises the following steps:
s1: the method comprises the steps that an image data acquisition and processing module is used for acquiring the flow condition of a campus crowd, and preprocessing including data enhancement, data annotation, data set division and the like is carried out according to flow data;
s2: identifying target identity information based on pedestrian face identification and pedestrian posture identification by using an identity identification module;
s3: detecting the pedestrian target state of the image data by using a target detection module;
s4: the track recognition module is used for detecting the action range and the action mode of the pedestrian;
s5: data detected by the identity identification module, the target detection module and the track identification module are acquired by the epidemic situation risk detection module to identify the flow of the pedestrians in the campus, and the flow condition of the pedestrians in the campus is researched, judged and early warned.
Further setting: the steps S1, S2, S3, S4, and S5 specifically include the following steps:
s1-1: acquiring a campus crowd flowing video by using an image data acquisition submodule through externally accessing aerial equipment and a plurality of cameras, detecting and tracking different samples in the acquired image video, establishing an image data set, using an image data preprocessing submodule for operating the acquired image video, wherein the image data preprocessing submodule comprises the steps of turning, rotating, shearing, splicing and mosaic enhancing the image video, carrying out image morphological operation on the image video, selecting a modeling image video by using an image pyramid technology, manually marking the image by using an image marking software tool, determining the size of a prior marking frame by using a clustering technology, acquiring the image data set, and dividing the collected data into a training set, a verification set and a test set according to the characteristics of the data set;
s2-1: the method comprises the steps that a face recognition submodule is externally connected into a campus face library, a Cascade Cascade classifier is built on the basis of PCA dimension reduction and LDA feature extraction technologies, key point features and gray level changes of a face are recognized, a face recognition model is trained according to different face feature recognition, targets in the campus face library are matched through the recognized face facial features, accurate identity recognition is conducted, a posture recognition submodule is used for image segmentation modeling based on mask R-CNN to segment pedestrian objects from image videos, capture of position information of the human posture key points in a human behavior expression process is achieved through a 2D key point detection algorithm, key characterization points of the human body are extracted, and identity information of the targets is determined;
s3-1: detecting a pedestrian target in an image video by using a pedestrian detection submodule, constructing a multi-target detection model based on a transform framework, extracting the characteristics of the pedestrian target in the video image primarily by using a CNN convolutional neural network through introducing a multi-head self-attention mechanism, performing accurate filtering and nonlinear transformation on the extracted characteristics of the pedestrian target by constructing the transform network, constructing a prediction model through associating characteristic information, a classification label and a detection frame position, and obtaining an optimal target detection model through multi-round iterative training so as to realize the detection label of the pedestrian target; the pedestrian detection submodule carries out target tracking on a marked pedestrian target, a TransTransTransck target tracking model is constructed, a pedestrian target feature map of a current frame is used as an input key, a series of pedestrian target features of a past frame and a series of learnable queries are used as input queries, a learnable query is constructed to detect a new pedestrian target and output a detection boundary frame, the pedestrian target features of the past frame are generated by past frame detection and used for positioning an object and an output track frame which exist in the current frame, output is completed by intersection and matching of the detection frame and the track frame, the pedestrian density detection submodule clusters the positions of the pedestrian target marks by using a clustering algorithm, slow traffic density is visually presented by assisting a thermodynamic diagram mode, and the mask detection submodule is used for detecting whether a school park in an image video wears a mask or not;
s4-1: automatically studying and judging whether pedestrians cross a reference line in the image video by using a line crossing detection submodule, if the line crossing occurs, automatically sending early warning information, calculating the distance between every two pedestrians in the image video by using a distance detection submodule, judging whether potential epidemic situation spreading risks exist between the two pedestrians, and estimating the speed of a target pedestrian in the image video by using a speed detection submodule;
s5-1: the method comprises the steps of identifying an epidemic situation propagation high-risk area with high school people flow and high track overlapping degree in an image video by using a high-risk area identification submodule, monitoring violation behaviors of school personnel in the image video in real time by using an illegal personnel monitoring submodule, counting the personnel flow entering and exiting different areas in the image video by using a personnel flow counting submodule, judging potential propagation risks by using a propagation risk early-warning submodule through the image video, and providing a quick early-warning service for a school manager.
The invention has the beneficial effects that:
the campus epidemic situation monitoring and early warning system overcomes the defects of time and labor waste, limited monitoring range and low response speed of the existing mainstream manual inspection mode and epidemic situation risk prevention and control technology products, realizes accurate campus epidemic situation prevention and control by accurately extracting refined track information of people going out in a campus and assisting campus epidemic situation flow adjustment data, and provides accurate epidemic situation risk monitoring and early warning service for campus managers at people, places and moments.
Drawings
FIG. 1 is a schematic diagram of an overall module connection structure of the smart campus epidemic situation risk monitoring and early warning system provided by the invention;
FIG. 2 is a schematic diagram of the main steps of the smart campus epidemic risk monitoring and early warning method according to the present invention;
FIG. 3 is a schematic diagram illustrating specific steps of the smart campus epidemic risk monitoring and early warning method according to the present invention;
fig. 4 is a schematic view of a specific implementation flow of the smart campus epidemic risk monitoring and early warning method provided by the present invention.
Detailed Description
The present invention will be further illustrated with reference to the accompanying drawings and specific embodiments, which are to be understood as merely illustrative of the invention and not as limiting the scope of the invention.
Referring to fig. 1 to 4, an embodiment of the invention provides an epidemic situation risk monitoring and early warning system for a smart campus, as shown in fig. 1, the system includes an image data acquisition and processing module, an identity identification module, a target detection module, a track identification module, and an epidemic situation risk detection module.
The image data acquisition processing module is used for acquiring the flow condition of the school crowd and carrying out preprocessing according to the flow data, wherein the preprocessing comprises data enhancement, data annotation, data set division and the like;
specifically, the image data acquisition and processing module comprises an image data acquisition submodule and an image data preprocessing submodule;
the image data acquisition submodule acquires the campus crowd flowing video through externally accessed aerial photography equipment and a plurality of cameras, and detects and tracks different samples in the acquired image video to establish an image data set;
the image data preprocessing submodule is used for operating the acquired image video, and comprises the steps of turning, rotating, shearing, splicing and mosaic enhancing the image video, carrying out image morphological operation on the image video, selecting a modeling image video by using an image pyramid technology, carrying out artificial annotation on the image by using an image annotation software tool, determining the size of a prior marking frame by using a clustering technology, acquiring an image data set, and dividing the collected data into a training set, a verification set and a test set according to the characteristics of the data set.
The identity distinguishing module is used for identifying the identity information of the target based on the face recognition and the gesture recognition of the pedestrian.
Specifically, the identity recognition module includes a face recognition sub-module and a gesture recognition sub-module.
The face recognition sub-module is externally connected into a campus face library, a Cascade Cascade classifier is constructed based on PCA dimension reduction and LDA feature extraction technologies, the feature of key points of the face and the gray level change of the key points of the face are recognized, a face recognition model is trained according to different face feature recognition, and the recognized face features are matched with targets in the campus face library to perform accurate identity recognition.
The posture recognition sub-module is based on image segmentation modeling of Masked R-CNN to segment a pedestrian object from an image video, and captures human posture key position information in a human behavior expression process by using a 2D key point detection algorithm to extract human key representation points and determine target identity information.
The target detection module is used for detecting the pedestrian target state of the image data.
Specifically, the object detection module includes a pedestrian detection sub-module, a pedestrian density detection sub-module, and a mask detection sub-module.
The pedestrian detection submodule is used for detecting a pedestrian target in an image video, constructing a multi-target detection model based on a transform framework, and realizing detection marking of the pedestrian target by introducing a multi-head self-attention mechanism, wherein a CNN convolutional neural network is used for carrying out primary feature extraction on pedestrian target features in a video image, a transform network is constructed for carrying out accurate filtering and nonlinear transformation on the extracted pedestrian target features, a prediction model is constructed by associating feature information, classification labels and detection frame positions, and an optimal target detection model is obtained through multi-round iterative training.
The pedestrian detection sub-module carries out target tracking on the marked pedestrian target, constructs a TransTransTransTransTransTransck target tracking model, takes a pedestrian target feature map of a current frame as an input key, takes pedestrian target features of a series of past frames and a series of learnable queries as input queries, and detects a new pedestrian target and outputs a detection boundary frame by constructing the learnable queries, wherein the pedestrian target features of the past frames are generated by past frame detection and used for positioning an object and an output track frame existing in the current frame, and the output is completed by intersection and matching of the detection frame and the track frame.
The pedestrian density detection submodule utilizes a clustering algorithm to cluster the positions of pedestrian target marks and visually presents the slow traffic density in a thermodynamic diagram mode.
The mask detection submodule is used for detecting whether school garden pedestrians wear the mask in the image video.
The track recognition module is used for detecting the action range and the action mode of the pedestrian.
Specifically, the trajectory identification module includes an over-line detection sub-module, a distance detection sub-module, and a speed detection sub-module.
The line crossing detection submodule is used for automatically judging whether a pedestrian crosses a reference line in an image video, and if the line crossing occurs, early warning information is automatically sent out.
The distance detection submodule is used for calculating the distance between every two pedestrians in the image video and judging whether the two pedestrians have potential epidemic propagation risks.
The speed detection submodule is used for estimating the speed of the target pedestrian in the image video.
The epidemic situation risk detection module is used for acquiring data detected by the identity identification module, the target detection module and the track identification module to identify the flow of the pedestrians in the campus, and studying, judging and early warning the flow condition of the crowds in the campus. The epidemic risk detection module acquires the ID of a person from the identity identification module, acquires the category (such as people, vehicles and the like) of the object and the position information (image coordinates of the object) in the image from the object detection module, and acquires the continuous image coordinate position information of the person in the image, which is changed along with time, from the track identification.
Specifically, the epidemic situation risk detection module includes a high risk area identification submodule, an illegal person monitoring submodule, a personnel flow statistics submodule, and a propagation risk early warning submodule.
The high risk area identification submodule is used for identifying epidemic propagation high risk areas with large school garden people flow and high track overlapping degree in the image video.
And the illegal person monitoring submodule is used for monitoring the illegal behaviors of the campus persons in the image video in real time.
And the personnel flow statistics submodule is used for counting the personnel flow entering and exiting different areas of the campus in the image video.
And the propagation risk early warning submodule is used for studying and judging potential propagation risks through the image video and providing a quick early warning service for a campus manager, and the quick early warning service comprises system early warning, short message early warning and mail early warning. The system early warning carries out risk early warning through an information bullet frame in the system, the short message early warning automatically triggers an early warning mechanism according to a study and judgment risk threshold value, early warning short messages are timely sent to a campus manager, and the mail early warning is carried out by butting an external mail system through the system, compiling an automatic mail sending program and carrying out mail early warning. Specifically, when certain events are triggered through image judgment, such as no mask, too close pedestrian distance, offline personnel and non-campus personnel, the risk propagation early warning sub-module sends early warning information to a system, a mailbox and a mobile phone.
As shown in fig. 2, a method for monitoring and early warning of epidemic situation in a smart campus comprises the following steps:
s1: the image data acquisition and processing module is used for acquiring the flow condition of the school crowd and carrying out preprocessing, including preprocessing such as data enhancement, data annotation and data set division, according to the flow data.
S2: and identifying the target identity information based on the face identification and the gesture identification of the pedestrian by using an identity identification module.
S3: and detecting the pedestrian target state of the image data by using a target detection module.
S4: and detecting the action range and the action mode of the pedestrian by using the track identification module.
S5: data detected by the identity identification module, the target detection module and the track identification module are acquired by the epidemic situation risk detection module to identify the flow of the pedestrians in the campus, and the flow condition of the crowds in the campus is researched, judged and early warned.
As shown in fig. 3: further, the steps S1, S2, S3, S4, and S5 further include the following steps:
s1-1: the method comprises the steps of utilizing an image data acquisition submodule to acquire campus crowd flowing videos through an aerial photographing device and a plurality of cameras which are externally accessed, detecting and tracking different samples in the acquired image videos, establishing an image data set, utilizing an image data preprocessing submodule to operate the acquired image videos, turning, rotating, shearing, splicing and mosaic enhancing the image videos, carrying out image morphological operation on the image videos, selecting and modeling the image videos by utilizing an image pyramid technology, manually marking the images by utilizing an image marking software tool, determining the sizes of prior marking frames by utilizing a clustering technology, acquiring the image data set, and dividing collected data into a training set, a verification set and a test set according to the characteristics of the data set.
S2-1: the method comprises the steps of externally connecting a face recognition submodule into a campus face library, constructing a Cascade Cascade classifier based on PCA dimension reduction and LDA feature extraction technologies, recognizing key point features and gray level changes of a face, recognizing according to different face features, training a face recognition model, matching targets in the campus face library through the recognized face features, carrying out accurate identity recognition, utilizing a posture recognition submodule to carry out image segmentation modeling based on mask R-CNN to segment pedestrian objects from image videos, utilizing a 2D key point detection algorithm to capture position information of the human posture key points in a human behavior expression process, extracting human key characterization points and determining target identity information.
S3-1: the method comprises the steps of detecting a pedestrian target in an image video by using a pedestrian detection submodule, constructing a multi-target detection model based on a transform architecture, and detecting whether a traffic mask is visually detected or not by using a multi-head self-attention mechanism, wherein a CNN convolutional neural network is used for carrying out primary feature extraction on pedestrian target features in the video image, a transform network is used for carrying out precise filtering and nonlinear transformation on the extracted pedestrian target features, a prediction model is constructed by associating feature information, a classification label and a detection frame position, an optimal target detection model is obtained by multi-round iterative training, so that detection marking on the pedestrian target is realized, the pedestrian detection submodule carries out target tracking on the marked pedestrian target, a TransTransTransTransTrans target tracking model is constructed, a pedestrian target feature map of a current frame is taken as an input key, a series of pedestrian target features of past frames and a series of learnable queries are taken as input queries, a learnable query is constructed to detect a new pedestrian target and output a detection boundary frame, wherein the pedestrian target features of the past frame are generated by past frame detection, the object and the output track in the current frame are used for positioning and outputting a slow output track comparison frame, and a clustering density comparison mask is used for detecting whether a traffic mask, and a traffic density comparison algorithm is carried out by using an auxiliary clustering algorithm.
S4-1: the method comprises the steps that whether pedestrians cross a reference line in an image video or not is automatically judged by using a line crossing detection submodule, if line crossing occurs, early warning information is automatically sent out, the distance detection submodule is used for calculating the distance between every two pedestrians in the image video, whether potential epidemic situation propagation risks exist in the two pedestrians or not is judged, and the speed detection submodule is used for estimating the speed of a target pedestrian in the image video.
S5-1: the method comprises the steps of identifying an epidemic situation propagation high-risk area with high school people flow and high track overlapping degree in an image video by using a high-risk area identification submodule, monitoring violation behaviors of school personnel in the image video in real time by using an illegal personnel monitoring submodule, counting the personnel flow entering and exiting different areas in the image video by using a personnel flow counting submodule, judging potential propagation risks by using a propagation risk early-warning submodule through the image video, and providing a quick early-warning service for a school manager.
It should be noted that the above-mentioned contents only illustrate the technical idea of the present invention, and the protection scope of the present invention is not limited thereby, and it will be apparent to those skilled in the art that several modifications and embellishments can be made without departing from the principle of the present invention, and these modifications and embellishments fall within the protection scope of the claims of the present invention.

Claims (8)

1. The utility model provides a wisdom campus epidemic situation risk monitoring and early warning system which characterized in that includes: the system comprises an image data acquisition and processing module, an identity distinguishing module, a target detection module, a track identification module and an epidemic risk detection module;
the image data acquisition and processing module is used for acquiring the flow condition of the school crowd and preprocessing the flow data, including data enhancement, data annotation, data set division and other preprocessing, and comprises an image data acquisition submodule and an image data preprocessing submodule;
the identity distinguishing module is used for distinguishing target identity information based on pedestrian face recognition and pedestrian posture recognition and comprises a face recognition submodule and a posture recognition submodule;
the target detection module is used for detecting the pedestrian target state of the image data and comprises a pedestrian detection sub-module, a pedestrian density detection sub-module and a mask detection sub-module;
the track identification module is used for detecting the action range and the action mode of the pedestrian and comprises a line crossing detection submodule, a distance detection submodule and a speed detection submodule;
the epidemic situation risk detection module is used for acquiring data detected by the identity identification module, the target detection module and the track identification module to identify campus pedestrian flow, studying and judging and early warning the campus crowd flow condition, and comprises a high-risk area identification submodule, an illegal person monitoring submodule, a personnel flow counting submodule and a propagation risk early warning submodule.
2. The intelligent campus epidemic risk monitoring and early warning system of claim 1, wherein:
the image data acquisition submodule acquires the campus crowd flowing video through externally accessed aerial photography equipment and a plurality of cameras, and detects and tracks different samples in the acquired image video to establish an image data set;
the image data preprocessing submodule is used for operating the acquired image video, and comprises the steps of turning, rotating, shearing, splicing and mosaic enhancing the image video, carrying out image morphological operation on the image video, selecting a modeling image video by using an image pyramid technology, carrying out artificial annotation on the image by using an image annotation software tool, determining the size of a prior marking frame by using a clustering technology, acquiring an image data set, and dividing the collected data into a training set, a verification set and a test set according to the characteristics of the data set.
3. The intelligent campus epidemic risk monitoring and early warning system of claim 1, wherein:
the face recognition sub-module is externally connected into a campus face library, a Cascade Cascade classifier is built based on PCA dimension reduction and LDA feature extraction technologies, face key point features and gray level changes of the face are recognized, a face recognition model is trained according to different face feature recognition, and accurate identity recognition is carried out by matching recognized face features with targets in the campus face library;
the posture recognition sub-module is based on image segmentation modeling of Masked R-CNN to segment pedestrian objects from image videos, and captures human body posture key position information in a human body behavior expression process by using a 2D key point detection algorithm to extract human body key representation points and determine target identity information.
4. The intelligent campus epidemic risk monitoring and early warning system of claim 1, wherein:
the pedestrian detection submodule is used for detecting a pedestrian target in an image video, constructing a multi-target detection model based on a transform framework, extracting the characteristics of the pedestrian target in the video image primarily by using a CNN convolutional neural network through introducing a multi-head self-attention mechanism, performing accurate filtering and nonlinear transformation on the extracted characteristics of the pedestrian target through constructing the transform network, constructing a prediction model through associating characteristic information, a classification label and a detection frame position, and obtaining an optimal target detection model through multi-round iterative training so as to realize detection marking of the pedestrian target;
the pedestrian detection submodule carries out target tracking on a marked pedestrian target, constructs a TransTransTransTransTransTransTransTransck target tracking model, takes a pedestrian target feature map of a current frame as an input key, takes pedestrian target features of a series of past frames and a series of learnable queries as input queries, and detects a new pedestrian target and outputs a detection boundary frame by constructing the learnable queries, wherein the pedestrian target features of the past frames are generated by past frame detection and used for positioning an object and an output track frame existing in the current frame, and the output is completed by intersection and matching of the detection frame and the track frame;
the pedestrian density detection submodule utilizes a clustering algorithm to cluster pedestrian target mark positions and visually presents the slow traffic density in a thermodynamic diagram manner;
the mask detection submodule is used for detecting whether school garden pedestrians wear the mask in the image video.
5. The intelligent campus epidemic risk monitoring and early warning system of claim 1, wherein:
the line crossing detection submodule is used for automatically judging whether a pedestrian crosses a reference line in an image video, and if the pedestrian crosses the reference line, early warning information is automatically sent out;
the distance detection submodule is used for calculating the distance between every two pedestrians in the image video and judging whether the two pedestrians have potential epidemic situation spread risks or not;
the speed detection submodule is used for estimating the speed of a target pedestrian in the image video.
6. The intelligent campus epidemic risk monitoring and early warning system of claim 1, wherein:
the high risk area identification submodule is used for identifying epidemic situation propagation high risk areas with large school garden people flow and high track overlapping degree in the image video;
the illegal person monitoring submodule is used for monitoring the illegal behavior of the campus person in the image video in real time;
the personnel flow statistics submodule is used for counting the personnel flow entering and exiting different areas of the campus in the image video;
the propagation risk early warning submodule is used for studying and judging potential propagation risks through image videos and providing quick early warning service for a campus manager, and comprises system early warning, short message early warning and mail early warning, wherein the system early warning carries out risk early warning through an information bullet frame inside the system, the short message early warning automatically triggers an early warning mechanism according to a study and judgment risk threshold value, early warning short messages are timely sent to the campus manager, the mail early warning is connected with an external mail system through the system, an automatic mail sending program is compiled, and mail early warning is carried out.
7. A smart campus epidemic situation risk monitoring and early warning method is characterized by comprising the following steps:
s1: the method comprises the steps that an image data acquisition and processing module is used for acquiring the flow condition of a campus crowd, and preprocessing including data enhancement, data annotation, data set division and the like is carried out according to flow data;
s2: identifying target identity information based on pedestrian face identification and pedestrian posture identification by using an identity identification module;
s3: detecting the pedestrian target state of the image data by using a target detection module;
s4: the track identification module is used for detecting the action range and the action mode of the pedestrian;
s5: data detected by the identity identification module, the target detection module and the track identification module are acquired by the epidemic situation risk detection module to identify the flow of the pedestrians in the campus, and the flow condition of the pedestrians in the campus is researched, judged and early warned.
8. The method as claimed in claim 8, wherein the steps S1, S2, S3, S4 and S5 include the following steps:
s1-1: acquiring a campus crowd flowing video by using an image data acquisition submodule through externally accessing aerial equipment and a plurality of cameras, detecting and tracking different samples in the acquired image video, establishing an image data set, using an image data preprocessing submodule for operating the acquired image video, wherein the image data preprocessing submodule comprises the steps of turning, rotating, shearing, splicing and mosaic enhancing the image video, carrying out image morphological operation on the image video, selecting a modeling image video by using an image pyramid technology, manually marking the image by using an image marking software tool, determining the size of a prior marking frame by using a clustering technology, acquiring the image data set, and dividing the collected data into a training set, a verification set and a test set according to the characteristics of the data set;
s2-1: the method comprises the steps that a face recognition submodule is externally connected into a campus face library, a Cascade Cascade classifier is built based on PCA dimension reduction and LDA feature extraction technologies, face key point features and gray level changes are recognized, a face recognition model is trained according to different face feature recognition, targets in the campus face library are matched through the recognized face features, accurate identity recognition is conducted, a posture recognition submodule is used for image segmentation modeling based on Masked R-CNN to segment pedestrian objects from image videos, capture of position information of human body posture key points in a human body behavior expression process is achieved through a 2D key point detection algorithm, human body key characterization points are extracted, and target identity information is determined;
s3-1: detecting a pedestrian target in an image video by using a pedestrian detection submodule, constructing a multi-target detection model based on a transform architecture, extracting the characteristics of the pedestrian target in the video image by introducing a multi-head self-attention mechanism, performing primary characteristic extraction on the characteristics of the pedestrian target by using a CNN convolutional neural network, performing precise filtering and nonlinear transformation on the extracted characteristics of the pedestrian target by constructing a transform network, constructing a prediction model by associating characteristic information, a classification label and a detection frame position, and obtaining an optimal target detection model by multi-round iterative training so as to realize detection marking of the pedestrian target, performing target tracking on the marked pedestrian target by using a pedestrian detection submodule, constructing a TransTransTransTransTransTransLock target tracking model, taking a pedestrian target characteristic map of a current frame as an input key, taking the target characteristics of a series of past frames and the series of learnable queries as input queries, detecting a new pedestrian target and outputting a detection boundary frame by constructing the learnable queries, wherein the pedestrian target characteristics of the past frames are generated by detection of the past frames, are used for positioning objects and outputting the current frame in the current frame, outputting a pedestrian target characteristic map, and outputting a comparison frame, and outputting a comparison track, and outputting a detection algorithm for detecting whether the pedestrian density of the mask is visually displayed by using a pedestrian detection mask, and performing a hot-assisted clustering algorithm on the detection mask;
s4-1: automatically studying and judging whether pedestrians cross a reference line in the image video by using a line crossing detection submodule, if the line crossing occurs, automatically sending early warning information, calculating the distance between every two pedestrians in the image video by using a distance detection submodule, judging whether potential epidemic situation spreading risks exist between the two pedestrians, and estimating the speed of a target pedestrian in the image video by using a speed detection submodule;
s5-1: the method comprises the steps of identifying an epidemic situation propagation high-risk area with high school people flow and high track overlapping degree in an image video by using a high-risk area identification submodule, monitoring violation behaviors of school personnel in the image video in real time by using an illegal personnel monitoring submodule, counting the personnel flow entering and exiting different areas in the image video by using a personnel flow counting submodule, judging potential propagation risks by using a propagation risk early-warning submodule through the image video, and providing a quick early-warning service for a school manager.
CN202210765657.1A 2022-07-01 2022-07-01 Intelligent campus epidemic risk monitoring and early warning system and method Pending CN115169673A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210765657.1A CN115169673A (en) 2022-07-01 2022-07-01 Intelligent campus epidemic risk monitoring and early warning system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210765657.1A CN115169673A (en) 2022-07-01 2022-07-01 Intelligent campus epidemic risk monitoring and early warning system and method

Publications (1)

Publication Number Publication Date
CN115169673A true CN115169673A (en) 2022-10-11

Family

ID=83488965

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210765657.1A Pending CN115169673A (en) 2022-07-01 2022-07-01 Intelligent campus epidemic risk monitoring and early warning system and method

Country Status (1)

Country Link
CN (1) CN115169673A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117079351A (en) * 2023-10-12 2023-11-17 成都崇信大数据服务有限公司 Method and system for analyzing personnel behaviors in key areas

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090324010A1 (en) * 2008-06-26 2009-12-31 Billy Hou Neural network-controlled automatic tracking and recognizing system and method
CN109886245A (en) * 2019-03-02 2019-06-14 山东大学 A kind of pedestrian detection recognition methods based on deep learning cascade neural network
CN109886179A (en) * 2019-02-18 2019-06-14 深圳视见医疗科技有限公司 The image partition method and system of cervical cell smear based on Mask-RCNN
CN112992372A (en) * 2021-03-09 2021-06-18 深圳前海微众银行股份有限公司 Epidemic situation risk monitoring method, device, equipment, storage medium and program product
CN113327103A (en) * 2021-08-03 2021-08-31 深圳市知酷信息技术有限公司 Intelligent campus epidemic situation on-line monitoring and early warning method, system and storage medium
CN113673489A (en) * 2021-10-21 2021-11-19 之江实验室 Video group behavior identification method based on cascade Transformer
CN114141370A (en) * 2021-11-08 2022-03-04 广东省电信规划设计院有限公司 Information management method and device for epidemic situation prevention and control and computer storage medium
CN114202740A (en) * 2021-12-07 2022-03-18 大连理工大学宁波研究院 Pedestrian re-identification method based on multi-scale feature fusion

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090324010A1 (en) * 2008-06-26 2009-12-31 Billy Hou Neural network-controlled automatic tracking and recognizing system and method
CN109886179A (en) * 2019-02-18 2019-06-14 深圳视见医疗科技有限公司 The image partition method and system of cervical cell smear based on Mask-RCNN
CN109886245A (en) * 2019-03-02 2019-06-14 山东大学 A kind of pedestrian detection recognition methods based on deep learning cascade neural network
CN112992372A (en) * 2021-03-09 2021-06-18 深圳前海微众银行股份有限公司 Epidemic situation risk monitoring method, device, equipment, storage medium and program product
CN113327103A (en) * 2021-08-03 2021-08-31 深圳市知酷信息技术有限公司 Intelligent campus epidemic situation on-line monitoring and early warning method, system and storage medium
CN113673489A (en) * 2021-10-21 2021-11-19 之江实验室 Video group behavior identification method based on cascade Transformer
CN114141370A (en) * 2021-11-08 2022-03-04 广东省电信规划设计院有限公司 Information management method and device for epidemic situation prevention and control and computer storage medium
CN114202740A (en) * 2021-12-07 2022-03-18 大连理工大学宁波研究院 Pedestrian re-identification method based on multi-scale feature fusion

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117079351A (en) * 2023-10-12 2023-11-17 成都崇信大数据服务有限公司 Method and system for analyzing personnel behaviors in key areas
CN117079351B (en) * 2023-10-12 2024-01-30 成都崇信大数据服务有限公司 Method and system for analyzing personnel behaviors in key areas

Similar Documents

Publication Publication Date Title
Fang et al. Detecting non-hardhat-use by a deep learning method from far-field surveillance videos
CN110738127B (en) Helmet identification method based on unsupervised deep learning neural network algorithm
CN108062349B (en) Video monitoring method and system based on video structured data and deep learning
Han et al. Deep learning-based workers safety helmet wearing detection on construction sites using multi-scale features
CN107145851A (en) Constructions work area dangerous matter sources intelligent identifying system
CN113516076B (en) Attention mechanism improvement-based lightweight YOLO v4 safety protection detection method
CN105070053A (en) Intelligent traffic monitoring camera for identifying vehicle illegal movement modes
CN106156688A (en) A kind of dynamic human face recognition methods and system
Hsu et al. Passenger flow counting in buses based on deep learning using surveillance video
CN105303191A (en) Method and apparatus for counting pedestrians in foresight monitoring scene
CN104091176A (en) Technology for applying figure and head portrait comparison to videos
CN106355154B (en) Method for detecting frequent passing of people in surveillance video
CN102902986A (en) Automatic gender identification system and method
CN112396658A (en) Indoor personnel positioning method and positioning system based on video
CN113903081A (en) Visual identification artificial intelligence alarm method and device for images of hydraulic power plant
CN107977639A (en) A kind of face definition judgment method
CN111274886B (en) Deep learning-based pedestrian red light running illegal behavior analysis method and system
CN110852179B (en) Suspicious personnel invasion detection method based on video monitoring platform
Park et al. Detection of construction workers in video frames for automatic initialization of vision trackers
CN112183472A (en) Method for detecting whether test field personnel wear work clothes or not based on improved RetinaNet
CN104463232A (en) Density crowd counting method based on HOG characteristic and color histogram characteristic
CN105657372A (en) Method and system for realizing intelligent detection and early warning of on-duty guard posts by videos
CN115169673A (en) Intelligent campus epidemic risk monitoring and early warning system and method
CN105095891A (en) Human face capturing method, device and system
CN113537019A (en) Detection method for identifying wearing of safety helmet of transformer substation personnel based on key points

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination