CN111611927A - Method for identifying unsafe behaviors of coal mine workers based on human body postures - Google Patents

Method for identifying unsafe behaviors of coal mine workers based on human body postures Download PDF

Info

Publication number
CN111611927A
CN111611927A CN202010438252.8A CN202010438252A CN111611927A CN 111611927 A CN111611927 A CN 111611927A CN 202010438252 A CN202010438252 A CN 202010438252A CN 111611927 A CN111611927 A CN 111611927A
Authority
CN
China
Prior art keywords
human body
coal mine
unsafe behaviors
joints
behaviors
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010438252.8A
Other languages
Chinese (zh)
Inventor
陈方云
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Changsha Mingben Information Technology Co ltd
Original Assignee
Changsha Mingben Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Changsha Mingben Information Technology Co ltd filed Critical Changsha Mingben Information Technology Co ltd
Priority to CN202010438252.8A priority Critical patent/CN111611927A/en
Publication of CN111611927A publication Critical patent/CN111611927A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G06V10/464—Salient features, e.g. scale invariant feature transforms [SIFT] using a plurality of salient features, e.g. bag-of-words [BoW] representations

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Biophysics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Psychiatry (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Biology (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a method for identifying unsafe behaviors of coal mine workers based on human body postures, which comprises the steps of firstly, presetting unsafe behaviors of a plurality of coal mine workers to obtain video information of the unsafe behaviors, determining the positions and the classes of human body joints in the video information through a human body detector, and calculating affinity vector fields of all groups of adjacent human body joints in the unsafe behaviors to serve as threshold values; then, acquiring a working image of a coal mine worker through a camera, and predicting the position and the category of a human joint in the working image by using a regression method; finally, calculating an affinity vector field of adjacent human body joints in the working image and comparing the affinity vector field with a threshold, judging that the behavior of a worker is illegal within a certain difference range, and alarming; if the coal mine worker is not in the range, the coal mine worker can be judged to be illegal, and the coal mine worker can give an alarm to the three illegal behaviors, so that the three illegal behaviors are reduced.

Description

Method for identifying unsafe behaviors of coal mine workers based on human body postures
Technical Field
The invention relates to the technical field of safety monitoring, in particular to a method for identifying unsafe behaviors of coal mine workers based on human body postures.
Background
In coal mine production, accidents caused by illegal operation, illegal command and work discipline violation are reflected by inevitable results of frequent behaviors of personnel due to non-normative behaviors of the personnel. The main body of the 'three-phase violation' is a person, and the 'three-phase violation' phenomena of field management personnel and operation workers occur occasionally, are forbidden frequently, and seriously threaten the safety production of mines and the life safety of workers.
At present, the coal mine industry mainly monitors three violations by video monitoring and on-site manual monitoring, and a few mining areas introduce automatic monitoring systems to establish an artificial intelligent model of the three violations for safety behavior alarming. However, due to the particularity of the underground environment of the coal mine and the efficiency of a computer target recognition algorithm and a convolutional neural network deep learning algorithm, the common three-default behavior perception method has the following defects:
1. the efficiency of a common target recognition algorithm needs to be improved, underground light is dark, the condition is complex, and in order to meet the requirement of a computer vision algorithm, a high-definition camera which is expensive must be installed again.
2. In darker light or when the helmet is not of a conventional kind, the helmet cannot be identified.
3. The remote fuzzy human face is difficult to identify, and the identity of the violator is difficult to confirm.
4. The process of analyzing and recognizing the attitude structured data into behaviors in the attitude analysis result of coal miners is easily interfered by factors such as underground environment, light, shielding and the like, and false alarm conditions exist in the 'three-violation' behaviors. Aiming at the diversity and complexity of data, a behavior recognition algorithm with strong robustness needs to be added.
5. In order to reduce overfitting, a large amount of data is needed for model training of the common deep learning model, a sufficient data set is difficult to provide in the conventional coal mine production process, three violations are various, and a three violating behavior artificial intelligence model with high accuracy can be trained by small sample data urgently.
In order to effectively reduce and eliminate the phenomenon of 'three violations', starting from aspects such as safety consciousness, quality improvement, field environment and field management, and the like, equipment for carrying out real-time analysis and early warning on a deep learning artificial intelligent model for sensing 'three violations' behaviors of underground coal mine personnel with low cost, high efficiency and high accuracy is urgently needed.
Disclosure of Invention
Aiming at the defects in reality, the invention provides a method for identifying unsafe behaviors of coal mine workers based on human body postures.
In order to solve the technical problems, the technical scheme of the invention is as follows:
a method for identifying unsafe behaviors of coal mine workers based on human body postures comprises the following steps:
s1, presetting unsafe behaviors of a plurality of coal mine workers, acquiring the video information of the unsafe behaviors, determining the positions and the types of human body joints in the video information through a human body detector, and calculating affinity vector fields of all groups of adjacent human body joints in the unsafe behaviors to serve as threshold values;
s2, acquiring a working image of a coal mine worker through a camera, and predicting the position and the type of a human joint in the working image by using a regression method;
s3, calculating an affinity vector field of adjacent human body joints in the working image, comparing the affinity vector field with a threshold value, judging that the behavior of a worker is illegal within a certain difference range, and alarming; if the value is not in the range, the behavior is judged not to be illegal.
Preferably, in step S2, after the working image is acquired, the color original is first input into the first ten layers of VGG-16 to obtain a Feature map, and then input into the first loop to obtain the part detection maps S1And a sum vector field L1The result S of the previous cycle among the subsequent cycles1And L1Adding Feature map as input, and obtaining final result S after a plurality of cyclestAnd LtThen to the obtained StAnd LtThe position and the category of the human body joint in the working image are predicted by using a regression method.
Preferably, the affinity vector field calculation function is as follows,
Figure BDA0002503113570000031
Figure BDA0002503113570000032
Figure BDA0002503113570000033
Figure BDA0002503113570000034
Figure BDA0002503113570000035
Figure BDA0002503113570000036
preferably, in step S3, when calculating the affinity vector fields of adjacent human joints in the working image, the link trends (E) of all edges of the complete bipartite graph are calculated between the adjacent joint points, the optimal match is calculated,
the calculation formula is as follows:
Figure BDA0002503113570000037
Figure BDA0002503113570000038
Figure BDA0002503113570000039
Figure BDA00025031135700000310
Figure BDA00025031135700000311
preferably, the function of the human joint position prediction is as follows:
Figure BDA0002503113570000041
Figure BDA0002503113570000042
Figure BDA0002503113570000043
preferably, in the step S2, the positions of the human joints in the working image are predicted by using a regression method, and a plurality of smaller anchors are used to extract the features of each joint position in a larger image range.
Preferably, the human joint category includes the following parts: comprises left and right feet, left and right knees, left and right waist, left and right head, left and right elbows, left and right shoulders, left and right ears, left and right eye joints, nose joints and head joints positioned between the left and right shoulders.
By adopting the technical scheme, the method has the following beneficial effects:
1. data acquisition is based on original camera completely, saves the hardware spending, has avoided the repeated construction.
2. The computer vision target detection algorithm developed independently has higher accuracy, and various safety helmets can be accurately identified under dark light.
3. The independently developed computer vision target detection algorithm has higher accuracy, and the remote fuzzy human face can be accurately identified and positioned.
4. The three-violation behavior perception deep learning artificial intelligence model is improved by adopting the deep neural network classification algorithm of the structured data set, and the false alarm rate is greatly reduced.
5. The three-violation behavior perception deep learning artificial intelligence model is improved by adopting a deep neural network classification algorithm of the structured data set, and a more accurate model can be trained by using less data.
6. Detecting based on the multi-scale characteristic image: and predicting on the convolution characteristic graphs of multiple scales to detect targets with different sizes, so that the detection precision of small target objects is improved to a certain extent.
7. Candidate areas are sampled on feature maps with different scales, and the recall rate of detection and the detection effect of small targets are improved to a certain extent.
Drawings
FIG. 1 is a diagram of a target detection network architecture in accordance with the present invention;
FIG. 2 is a schematic diagram of target coordinates and category information;
FIG. 3 is a schematic view of an affinity vector field;
FIG. 4 is a graph of link trends for computing all edges of a complete bipartite graph between adjacent joint points;
FIG. 5 is a label diagram of joint position data;
FIG. 6 is a schematic of a cycle;
FIG. 7 is a schematic representation of human joint classes.
Detailed Description
The following further describes embodiments of the present invention with reference to the drawings. It should be noted that the description of the embodiments is provided to help understanding of the present invention, but the present invention is not limited thereto. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The embodiment provides a method for identifying unsafe behaviors of coal mine workers based on human body postures, which comprises the following steps:
s1, presetting unsafe behaviors of a plurality of coal mine workers, acquiring the video information of the unsafe behaviors, determining the positions and the types of human body joints in the video information through a human body detector, and calculating affinity vector fields of all groups of adjacent human body joints in the unsafe behaviors to serve as threshold values;
s2, acquiring a working image of a coal mine worker through a camera, and predicting the position and the type of a human joint in the working image by using a regression method;
s3, calculating an affinity vector field of adjacent human body joints in the working image, comparing the affinity vector field with a threshold value, judging that the behavior of a worker is illegal within a certain difference range, and alarming; if the value is not in the range, the behavior is judged not to be illegal.
In step S2, after the working image is acquired, the color original is first input to the first ten layers of VGG-16 to obtain a Feature map, and then input to the first cycle (see fig. 6) to obtain the part detection maps S1And a sum vector field L1The result S of the previous cycle among the subsequent cycles1And L1Adding Feature map as input, and obtaining final result S after a plurality of cyclestAnd LtThen to the obtained StAnd LtThe position and the category of the human body joint in the working image are predicted by using a regression method.
Wherein the affinity vector field calculation function (see FIG. 3) is as follows,
Figure BDA0002503113570000061
Figure BDA0002503113570000062
Figure BDA0002503113570000063
Figure BDA0002503113570000064
Figure BDA0002503113570000065
Figure BDA0002503113570000066
in step S3, when calculating the affinity vector fields of adjacent human joints in the working image, the link trends (E) of all edges of the complete bipartite graph are calculated between adjacent joint points, and the optimal matching is calculated (refer to fig. 4), which has the following calculation formula:
Figure BDA0002503113570000071
Figure BDA0002503113570000072
Figure BDA0002503113570000073
Figure BDA0002503113570000074
Figure BDA0002503113570000075
wherein the function of the human joint position prediction (see fig. 5) is as follows:
Figure BDA0002503113570000076
Figure BDA0002503113570000077
Figure BDA0002503113570000078
in step S2, the position of the human joint in the working image is predicted by using a regression method (refer to fig. 2), and features of each joint position are extracted by using a plurality of smaller anchors in a larger image range.
Wherein the human joint classes (see fig. 7) include the following parts: comprises left and right feet, left and right knees, left and right waist, left and right head, left and right elbows, left and right shoulders, left and right ears, left and right eye joints, nose joints and head joints positioned between the left and right shoulders.
The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. It will be apparent to those skilled in the art that various changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, and the scope of protection is still within the scope of the invention.

Claims (7)

1. A method for identifying unsafe behaviors of coal mine workers based on human body postures is characterized by comprising the following steps: the method comprises the following steps:
s1, presetting unsafe behaviors of a plurality of coal mine workers, acquiring the video information of the unsafe behaviors, determining the positions and the types of human body joints in the video information through a human body detector, and calculating affinity vector fields of all groups of adjacent human body joints in the unsafe behaviors to serve as threshold values;
s2, acquiring a working image of a coal mine worker through a camera, and predicting the position and the type of a human joint in the working image by using a regression method;
s3, calculating an affinity vector field of adjacent human body joints in the working image, comparing the affinity vector field with a threshold value, judging that the behavior of a worker is illegal within a certain difference range, and alarming; if the value is not in the range, the behavior is judged not to be illegal.
2. The method for identifying unsafe behaviors of coal mine workers based on human body gestures according to claim 1, wherein the method comprises the following steps: in step S2, after the working image is acquired, the color original image is first input into the first ten layers of VGG-16 to obtain a Feature map, and then input into the first loop to obtain the part detection maps S1And a sum vector field L1The result S of the previous cycle among the subsequent cycles1And L1Adding Feature map as input, and obtaining final result S after a plurality of cyclestAnd LtThen to the obtained StAnd LtThe position and the category of the human body joint in the working image are predicted by using a regression method.
3. The method for identifying unsafe behaviors of coal mine workers based on human body gestures according to claim 2, wherein the method comprises the following steps: the affinity vector field calculation function is as follows,
Figure FDA0002503113560000021
Figure FDA0002503113560000022
Figure FDA0002503113560000023
Figure FDA0002503113560000024
Figure FDA0002503113560000025
Figure FDA0002503113560000026
4. the method for identifying unsafe behaviors of coal mine workers based on human body gestures according to claim 3, wherein the method comprises the following steps: in step S3, when calculating the affinity vector fields of adjacent human joints in the working image, the link trends (E) of all edges of the complete bipartite graph are calculated between adjacent joint points, and the optimal matching is calculated, the calculation formula is as follows:
Figure FDA0002503113560000027
Figure FDA0002503113560000028
Figure FDA0002503113560000029
Figure FDA00025031135600000210
Figure FDA00025031135600000211
5. the method for identifying unsafe behaviors of coal mine workers based on human body gestures according to claim 2, wherein the method comprises the following steps: the function of the human joint position prediction is as follows:
Figure FDA0002503113560000031
Figure FDA0002503113560000032
Figure FDA0002503113560000033
6. the method for identifying unsafe behaviors of coal mine workers based on human body gestures according to claim 1, wherein the method comprises the following steps: in the step S2, the positions of human joints in the working image are predicted by using a regression method, and the characteristics of each joint position are extracted by adopting a plurality of smaller anchors in a larger image range.
7. The method for identifying unsafe behaviors of coal mine workers based on human body gestures according to claim 1, wherein the method comprises the following steps: the human joint category comprises the following parts: comprises left and right feet, left and right knees, left and right waist, left and right head, left and right elbows, left and right shoulders, left and right ears, left and right eye joints, nose joints and head joints positioned between the left and right shoulders.
CN202010438252.8A 2020-05-21 2020-05-21 Method for identifying unsafe behaviors of coal mine workers based on human body postures Pending CN111611927A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010438252.8A CN111611927A (en) 2020-05-21 2020-05-21 Method for identifying unsafe behaviors of coal mine workers based on human body postures

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010438252.8A CN111611927A (en) 2020-05-21 2020-05-21 Method for identifying unsafe behaviors of coal mine workers based on human body postures

Publications (1)

Publication Number Publication Date
CN111611927A true CN111611927A (en) 2020-09-01

Family

ID=72199780

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010438252.8A Pending CN111611927A (en) 2020-05-21 2020-05-21 Method for identifying unsafe behaviors of coal mine workers based on human body postures

Country Status (1)

Country Link
CN (1) CN111611927A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109376673A (en) * 2018-10-31 2019-02-22 南京工业大学 Method for identifying unsafe behaviors of underground coal mine personnel based on human body posture estimation
CN109492581A (en) * 2018-11-09 2019-03-19 中国石油大学(华东) A kind of human motion recognition method based on TP-STG frame
CN109949368A (en) * 2019-03-14 2019-06-28 郑州大学 A kind of human body three-dimensional Attitude estimation method based on image retrieval
CN111144263A (en) * 2019-12-20 2020-05-12 山东大学 Construction worker high-fall accident early warning method and device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109376673A (en) * 2018-10-31 2019-02-22 南京工业大学 Method for identifying unsafe behaviors of underground coal mine personnel based on human body posture estimation
CN109492581A (en) * 2018-11-09 2019-03-19 中国石油大学(华东) A kind of human motion recognition method based on TP-STG frame
CN109949368A (en) * 2019-03-14 2019-06-28 郑州大学 A kind of human body three-dimensional Attitude estimation method based on image retrieval
CN111144263A (en) * 2019-12-20 2020-05-12 山东大学 Construction worker high-fall accident early warning method and device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
朱建宝;许志龙;孙玉玮;马青山;: "基于OpenPose人体姿态识别的变电站危险行为检测" *

Similar Documents

Publication Publication Date Title
CN111898514B (en) Multi-target visual supervision method based on target detection and action recognition
CN111445524B (en) Scene understanding-based construction site worker unsafe behavior identification method
Fang et al. Computer vision aided inspection on falling prevention measures for steeplejacks in an aerial environment
CN111144263B (en) Construction worker high-falling accident early warning method and device
CN110188724A (en) The method and system of safety cap positioning and color identification based on deep learning
CN111241959A (en) Method for detecting person without wearing safety helmet through construction site video stream
CN111539276B (en) Method for detecting safety helmet in real time in power scene
CN110414400B (en) Automatic detection method and system for wearing of safety helmet on construction site
CN111428617A (en) Video image-based distribution network violation maintenance behavior identification method and system
CN111062303A (en) Image processing method, system and computer storage medium
CN112347916A (en) Power field operation safety monitoring method and device based on video image analysis
CN112184773A (en) Helmet wearing detection method and system based on deep learning
CN112966618B (en) Dressing recognition method, apparatus, device and computer readable medium
CN112800901A (en) Mine personnel safety detection method based on visual perception
CN116862244B (en) Industrial field vision AI analysis and safety pre-warning system and method
CN113361420A (en) Mine fire monitoring method, device and equipment based on robot and storage medium
CN112800975A (en) Behavior identification method in security check channel based on image processing
KR101972055B1 (en) CNN based Workers and Risky Facilities Detection System on Infrared Thermal Image
CN117612249A (en) Underground miner dangerous behavior identification method and device based on improved OpenPose algorithm
CN111291728A (en) Detection system, detection equipment and detection method for illegal crossing of transmission belt behavior
CN111611927A (en) Method for identifying unsafe behaviors of coal mine workers based on human body postures
CN116524428A (en) Electric power operation safety risk identification method based on target detection and scene fusion
CN111274888B (en) Helmet and work clothes intelligent identification method based on wearable mobile glasses
CN114359831A (en) Risk omen reasoning-oriented intelligent identification system and method for worker side-falling
CN112668535A (en) YOLOv3 model-based coal mine monkey vehicle violation detection and early warning method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination