CN108710839A - A kind of sentry's drowsiness intelligent monitor system based on deep learning computer vision - Google Patents
A kind of sentry's drowsiness intelligent monitor system based on deep learning computer vision Download PDFInfo
- Publication number
- CN108710839A CN108710839A CN201810434067.4A CN201810434067A CN108710839A CN 108710839 A CN108710839 A CN 108710839A CN 201810434067 A CN201810434067 A CN 201810434067A CN 108710839 A CN108710839 A CN 108710839A
- Authority
- CN
- China
- Prior art keywords
- sentry
- drowsiness
- deep learning
- computer vision
- system based
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 206010041349 Somnolence Diseases 0.000 title claims abstract description 23
- 238000013135 deep learning Methods 0.000 title claims abstract description 12
- 238000012545 processing Methods 0.000 claims abstract description 11
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 8
- 238000007405 data analysis Methods 0.000 claims abstract description 7
- 238000012549 training Methods 0.000 claims abstract description 5
- 238000012544 monitoring process Methods 0.000 claims abstract description 4
- 238000013480 data collection Methods 0.000 claims 1
- 239000011521 glass Substances 0.000 abstract description 2
- 210000000744 eyelid Anatomy 0.000 description 8
- 210000003128 head Anatomy 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 210000001508 eye Anatomy 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 206010057315 Daydreaming Diseases 0.000 description 2
- 230000004913 activation Effects 0.000 description 2
- 210000005252 bulbus oculi Anatomy 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000000034 method Methods 0.000 description 2
- 208000032140 Sleepiness Diseases 0.000 description 1
- 206010048232 Yawning Diseases 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000035479 physiological effects, processes and functions Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 230000037321 sleepiness Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/166—Detection; Localisation; Normalisation using acquisition arrangements
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Emergency Management (AREA)
- Image Analysis (AREA)
Abstract
The present invention relates to the technical fields of guard-on-duty condition monitoring system, more particularly, to a kind of sentry's drowsiness intelligent monitor system based on deep learning computer vision.A kind of sentry's drowsiness intelligent monitor system based on deep learning computer vision, wherein including data acquisition unit, data processing unit, data analysis unit and alarm unit, four units are sequentially connected.The present invention carries out Face datection using convolutional neural networks and face is aligned, model training is carried out by using abundant training data, actually available Generalization Capability can be reached, the state for identifying two characteristic portions of glasses and face of energy robust, and the pose estimation of out-feed head position, accuracy rate are high.
Description
Technical field
The present invention relates to the technical fields of guard-on-duty condition monitoring system, and depth is based on more particularly, to one kind
Practise sentry's drowsiness intelligent monitor system of computer vision.
Background technology
Existing sentry's sleepiness prevention system device is based primarily upon traditional computer vision technique, i.e., is identified from image artificial
Information in the signature analysis image of experience.Notification number is the Chinese invention patent of CN103632485A, is existed using lamp
Eyeball surface forms convex mirror effect, is combined using recurrence LBP operators, Gabor characteristic and color and carries out eyelid states identification,
Eyelid states analysis system is to person on duty's ocular movemeut status real time monitor.When person on duty is in closed-eye state for a long time, it is
System pops up red alarm information by sound and on picture immediately, carries out live prompt and alarm.Based on engineer's
Characteristics of image identifies that Generalization Capability is low, and accuracy of detection is not high, it is more likely that can't detect eyelid mesh because of the variation of environment
Mark.
In addition, being analyzed according to physiology, people usually will appear the alternating that opens and closes eyes before being absorbed in sleep, frequent drowsiness of nodding
The vigilance of job position request has been not achieved in the sentry of phenomenon, this stage, it is possible to therefore lead to accident on duty.Existing scheme
In, the just activation alarm when sentry is in closed-eye state for a long time, sentry has had completely passed into sleep, such alarm at this time
Clearly not in time.
The defect of the above-mentioned prior art is：
1, robustness is not high.Major embodiment from two aspect：（1）It is inaccurate based on traditional machine vision technique target identification,
Eyelid states analysis system forms convex mirror effect using lamp in eyeball surface in existing scheme, is calculated using recurrence LBP
Son, Gabor characteristic and color joint carry out eyelid states identification.The greatest drawback of wherein LBP operators is that it is covered only
Zonule within the scope of one radii fixus, this obviously cannot meet the needs of different sizes and frequency texture, be unsuitable for appearance
The different face of the colour of skin；And post is usually indoor environment, lacks lamp, convex lens effect unobvious, this all leads
It is not very stable to the tracking of eyelid states to cause existing scheme, eyelid target easy to be lost or causes to judge by accident.（2）Detect feature list
One, inadequate robust.The drowsiness of people often with yawn and puts first-class behavior act, be based only on the closure of eyelid to determine whether
Drowsiness has ignored the useful informations such as face and head pose so that differentiates that robustness is not high.
2, it alarms not in time.When sentry is in closed-eye state for a long time, just activation alarm, sentry have completely passed into
Sleep, such alarm are not in time.Because sentry would generally open and close eyes by one before being absorbed in sleep, nod frequently
The requirement in post has been not achieved in the state of mind in sleepy stage, the sentry in this stage, accident on duty may be caused to occur, more
Effective alarm should be able to identify sentry's state of mind in this stage and send out alarm.
Invention content
The present invention is at least one defect overcome described in the above-mentioned prior art, is provided a kind of based on deep learning computer
Sentry's drowsiness intelligent monitor system of vision introduces depth convolutional neural networks and carries out Face datection, face alignment and head appearance
Gesture is estimated, the robustness of system can be substantially improved；The time series modeling of state is carried out using Hidden Markov Model, it is complete in sentry
It identifies drowsiness before being absorbed in sleep and sends out alarm, solve the problems, such as alarm not in time.
The technical scheme is that：A kind of sentry's drowsiness intelligent monitor system based on deep learning computer vision,
Wherein, including data acquisition unit, data processing unit, data analysis unit and alarm unit, four units are sequentially connected.
Further, the data acquisition unit is the clear video camera of a station symbol, mounted on security personnel front upper place, can be clapped
Take the photograph positive position.
Further, the data processing unit is a host or server for being equipped with high-performance GPU, is deployed with pre-
First trained multitask concatenated convolutional neural network（MTCNN）, Face datection, face alignment and head pose can be realized simultaneously
Estimation.Specifically, the feature that this neural network model extracts multiple convolutional layers does multi-task learning, the shallower convolutional layer packet of the number of plies
Containing more details, local message, in crucial point location and Attitude estimation, include then for the deeper convolutional layer of the number of plies compared with
More Global Informations is used for Face datection.Build fusion-CNN by the characteristic information of multilayer being attached to linearly or nonlinearly
On one piece of subspace, multi-task learning, the facial characteristics and posture of output time are done using the fusion-CNN features exported
Estimation.
Further, the data analysis unit is deployed in same computing platform, with the output of data processing unit
As input, a trained HMM model is run, by the input in a period of time, sequentially analyzes the spiritual shape of sentry
State.
Further, the alarm unit is made of local warning device and backstage warning device two parts, by same
The control of a controller, when sentry leaves the post, drowsiness when, local warning device can carry out auditory tone cues, colleague backstage automatically at the scene
Warning device can be displayed in red warning message in monitored picture, and monitoring personnel on duty is prompted to be supervised in time.
Compared with prior art, advantageous effect is：The present invention carries out Face datection and face pair using convolutional neural networks
Together, model training is carried out by using abundant training data, actually available Generalization Capability can be reached, energy robust identifies
The state of two characteristic portions of glasses and face, and the pose estimation of out-feed head position, accuracy rate are high.By learning in advance
The Hidden Markov Model of foundation can utilize the sequential export of neural network, include the sequential of eyes face and head pose
State, judge when pawn ahead whether be in doze state, sentry be fully sunk in sleep before send out alarm, it is ensured that alarm and
Shi Xing.
Description of the drawings
Fig. 1 is overall schematic of the present invention.
Specific implementation mode
The attached figures are only used for illustrative purposes and cannot be understood as limitating the patent；It is attached in order to more preferably illustrate the present embodiment
Scheme certain components to have omission, zoom in or out, does not represent the size of actual product；To those skilled in the art,
The omitting of some known structures and their instructions in the attached drawings are understandable.Being given for example only property of position relationship described in attached drawing
Illustrate, should not be understood as the limitation to this patent.
As shown in Figure 1, a kind of sentry's drowsiness intelligent monitor system based on deep learning computer vision, wherein including
Data acquisition unit, data processing unit, data analysis unit and alarm unit, four units are sequentially connected.
Data acquisition unit includes a SD video camera, is installed in 5 meters of monitored security personnel front upper place, can be in the past
To shooting face.Data processing unit runs a deep neural network model operation, this deep neural network model is advance
Trained multitask concatenated convolutional neural network（MTCNN）, and be deployed on local data processing server.It receives real-time
VGG video images are as input, Face datection, face characteristic and the head pose of output time.Data analysis unit is deployed in
Same computing platform runs an advance trained HMM model, receives the output of data processing unit as input, is based on
Time sequence status judges whether sentry is sleepy, absent-minded, once judging that drowsiness and absent-minded probability are more than given threshold, then activates alarm
Device, onsite alarming device will send out prompt tone and wake sentry immediately, at the same positioned at the backstage warning device of duty room will regarding
Eye-catching mark in frequency picture prompts operator on duty to sentry's supervision.
Obviously, the above embodiment of the present invention be only to clearly illustrate example of the present invention, and not be pair
The restriction of embodiments of the present invention.For those of ordinary skill in the art, may be used also on the basis of the above description
To make other variations or changes in different ways.There is no necessity and possibility to exhaust all the enbodiments.It is all this
All any modification, equivalent and improvement etc., should be included in the claims in the present invention made by within the spirit and principle of invention
Protection domain within.
Claims (5)
1. a kind of sentry's drowsiness intelligent monitor system based on deep learning computer vision, which is characterized in that adopted including data
Collection unit, data processing unit, data analysis unit and alarm unit, four units are sequentially connected.
2. a kind of sentry's drowsiness intelligent monitor system based on deep learning computer vision according to claim 1,
It is characterized in that：The data acquisition unit is the clear video camera of a station symbol, mounted on security personnel front upper place, can shoot it is positive
Position.
3. a kind of sentry's drowsiness intelligent monitor system based on deep learning computer vision according to claim 1,
It is characterized in that：The data processing unit is a host or server for being equipped with high-performance GPU, is deployed with advance training
Good multitask concatenated convolutional neural network can realize Face datection, face alignment and head pose estimation simultaneously.
4. a kind of sentry's drowsiness intelligent monitor system based on deep learning computer vision according to claim 1,
It is characterized in that：The data analysis unit is deployed in same computing platform, using the output of data processing unit as input,
A trained HMM model is run, by the input in a period of time, sequentially analyzes the state of mind of sentry.
5. a kind of sentry's drowsiness intelligent monitor system based on deep learning computer vision according to claim 1,
It is characterized in that：The alarm unit is made of local warning device and backstage warning device two parts, by the same controller
Control, when sentry leaves the post, drowsiness when, local warning device can carry out auditory tone cues automatically at the scene, backstage warning device of working together
It can be displayed in red warning message in monitored picture, monitoring personnel on duty is prompted to be supervised in time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810434067.4A CN108710839A (en) | 2018-05-08 | 2018-05-08 | A kind of sentry's drowsiness intelligent monitor system based on deep learning computer vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810434067.4A CN108710839A (en) | 2018-05-08 | 2018-05-08 | A kind of sentry's drowsiness intelligent monitor system based on deep learning computer vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108710839A true CN108710839A (en) | 2018-10-26 |
Family
ID=63868833
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810434067.4A Pending CN108710839A (en) | 2018-05-08 | 2018-05-08 | A kind of sentry's drowsiness intelligent monitor system based on deep learning computer vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108710839A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113967014A (en) * | 2021-12-22 | 2022-01-25 | 成都航空职业技术学院 | Student behavior analysis device, system and method based on big data |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201682586U (en) * | 2010-05-07 | 2010-12-22 | 深圳市飞瑞斯科技有限公司 | Post electronic eye device with fatigue detection |
| CN102647581A (en) * | 2012-04-27 | 2012-08-22 | 浙江晨鹰科技有限公司 | Video monitoring device and system |
| CN102647580A (en) * | 2012-04-27 | 2012-08-22 | 浙江晨鹰科技有限公司 | Video monitoring method and system |
| CN103632485A (en) * | 2013-11-01 | 2014-03-12 | 安徽博仑特智能科技有限公司 | Guard-on-duty intelligent sleep preventing system device |
| CN104038733A (en) * | 2014-05-21 | 2014-09-10 | 天津市亚安科技股份有限公司 | Sentinel early warning monitoring system |
| CN104243917A (en) * | 2014-09-02 | 2014-12-24 | 佛山市新东方电子技术工程有限公司 | Video monitoring device alarm protecting and monitoring system and control method thereof |
| CN106650621A (en) * | 2016-11-18 | 2017-05-10 | 广东技术师范学院 | Deep learning-based emotion recognition method and system |
| CN107016370A (en) * | 2017-04-10 | 2017-08-04 | 电子科技大学 | One kind is based on the enhanced partial occlusion face identification method of data |
| CN107506707A (en) * | 2016-11-30 | 2017-12-22 | 奥瞳***科技有限公司 | Using the Face datection of the small-scale convolutional neural networks module in embedded system |
| CN107633203A (en) * | 2017-08-17 | 2018-01-26 | 平安科技（深圳）有限公司 | Facial emotions recognition methods, device and storage medium |
| CN107748858A (en) * | 2017-06-15 | 2018-03-02 | 华南理工大学 | A kind of multi-pose eye locating method based on concatenated convolutional neutral net |
| CN107967456A (en) * | 2017-11-27 | 2018-04-27 | 电子科技大学 | A kind of multiple neural network cascade identification face method based on face key point |
-
2018
- 2018-05-08 CN CN201810434067.4A patent/CN108710839A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201682586U (en) * | 2010-05-07 | 2010-12-22 | 深圳市飞瑞斯科技有限公司 | Post electronic eye device with fatigue detection |
| CN102647581A (en) * | 2012-04-27 | 2012-08-22 | 浙江晨鹰科技有限公司 | Video monitoring device and system |
| CN102647580A (en) * | 2012-04-27 | 2012-08-22 | 浙江晨鹰科技有限公司 | Video monitoring method and system |
| CN103632485A (en) * | 2013-11-01 | 2014-03-12 | 安徽博仑特智能科技有限公司 | Guard-on-duty intelligent sleep preventing system device |
| CN104038733A (en) * | 2014-05-21 | 2014-09-10 | 天津市亚安科技股份有限公司 | Sentinel early warning monitoring system |
| CN104243917A (en) * | 2014-09-02 | 2014-12-24 | 佛山市新东方电子技术工程有限公司 | Video monitoring device alarm protecting and monitoring system and control method thereof |
| CN106650621A (en) * | 2016-11-18 | 2017-05-10 | 广东技术师范学院 | Deep learning-based emotion recognition method and system |
| CN107506707A (en) * | 2016-11-30 | 2017-12-22 | 奥瞳***科技有限公司 | Using the Face datection of the small-scale convolutional neural networks module in embedded system |
| CN107016370A (en) * | 2017-04-10 | 2017-08-04 | 电子科技大学 | One kind is based on the enhanced partial occlusion face identification method of data |
| CN107748858A (en) * | 2017-06-15 | 2018-03-02 | 华南理工大学 | A kind of multi-pose eye locating method based on concatenated convolutional neutral net |
| CN107633203A (en) * | 2017-08-17 | 2018-01-26 | 平安科技（深圳）有限公司 | Facial emotions recognition methods, device and storage medium |
| CN107967456A (en) * | 2017-11-27 | 2018-04-27 | 电子科技大学 | A kind of multiple neural network cascade identification face method based on face key point |
Non-Patent Citations (2)
| Title |
|---|
| 王冬雪 等: "基于CNN 特征与HMM 时序建模的人体行为识别研究", 《中国科技论文在线》 * |
| 陈龙 等: "利用综合光流直方图的人群异常行为检测", 《计算机工程》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113967014A (en) * | 2021-12-22 | 2022-01-25 | 成都航空职业技术学院 | Student behavior analysis device, system and method based on big data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20240265705A1 (en) | Face concealment detection | |
| CN105913559B (en) | A kind of ATM in bank intelligent control method based on body-sensing technology | |
| CN104637246B (en) | Driver multi-behavior early warning system and danger evaluation method | |
| CN110044486A (en) | Method, apparatus, the equipment of repetition of alarms are avoided for human body inspection and quarantine system | |
| CN113223046B (en) | Method and system for identifying prisoner behaviors | |
| CN109614906A (en) | A kind of security system and security alarm method based on deep learning | |
| Ying et al. | The monitoring method of driver's fatigue based on neural network | |
| CN109492548A (en) | The preparation method of region mask picture based on video analysis | |
| CN204926116U (en) | Picture system is declared in safety inspection containing video analysis | |
| CN110309760A (en) | The method that the driving behavior of driver is detected | |
| Devi et al. | Fuzzy based driver fatigue detection | |
| CN103632485A (en) | Guard-on-duty intelligent sleep preventing system device | |
| Valsan et al. | Monitoring driver’s drowsiness status at night based on computer vision | |
| CN108710839A (en) | A kind of sentry's drowsiness intelligent monitor system based on deep learning computer vision | |
| CN106448055A (en) | Method and device for monitoring and alarming | |
| Reddy et al. | Soft Computing Techniques for Driver Alertness | |
| TW201140505A (en) | Surveillance video fire detecting and extinguishing system | |
| Liu et al. | Design and implementation of multimodal fatigue detection system combining eye and yawn information | |
| CN110148234A (en) | Campus brush face picks exchange method, storage medium and system | |
| CN113989789A (en) | Face fatigue detection method based on multi-feature fusion in teaching scene | |
| TWI657379B (en) | Intelligent deep learning drowning rescue system | |
| Chaudhary et al. | An improved real time driver drowsiness detection system | |
| Gupta et al. | Real time driver drowsiness detecion using transfer learning | |
| CN111209814A (en) | Face recognition campus area early warning method and system for K12 education stage | |
| Kodikara et al. | Surveillance based Child Kidnap Detection and Prevention Assistance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181026 |
|
| RJ01 | Rejection of invention patent application after publication |