CN108647616A - Real-time drowsiness detection method based on facial characteristics - Google Patents
Real-time drowsiness detection method based on facial characteristics Download PDFInfo
- Publication number
- CN108647616A CN108647616A CN201810407011.XA CN201810407011A CN108647616A CN 108647616 A CN108647616 A CN 108647616A CN 201810407011 A CN201810407011 A CN 201810407011A CN 108647616 A CN108647616 A CN 108647616A
- Authority
- CN
- China
- Prior art keywords
- eyes
- detection
- time
- state
- face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Human Computer Interaction (AREA)
- Probability & Statistics with Applications (AREA)
- Geometry (AREA)
- Image Processing (AREA)
- Emergency Alarm Devices (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The real-time drowsiness detection method based on facial characteristics that the invention discloses a kind of, including Face datection, close one's eyes detection and detection of yawning；Face is detected using Viola Jones, it is by segmentation that skin area is independent so that brightness is constant；Detection of closing one's eyes uses sobel edge detections, and detection of yawning is partitioned into face region using k means clusters；The feature that above-mentioned each stage obtains is attached, sleepiness state or normal condition are then obtained by a binary linear support vector machine classifier characteristic of division vector.The present invention without any sensor or wearable device, and can be worked well in various lighting conditions by monitoring that the state of eyes and face realizes drowsiness detection.
Description
Technical field
The present invention relates to drowsiness detection technologies, and in particular to a kind of real-time drowsiness detection method based on facial characteristics.
Background technology
In the past ten years, drowsiness detection has been widely studied.At present more commonly used drowsiness detection be HOG and
SVM methods, used data are marked manually based on people.Compared with manual mark, intelligentized drowsiness detection will be more
It is accurate to add, because the decision of people is easier to make mistakes than intelligence.In addition, also by smartwatch and including sensor identification
Headband is come the method that detects sleepiness, but this method is limited by driver's selection, and driver may be unwilling to wear headband.
Supervised learning method needs highly reliable brass tacks.The eyes defined using the mankind in research before
Feature, since calculating eyelid movement.This means that this method is confined to the information of eyelid movement offer.Therefore, only extraction and
It may result in using those artificial features and lose some significant information, sleepy state may occur not to be detected but.Cause
This, can only be identified tired using more complicated algorithm.For example the human facial expression recognition based on deep learning is used, still,
The disadvantages of this method is that a large amount of data is needed to train neural network.
Invention content
The real-time drowsiness detection method based on facial characteristics that the purpose of the present invention is to provide a kind of, by monitor eyes and
The state of face realizes drowsiness detection.
Realize that the technical solution of the object of the invention is：A kind of real-time drowsiness detection method based on facial characteristics, including with
Lower step：
Step 1, Face datection and partitioning into skin
Facial feature detection is carried out using Viola Jones, after detecting face, by converting the image into the domains YCbCr
To execute partitioning into skin；
Step 2, it closes one's eyes and detects
Two eyes are identified using edge detection, and determine the center of eyes according to the symmetry of single eyes, finally
Determine pupil；If eyes are opened, step 3 is executed；If eyes closed, and be more than the first threshold time, then it is regarded as sleeping
Alarm is arranged during the state in meaning state, if the eyes closed time is less than the first threshold time, thens follow the steps 3；
Step 3, it yawns detection
Mouth region is detected using Viola Jones, and mouth region is split using k-means clusters, is passed through
Division to image pixel in region, judges whether face opens, and using related coefficient template matches into line trace, if mouth
Bar ETAD expected time of arrival and departure is more than the second threshold time, then is considered as the state of yawning；
Step 4, drowsiness detection
Classified using the binary SVM classifier of linear kernel, if detection eye is opened, is not detected and yawns, depending on
For normal condition；If detecting eye to open, and detect and yawn, is then considered as sleepiness state；If detecting eye to be closed
More than the first threshold time, then it is considered as sleepiness state；Alarm is opened in the case where being finally detected as sleepiness state.
Further, in step 2, the first threshold time is 3 seconds.
Further, in step 2, two eyes are identified using sobel edge detections.
Further, in step 3, the second threshold time is 3 seconds.
Compared with prior art, remarkable advantage of the invention is：The present invention is by monitoring that the state of eyes and face is realized
Drowsiness detection without any sensor or wearable device, and can work well in various lighting conditions.
Description of the drawings
Fig. 1 is the real-time drowsiness detection method flow diagram based on facial characteristics.
Specific implementation mode
In conjunction with Fig. 1, a kind of real-time drowsiness detection method based on facial characteristics includes the following steps：
Step 1, Face datection and partitioning into skin
Face datection uses the facial feature detection of Viola Jones, and the purpose is to reduce flase drop to the greatest extent to identify face
Portion's expression.Once detecting face, partitioning into skin is executed by converting the image into the domains YCbCr.Convert the image into YCbCr
The sharpest edges in domain are, only consider that chromatic component can eliminate the influence of brightness.In the domains RGB, each component of image is (i.e.
Red, green and blue) there is different brightness.But in the domains YCbCr, all luminance informations are all provided by Y-component, because
It is totally independent of brightness for Cb (blue) and Cr (red) component.It converts for RGB image to be divided into Y, Cb, Cr components in domain.
Step 2, detection of closing one's eyes
The most important factor that help detects driver fatigue is the state of eyes, that is, is opened or closed.Under sleepy state,
Eyelid flesh subconsciously attracts, and accelerates into sleep procedure.Then two eyes are separated using edge detection, and according to eyes
Symmetry determine the centers of eyes, finally determine pupil.If eyes are opened, it is considered as normal condition.If eyes closed,
And be more than the first threshold time, then it is regarded as sleepiness state, alarm is set during the state.
Edge detection can be considered as positioning the process of image pixel intensities conversion.The present invention uses Sobel edge detection methods,
One filter convolved image of Sobel detectors, both horizontally and vertically on separable and integer valued function.Therefore it is
A kind of computational methods of relatively economical.Secondly, the gradient being generated by it is approximately relative coarseness, is best suited for high frequency change
Change, such as the blink during fatigue.Determine the state of eyes in each frame using related coefficient template matching method.By filling
Divide the variation for considering connection pixel and obtains the region of eyes with the likelihood ratio of eye pixels.Sobel edge detection methods are also used
In the accurate and accurate boundary of detection eyes.The technology finds eyes, therefore can be with separate detection since left and right side
Eyes.The eyes detected are split from image and for generating eye template, in this way, use can be obtained
In the eye template of the quite stable of state analysis, and also reduce the influence of light reflection.In order to distinguish fatigue, it should accurately
Identify the state of eyes.There has been described two factors for influencing eyes size.The eyes of people are not of uniform size.Secondly, each frame
The distance between middle driver and video camera are changing.Therefore, selection pre-fixes the template of eyes under 12 × 30 size,
Then feature extraction is carried out.For the template of each eyes, region, average height and the ratio of width to height are the key that determine eye state
Feature.Eye state can be divided into three classes：It opens completely, partly open and be closed completely.Different eyes when opening and partly opening completely
Eyeball state is not distinguished well within the most of the time, may lead to more false alarms, meanwhile, the fortune of driver head
It is dynamic to cause to track failure to the eyes of driver.
Step 3, the detection yawned
Another significantly shows that the fatigue driving sign of a people on the face is to yawn, it is people's fatigue and will enter
The body reflection slept.Mouth region is detected using Viola Jones, and mouth region is split using k-means clusters,
By the division to image pixel in region, and suitable threshold value is defined, judge whether face opens, and utilizes phase relation digital-to-analogue
Plate is matched into line trace, is more than 3 seconds if face opens, is considered as and yawns.K means to divide the object into the k of mutual exclusion group
Class, the object in cluster each in this way is closest each other, and the object in other clusters is farthest.Each cluster is characterized in
Its barycenter or central point.Function executes k-means clusters, by using iterative algorithm, object is distributed to each cluster, is made each
Object is less than the object at a distance from other cluster barycenter at a distance from the cluster barycenter where it.The classification of image pixel is
Based on intensity of brightness.The present invention shows position and the opening state of face using k=2 in image segmentation field.
Step 4, drowsiness detection
Classified using the binary SVM classifier of linear kernel, if detection eye is opened, is not detected and yawns, depending on
For normal condition；If detecting eye to be closed, and detect and yawn, is then considered as sleepiness state；If detecting eye to be closed,
It is not detected and yawns, be then considered as sleepiness state；If detecting eye to open, and detect and yawn, is then considered as sleepiness shape
State.Alarm is opened in the case where being finally detected as sleepiness state.
Claims (4)
1. a kind of real-time drowsiness detection method based on facial characteristics, which is characterized in that include the following steps：
Step 1, Face datection and partitioning into skin
Facial feature detection is carried out using Viola Jones, after detecting face, is held by converting the image into the domains YCbCr
Row partitioning into skin；
Step 2, it closes one's eyes and detects
Two eyes are identified using edge detection, and determine the center of eyes according to the symmetry of single eyes, are finally determined
Pupil；If eyes are opened, step 3 is executed；If eyes closed, and be more than the first threshold time, then it is considered as sleepiness state,
Alarm is set during the state, if the eyes closed time is less than the first threshold time, thens follow the steps 3；
Step 3, it yawns detection
Mouth region is detected using Viola Jones, mouth region is split using k-means clusters, by area
The division of image pixel in domain, judges whether face opens, and using related coefficient template matches into line trace, if face
ETAD expected time of arrival and departure is more than the second threshold time, then is considered as the state of yawning；
Step 4, drowsiness detection
Classified using the binary SVM classifier of linear kernel, if detection eye is opened, is not detected and yawns, be considered as just
Normal state；If detecting eye to open, and detect and yawn, is then considered as sleepiness state；If detect eye closure of more than
The first threshold time is then considered as sleepiness state；Alarm is opened in the case where being finally detected as sleepiness state.
2. the real-time drowsiness detection method according to claim 1 based on facial characteristics, which is characterized in that step 2 is closed one's eyes
In detection, the first threshold time is 3 seconds.
3. the real-time drowsiness detection method according to claim 1 based on facial characteristics, which is characterized in that in step 2, make
Two eyes are identified with sobel edge detections.
4. the real-time drowsiness detection method according to claim 1 based on facial characteristics, which is characterized in that step 3 beats Kazakhstan
It owes in detection, the second threshold time is 3 seconds.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810407011.XA CN108647616A (en) | 2018-05-01 | 2018-05-01 | Real-time drowsiness detection method based on facial characteristics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810407011.XA CN108647616A (en) | 2018-05-01 | 2018-05-01 | Real-time drowsiness detection method based on facial characteristics |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108647616A true CN108647616A (en) | 2018-10-12 |
Family
ID=63748459
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810407011.XA Pending CN108647616A (en) | 2018-05-01 | 2018-05-01 | Real-time drowsiness detection method based on facial characteristics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108647616A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104688251A (en) * | 2015-03-02 | 2015-06-10 | 西安邦威电子科技有限公司 | Method for detecting fatigue driving and driving in abnormal posture under multiple postures |
| CN104809445A (en) * | 2015-05-07 | 2015-07-29 | 吉林大学 | Fatigue driving detection method based on eye and mouth states |
| CN104992148A (en) * | 2015-06-18 | 2015-10-21 | 江南大学 | ATM terminal human face key points partially shielding detection method based on random forest |
| CN106372621A (en) * | 2016-09-30 | 2017-02-01 | 防城港市港口区高创信息技术有限公司 | Face recognition-based fatigue driving detection method |
| CN107220595A (en) * | 2017-05-03 | 2017-09-29 | 北京航空航天大学 | Fatigue monitoring system based on " face characteristic extraction " |
| CN107303907A (en) * | 2016-04-21 | 2017-10-31 | 现代自动车株式会社 | For the apparatus and method for the sleepiness for determining driver |
-
2018
- 2018-05-01 CN CN201810407011.XA patent/CN108647616A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104688251A (en) * | 2015-03-02 | 2015-06-10 | 西安邦威电子科技有限公司 | Method for detecting fatigue driving and driving in abnormal posture under multiple postures |
| CN104809445A (en) * | 2015-05-07 | 2015-07-29 | 吉林大学 | Fatigue driving detection method based on eye and mouth states |
| CN104992148A (en) * | 2015-06-18 | 2015-10-21 | 江南大学 | ATM terminal human face key points partially shielding detection method based on random forest |
| CN107303907A (en) * | 2016-04-21 | 2017-10-31 | 现代自动车株式会社 | For the apparatus and method for the sleepiness for determining driver |
| CN106372621A (en) * | 2016-09-30 | 2017-02-01 | 防城港市港口区高创信息技术有限公司 | Face recognition-based fatigue driving detection method |
| CN107220595A (en) * | 2017-05-03 | 2017-09-29 | 北京航空航天大学 | Fatigue monitoring system based on " face characteristic extraction " |
Non-Patent Citations (2)
| Title |
|---|
| B. N. MANU ET AL: "Facial Features Monitoring for Real Time Drowsiness Detection", 《2016 12TH INTERNATIONAL CONFERENCE ON INNOVATIONS IN INFORMATION TECHNOLOGY (IIT)》 * |
| PING WANG ET AL: "A method of detecting driver drowsiness state based on multi-features of face", 《2012 5TH INTERNATIONAL CONGRESS ON IMAGE AND SIGNAL PROCESSING》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Manu | Facial features monitoring for real time drowsiness detection | |
| CN107491740B (en) | Newborn pain recognition method based on facial expression analysis | |
| CN108053615B (en) | Method for detecting fatigue driving state of driver based on micro-expression | |
| US7620216B2 (en) | Method of tracking a human eye in a video image | |
| Asteriadis et al. | Facial feature detection using distance vector fields | |
| CN106682578B (en) | Weak light face recognition method based on blink detection | |
| CN109977930A (en) | Method for detecting fatigue driving and device | |
| Monwar et al. | Pain recognition using artificial neural network | |
| Singh et al. | Lie detection using image processing | |
| CN106203338B (en) | Human eye state method for quickly identifying based on net region segmentation and threshold adaptive | |
| Farooq et al. | Improved automatic localization of optic disc in Retinal Fundus using image enhancement techniques and SVM | |
| CN110598574A (en) | Intelligent face monitoring and identifying method and system | |
| D'orazio et al. | A neural system for eye detection in a driver vigilance application | |
| Rajevenceltha et al. | A novel approach for drowsiness detection using local binary patterns and histogram of gradients | |
| Ribarić et al. | A neural-network-based system for monitoring driver fatigue | |
| Alshaqaqi et al. | Vision based system for driver drowsiness detection | |
| Campadelli et al. | Localization of facial features and fiducial points | |
| CN108647616A (en) | Real-time drowsiness detection method based on facial characteristics | |
| AU2021103045A4 (en) | Drowsy detection using image processing | |
| Subasic et al. | Expert system segmentation of face images | |
| Hernández-Travieso et al. | Expression detector system based on facial images | |
| KR102399121B1 (en) | Early screening system and method for high-risk groups of suicide using artificial intelligence (AI)-based video and audio information | |
| Sharma et al. | Deep convolutional network based real time fatigue detection and drowsiness alertness system | |
| Allach et al. | Detection of driver's alertness level based on the Viola and Jones method and logistic regression analysis | |
| Sahare | Driver drowsiness detection system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181012 |