CN111695510A - Image-based fatigue detection method for computer operator - Google Patents
Image-based fatigue detection method for computer operator Download PDFInfo
- Publication number
- CN111695510A CN111695510A CN202010536537.5A CN202010536537A CN111695510A CN 111695510 A CN111695510 A CN 111695510A CN 202010536537 A CN202010536537 A CN 202010536537A CN 111695510 A CN111695510 A CN 111695510A
- Authority
- CN
- China
- Prior art keywords
- computer operator
- computer
- fatigue
- fatigue state
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Image Analysis (AREA)
Abstract
A method for image-based fatigue detection of a computer operator, comprising: step 1: taking pictures to construct a data set of 'computer operators', and classifying according to fatigue states; step 2: determining characteristics corresponding to pictures of different fatigue states of a computer operator; and step 3: judging the fatigue state of a new picture given in the test set according to the type characteristics extracted in the training process in the step 2; and 4, step 4: adjusting the weights and step sizes corresponding to different elements of the CNN model according to the accuracy of the test result in the step 3, and testing again until the accuracy of the test result meets the condition; and 5: deploying the CNN model after parameter adjustment to a computer system with a camera for operation; step 6: and acquiring the type of the fatigue state of the computer operator in real time through the camera and outputting the result. The invention acquires the head action and the facial features of the computer operator in real time and judges the fatigue state of the computer operator.
Description
Technical Field
The invention relates to a method for detecting fatigue of a computer operator, which is suitable for the situation that the state of the computer operator is unknown but the head action and the face characteristic can be clearly obtained by a camera.
Background
The camera can shoot images and convert the images into a format which can be processed by a computer; the image processing technology can analyze the obtained picture of the person to obtain the head action of the computer operator; the pattern recognition technology can process and interpret data through a computer to realize classification for different states; the data set is constructed according to the fatigue state of the computer operator, and effective training data can be provided for the model to improve the accuracy of the detection result.
Although the fatigue state of a computer operator is determined to be accurate by acquiring physiological signals, the existing methods such as physiological signal acquisition and the like used in the fatigue detection technology need to wear various instruments which are generally high in price and heavy and are not beneficial to popularization; the acquisition of some signals can also influence the normal operation of an operator, for example, the acquisition of eye movement signals can influence the wearing of myopia glasses by the operator, and certain errors can also occur due to the difference between the sizes and the positions of the eyes of each person; the currently disclosed universal data set has an overfitting phenomenon aiming at fatigue identification of a computer operator in a specific scene, and the effect is not good.
Disclosure of Invention
The present invention is directed to overcoming the above-mentioned drawbacks of the prior art and providing a method for detecting fatigue of a computer operator based on an image.
The invention can detect the attention focusing condition of the computer operator under the condition that the fatigue state of the computer operator is unknown but the head action and the facial features of the computer operator can be obtained through the camera, provides a detection method based on an image processing technology and a mode recognition technology, and provides a thought for the detection method of the fatigue state of the computer operator.
The technical scheme adopted by the invention for solving the technical problems is as follows:
a fatigue detection method for a computer operator based on images comprises the following steps:
step 1: aiming at a scene needing to detect the fatigue state of a computer operator, taking pictures to construct a data set of the computer operator, and classifying the pictures according to the type of the fatigue state;
step 2: training data of a training set part in a self-built 'computer operator' data set by using a CNN model Resnet50 model, and determining characteristics corresponding to pictures of different fatigue state types of a computer operator according to head actions of the computer operator in each picture and distance relations between pixel points of each facial characteristic by using the model;
and step 3: testing data of a training set part in a self-built 'computer operator' data set by using a CNN model Resnet50 model, and judging the fatigue state of a computer operator in a new picture given in a test set according to the characteristics of different fatigue state types of the computer operator in the picture extracted in the training process in the step 2 by using the model;
and 4, step 4: adjusting the weights and step sizes corresponding to different elements in the CNN model Resnet50 model according to the accuracy sum of the test result in the step 3, and testing again until the accuracy of the test result meets the condition;
and 5: deploying the CNN model after parameter adjustment to a computer system with a camera for operation;
step 6: and acquiring the type of the fatigue state of the computer operator in real time through the camera and outputting the result.
Preferably, the characteristics corresponding to the pictures of different fatigue states of the computer operator in the step 1 include: yawning, eye closure and drowsiness.
Preferably, the number of the photos taken in step 1 is more than 2000.
The invention provides a method for detecting the fatigue state of a computer operator under the condition that the fatigue state is unknown but the head action and the face characteristics of the computer operator can be obtained through a camera, provides a detection method based on an image processing technology and a mode recognition technology, and provides an idea for the detection method of the fatigue state of the operator. The camera can shoot images and convert the images into a format which can be processed by a computer; the image processing technology can analyze the obtained picture of the person to obtain the head action and the face characteristic condition of the operator; the pattern recognition technology is to process and interpret data through a computer to realize classification for different states; the fatigue state data set of the computer operator is constructed aiming at the fatigue state of the computer operator, so that effective training data can be provided for the model, and the accuracy of the detection result is improved.
Compared with the prior art, the technical scheme of the invention has the advantages that:
(1) the fatigue state of the computer operator in a specific occasion can be effectively detected by pertinently constructing a data set of the computer operator;
(2) the computer camera on the calling equipment detects the computer operator, so that the cost is lower, the practicability is higher, and the popularization is facilitated.
Drawings
FIG. 1: flow chart of the method of the invention.
Detailed Description
In order to facilitate the understanding and implementation of the present invention for those of ordinary skill in the art, the present invention is further described in detail below with reference to the accompanying drawings and examples.
A fatigue detection method for a computer operator based on images comprises the following steps:
step 1: aiming at a scene needing to detect the fatigue state of a computer operator, more than 2000 pictures are taken to construct a data set of the computer operator, the pictures are classified for each fatigue state, and the action difference of each type of picture is larger;
step 2: training data of a training set part in a self-built 'computer operator' data set by using a CNN model Resnet50 model, wherein the model can determine corresponding characteristics of the computer operator in different condition pictures according to the relations of head action of the computer operator in each picture, the distance between each pixel point of the face characteristic and the like;
and step 3: testing the data of a training set part in a self-built 'computer operator' data set by using the Resnet50 model, and judging the fatigue state of a computer operator in a new picture given in a test set according to the characteristics of different fatigue states of the computer operator in the picture extracted in the training process in the step 2 by using the model;
and 4, step 4: adjusting the weights and step sizes corresponding to different elements in the Resnet50 model according to the accuracy sum of the test result in the step 3, and testing again until the accuracy of the test result meets the condition;
and 5: and acquiring a fatigue state picture of a computer operator in real time through a camera of the equipment and outputting a result.
The CNN model and Resnet50 model referred to herein are described in Kaiming He, Xiangyu Zhuang, Shaoqingren, Jiann Sun et al, Deep research Learning for image recognition, which is published in CVPR 2016.
According to the invention, through arranging a deep learning environment and calling a computer camera, the head action and the face characteristic of a computer operator are obtained in real time, and the fatigue state of the computer operator is judged.
The embodiments described in this specification are merely illustrative of implementations of the inventive concept and the scope of the present invention should not be considered limited to the specific forms set forth in the embodiments but rather by the equivalents thereof as may occur to those skilled in the art upon consideration of the present inventive concept.
Claims (3)
1. A fatigue detection method for a computer operator based on images comprises the following steps:
step 1: aiming at a scene needing to detect the fatigue state of a computer operator, taking pictures to construct a data set of the computer operator, and classifying the pictures according to the type of the fatigue state;
step 2: training data of a training set part in a self-built 'computer operator' data set by using a CNN model Resnet50 model, and determining characteristics corresponding to pictures of different fatigue state types of a computer operator according to head actions of the computer operator in each picture and distance relations between pixel points of each facial characteristic by using the model;
and step 3: testing data of a training set part in a self-built 'computer operator' data set by using a CNN model Resnet50 model, and judging the fatigue state of a computer operator in a new picture given in a test set according to the characteristics of different fatigue state types of the computer operator in the picture extracted in the training process in the step 2 by using the model;
and 4, step 4: adjusting the weights and step sizes corresponding to different elements in the CNN model Resnet50 model according to the accuracy sum of the test result in the step 3, and testing again until the accuracy of the test result meets the condition;
and 5: deploying the CNN model after parameter adjustment to a computer system with a camera for operation;
step 6: and acquiring the type of the fatigue state of the computer operator in real time through the camera and outputting the result.
2. The method of claim 1, wherein the fatigue detection is performed by a computer operator based on an image of the subject, the method comprising: the characteristics corresponding to the pictures of different situations of the fatigue state of the computer operator comprise: yawning, eye closure and drowsiness.
3. The method of claim 1, wherein the fatigue detection is performed by a computer operator based on an image of the subject, the method comprising: the number of the taken photos in the step 1 is more than 2000.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010536537.5A CN111695510A (en) | 2020-06-12 | 2020-06-12 | Image-based fatigue detection method for computer operator |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010536537.5A CN111695510A (en) | 2020-06-12 | 2020-06-12 | Image-based fatigue detection method for computer operator |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111695510A true CN111695510A (en) | 2020-09-22 |
Family
ID=72480779
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010536537.5A Withdrawn CN111695510A (en) | 2020-06-12 | 2020-06-12 | Image-based fatigue detection method for computer operator |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111695510A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102096810A (en) * | 2011-01-26 | 2011-06-15 | 北京中星微电子有限公司 | Method and device for detecting fatigue state of user before computer |
| CN110020632A (en) * | 2019-04-12 | 2019-07-16 | 李守斌 | A method of the recognition of face based on deep learning is for detecting fatigue driving |
| CN110096957A (en) * | 2019-03-27 | 2019-08-06 | 苏州清研微视电子科技有限公司 | The fatigue driving monitoring method and system merged based on face recognition and Activity recognition |
| CN110334614A (en) * | 2019-06-19 | 2019-10-15 | 腾讯科技（深圳）有限公司 | A kind of fatigue state method for early warning, device, equipment and storage medium |
| CN111079476A (en) * | 2018-10-19 | 2020-04-28 | 上海商汤智能科技有限公司 | Driving state analysis method and device, driver monitoring system and vehicle |
-
2020
- 2020-06-12 CN CN202010536537.5A patent/CN111695510A/en not_active Withdrawn
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102096810A (en) * | 2011-01-26 | 2011-06-15 | 北京中星微电子有限公司 | Method and device for detecting fatigue state of user before computer |
| CN111079476A (en) * | 2018-10-19 | 2020-04-28 | 上海商汤智能科技有限公司 | Driving state analysis method and device, driver monitoring system and vehicle |
| CN110096957A (en) * | 2019-03-27 | 2019-08-06 | 苏州清研微视电子科技有限公司 | The fatigue driving monitoring method and system merged based on face recognition and Activity recognition |
| CN110020632A (en) * | 2019-04-12 | 2019-07-16 | 李守斌 | A method of the recognition of face based on deep learning is for detecting fatigue driving |
| CN110334614A (en) * | 2019-06-19 | 2019-10-15 | 腾讯科技（深圳）有限公司 | A kind of fatigue state method for early warning, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112380952B (en) | Power equipment infrared image real-time detection and identification method based on artificial intelligence | |
| US8314854B2 (en) | Apparatus and method for image recognition of facial areas in photographic images from a digital camera | |
| KR20170006355A (en) | Method of motion vector and feature vector based fake face detection and apparatus for the same | |
| CN103605969A (en) | Method and device for face inputting | |
| US20070116364A1 (en) | Apparatus and method for feature recognition | |
| KR101449744B1 (en) | Face detection device and method using region-based feature | |
| CN109190456B (en) | Multi-feature fusion overlook pedestrian detection method based on aggregated channel features and gray level co-occurrence matrix | |
| CN111401188B (en) | Traffic police gesture recognition method based on human body key point characteristics | |
| CN108960142B (en) | Pedestrian re-identification method based on global feature loss function | |
| WO2020195732A1 (en) | Image processing device, image processing method, and recording medium in which program is stored | |
| Dey et al. | Automated glaucoma detection from fundus images of eye using statistical feature extraction methods and support vector machine classification | |
| Ashwinkumar et al. | Deep learning based approach for facilitating online proctoring using transfer learning | |
| KR101343623B1 (en) | adaptive color detection method, face detection method and apparatus | |
| Basthomi et al. | Implementation of RFID attendance system with face detection using validation viola-jones and local binary pattern histogram method | |
| Arafah et al. | Face recognition system using Viola Jones, histograms of oriented gradients and multi-class support vector machine | |
| CN104615985B (en) | A kind of recognition methods of human face similarity degree | |
| CN107798282B (en) | Method and device for detecting human face of living body | |
| CN111695510A (en) | Image-based fatigue detection method for computer operator | |
| CN112907206B (en) | Business auditing method, device and equipment based on video object identification | |
| KR100472953B1 (en) | Face region detecting method using support vector machine | |
| CN111191519B (en) | Living body detection method for user access of mobile power supply device | |
| CN112418085A (en) | Facial expression recognition method under partial shielding working condition | |
| JP2021072106A (en) | Image processing system | |
| CN111126283A (en) | Rapid in-vivo detection method and system for automatically filtering fuzzy human face | |
| CN110059613A (en) | A kind of separation of video image pyrotechnics and detection method based on rarefaction representation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200922 |