CN114627450A - Fatigue state detection device and fatigue detection method - Google Patents

Fatigue state detection device and fatigue detection method Download PDF

Info

Publication number
CN114627450A
CN114627450A CN202011499151.8A CN202011499151A CN114627450A CN 114627450 A CN114627450 A CN 114627450A CN 202011499151 A CN202011499151 A CN 202011499151A CN 114627450 A CN114627450 A CN 114627450A
Authority
CN
China
Prior art keywords
threshold value
fatigue state
detection
driver
set threshold
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011499151.8A
Other languages
Chinese (zh)
Inventor
陈磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Bata Technology Co ltd
Original Assignee
Beijing Bata Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Bata Technology Co ltd filed Critical Beijing Bata Technology Co ltd
Priority to CN202011499151.8A priority Critical patent/CN114627450A/en
Publication of CN114627450A publication Critical patent/CN114627450A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/02Alarms for ensuring the safety of persons
    • G08B21/06Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B31/00Predictive alarm systems characterised by extrapolation or other computation using updated historic data

Landscapes

  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

A fatigue state detection device according to the present invention is a fatigue state detection device for detecting a fatigue state of a driver, and includes: an image acquisition unit that acquires a detection image; an eye opening degree calculation unit which calculates an eye opening degree in each of the plurality of detection images; a first threshold value acquisition unit that acquires a first threshold value for determining whether or not an eye is open; a first judging section for comparing the degree of opening of the eye in each of the detection images with a first threshold value to judge whether the eye is open; a counting operation part for counting the number of the detection images judged as the eyes are open and calculating the ratio of the number to the total number of the detection images; the fatigue state determination unit compares the ratio with a first set threshold value, determines that the driver is in a heavy fatigue state when the ratio is equal to or less than the first set threshold value, and determines that the driver is in a light fatigue state when the ratio is greater than the first set threshold value and equal to or less than a second set threshold value that is greater than the first set threshold value.

Description

Fatigue state detection device and fatigue detection method
Technical Field
The present invention relates to a fatigue detection device and a fatigue detection method, and more particularly to a fatigue detection system for early detecting, for example, a fatigue state of an operator by using an image processing technique and a pattern recognition technique.
Background
According to the data of traffic management department, fatigue driving has become the main factor of all traffic accidents, especially the occurrence of major traffic accidents. Aiming at fatigue driving, various detection and prevention devices for fatigue driving appear in the market, but the detection means mainly rely on identifying whether the driver has long-time eye closing action, and cannot provide effective early warning in an early fatigue state that eyes are gradually closed, so that sufficient reaction time cannot be provided for the driver, and traffic accidents are really and effectively avoided; meanwhile, due to individual difference and the influence of external light, the opening and closing degree of the upper eyelid and the lower eyelid is unstable, and the conventional detection method is very easy to generate false alarm, so that the attention of a driver is influenced, the psychological burden of the driver is increased, and further fatigue is easy to generate. Therefore, the existing fatigue detection method has the problems of serious delay of detection time and poor detection precision, and cannot achieve the effect of effectively preventing driving fatigue.
Therefore, in order to solve the above problems, it is necessary to design a more precise fatigue state detecting device and a fatigue detecting method to make up for the above disadvantages.
Disclosure of Invention
In view of the above situation, the present invention provides a fatigue state detection device and a fatigue detection method, which can provide early warning in a fatigue state in which eyes are gradually closed, so as to provide sufficient response time for a driver, and truly and effectively avoid traffic accidents; meanwhile, false alarms which are easily generated by the conventional detection method can be reduced, the influence on the attention of a driver can be avoided, the psychological burden of the driver is avoided from being increased, further fatigue is avoided from being generated, the problems of serious delay of detection time and poor detection precision existing in the conventional fatigue detection method can be avoided, and the driving fatigue can be effectively prevented.
A fatigue state detection device according to the present invention is a fatigue state detection device for detecting a fatigue state of a driver, the fatigue state detection device including:
an image acquisition unit that acquires a plurality of static detection images or a plurality of dynamic detection images of a predetermined time period in a region near a driver as detection images;
an eye opening degree calculating part which calculates the eye opening degree in each detection image aiming at the plurality of detection images acquired by the image acquiring part;
a first threshold value acquisition unit that acquires a first threshold value for determining whether or not an eye is open;
a first judging section that compares the degree of eye openness in each of the detection images calculated by the eye openness calculating section with a first threshold value to judge whether or not the eye in each of the detection images is open;
a counting operation part for counting the number of the detection images judged as the eyes are opened by the first judgment part and calculating the ratio of the number to the total number of the detection images;
and a fatigue state determination unit that compares the ratio calculated by the count calculation unit with a first set threshold value, determines that the driver is in a heavy fatigue state when the ratio is equal to or less than the first set threshold value, and determines that the driver is in a light fatigue state when the ratio is greater than the first set threshold value and equal to or less than a second set threshold value that is greater than the first set threshold value.
Further, the fatigue state detecting apparatus of the present invention, wherein,
the first threshold acquisition unit includes:
a target image acquisition unit that acquires a plurality of static target images or a moving target image of a predetermined time period in a region near a driver as target images;
an image processing unit that detects the iris boundary and the positions of the upper and lower eyelids in each target image, and calculates the ratio of the area of the region surrounded by the positions of the upper and lower eyelids and the iris boundary to the entire iris area as the degree of eye openness in the target image;
and a calculation unit that calculates an average value of the degrees of opening and closing of the plurality of eyes obtained by the image processing unit as a first threshold value.
Further, the fatigue state detecting apparatus of the present invention, wherein,
the first threshold value obtaining unit obtains, as the first threshold value, an average value of the plurality of eye opening degrees calculated by the eye opening degree calculating unit or a predetermined value set in advance.
Further, the fatigue state detecting device of the present invention, wherein,
the first set threshold is 0.2 and the second set threshold is 0.4.
A fatigue state detection method according to the present invention is a method for detecting a fatigue state of a driver, the method including:
a detection image acquisition step of acquiring a plurality of static detection images or dynamic detection images of a specified duration of time in an area near a driver as detection images;
an eye opening degree calculation step of calculating an eye opening degree in each detection image for the plurality of detection images acquired in the image acquisition step;
a first threshold value obtaining step of obtaining a first threshold value for judging whether the eye is open;
a first judgment step of comparing the degree of opening of the eyes in each detection image calculated by the eye opening/closing degree calculation step with a first threshold value to judge whether the eyes in each detection image are open or not;
a counting operation step of counting the number of the detection images judged as the eyes are open by the first judgment step and calculating the ratio of the number to the total number of the detection images;
and a fatigue state determination step of comparing the ratio calculated in the count calculation step with a first set threshold value, determining that the driver is in a severe fatigue state when the ratio is equal to or less than the first set threshold value, and determining that the driver is in a mild fatigue state when the ratio is greater than the first set threshold value and equal to or less than a second set threshold value that is greater than the first set threshold value.
ADVANTAGEOUS EFFECTS OF INVENTION
According to the fatigue state detection device and the fatigue state detection method, effective early warning can be provided in the fatigue state that eyes are gradually closed, so that sufficient reaction time can be provided for a driver, and traffic accidents can be really and effectively avoided; meanwhile, false alarms which are easily generated by the conventional detection method can be reduced, the influence on the attention of a driver can be avoided, the psychological burden of the driver is avoided from being increased, further fatigue is avoided from being generated, the problems of serious delay of detection time and poor detection precision existing in the conventional fatigue detection method can be avoided, and the driving fatigue can be effectively prevented.
Drawings
Fig. 1 is a block diagram showing a configuration of a fatigue detection device according to the present invention.
FIG. 2 is a schematic diagram of the pre-stored feature extraction according to the present invention.
FIG. 3 is a schematic diagram illustrating the fatigue state determination according to the present invention.
Detailed Description
The fatigue state detection device and the fatigue state detection method according to the present invention will be described in detail below with reference to the accompanying drawings.
The fatigue state detection device consists of an image acquisition device, a data processing device, a data transmission device and a background server.
The image acquisition device adopts an infrared camera to shoot and obtain a plurality of facial images of a driver and transmits the images to the data processing device;
the data processing device adopts an embedded processor, can realize the processing of an input image and has a certain storage function;
the background server can simultaneously carry out data uplink and downlink transmission with the data transmission device, has stronger calculation and storage functions, and can further process the acquired images and data.
Feature registration
For the acquired face images, the eye positions in each image are respectively detected, the average size of the eyes of the driver is extracted as a pre-stored characteristic (namely a first threshold value),
the extraction step of the pre-stored characteristics comprises the following steps:
the method comprises the steps of firstly, detecting the eye position in each human face image in a video or continuous image;
secondly, calculating the opening degree P of eyes in the image (i is a natural number) based on the position detection of the upper eyelid and the lower eyelid and the method of the ratio of the iris area surrounded by the upper eyelid, the lower eyelid and the iris boundary to the whole iris areai
Third stepCalculating the average opening degree of eyes in the video or the continuous images, and determining the average opening degree as the pre-stored characteristic Th of the driverp
Wherein the pre-stored characteristic ThpCan be determined by the following method:
calculating a degree of eye opening P in a given succession of video or imagesi(P1,P2,P3,...,PN),Pi∈[0,100]Then (P)1,P2,P3,...,PN) The histogram of (hist) can be calculated using the following method:
Figure BSA0000227576830000051
wherein N is a natural number representing the number of consecutive images; c1Is a constant.
Then, the prestored characteristic value of the eye opening and closing state of the driver is determined by the following equation.
Thp=K0*C2(wherein, C2Is a constant).
In the detection device, C can be set1=0.2,C2=0.25。
Further, the pre-stored characteristic value of the driver can be stored in the data processing device, and can also be uploaded to the background server through the data transmission device.
Further, the background server may issue the pre-stored characteristic values of the driver or other drivers to one or more other data processing devices for storage, so as to facilitate fatigue detection in one or more data processing devices.
Further, a face recognition algorithm is added in the data processing device to identify the driver, and the pre-stored characteristics are bound with the identity information.
Fatigue detection
Acquiring one or more facial images of the driver by using the acquisition device, and inputting the facial images into the image processing device;
and calculating the size of eyes in the input image by adopting the image processing device, comparing the size with the pre-stored characteristics, and judging whether the driver displayed by the image is in an early fatigue state or not.
The fatigue state judging method comprises the following steps:
if Pi≥ThpThen the eyes in the ith image are considered to be open. The number of times n that the eye is open in successive videos or images over a period of time may then be countedopenedOccupying the total detection frame number n of the imagetotalPercent of (c), namely:
Perclos=nopened/ntotal
when Perclos is less than or equal to Thd1If so, the driver is considered to be in a severe fatigue state at present;
when Th isd1<Perclos≤Thd2When the driver is in the light fatigue state, the driver is considered to be in the light fatigue state.
In the detection device, a first set threshold Th is setd10.2, the second set threshold Thd2=0.4。
The fatigue detection device can execute corresponding reminding or emergency response according to the judgment result and the device setting; meanwhile, the judgment result, the related images and the like are uploaded to the background server through the data transmission device.
The inventors have experimentally verified the effects of the present invention.
The results of the tests on the company database are as follows:
the database contains facial images of 100 persons, 50 images per person, of which 10 images of fatigue states exist. The detection by the fatigue state detection device of the present invention indicates an equal error rate of 0.05% in the database. Therefore, the fatigue state detection device of the present invention can quickly and stably detect the early fatigue phenomenon.
In summary, the fatigue detection device and method of the present invention are summarized as follows.
A fatigue state detection device according to the present invention is a fatigue state detection device for detecting a fatigue state of a driver, the fatigue state detection device including: an image acquisition unit (corresponding to an image acquisition device) that acquires a plurality of static detection images or dynamic detection images of a predetermined time period in an area near a driver as detection images;
an eye opening degree calculation section (corresponding to a data processing device) that calculates an eye opening degree in each of the plurality of detection images acquired by the image acquisition section, respectively;
a first threshold value acquisition section (corresponding to a data processing device) that acquires a first threshold value (corresponding to a pre-stored feature) for determining whether or not an eye is open;
a first judging section (corresponding to a data processing device) that compares the degree of eye openness in each of the detection images calculated by the eye openness calculating section with a first threshold value to judge whether or not the eye in each of the detection images is open;
a count operation section (corresponding to the data processing device) that counts the number of detected images judged to be open by the first judgment section, and calculates a ratio of the number to the total number of detected images;
the fatigue state determination unit (corresponding to the data processing device) compares the ratio calculated by the count calculation unit with a first set threshold value, determines that the driver is in a heavy fatigue state when the ratio is equal to or less than the first set threshold value, and determines that the driver is in a light fatigue state when the ratio is greater than the first set threshold value and equal to or less than a second set threshold value that is greater than the first set threshold value.
In addition, the fatigue state detecting device of the invention, wherein,
the first threshold acquisition unit includes:
a target image acquisition unit that acquires a plurality of static target images or a moving target image of a predetermined time period in a region near a driver as target images;
an image processing unit that detects the iris boundary and the positions of the upper and lower eyelids in each target image, and calculates the ratio of the area of the region surrounded by the positions of the upper and lower eyelids and the iris boundary to the entire iris area as the degree of eye openness in the target image;
and a calculation unit that calculates an average value of the degrees of opening and closing of the plurality of eyes obtained by the image processing unit as a first threshold value.
In addition, the fatigue state detecting device of the invention, wherein,
the first threshold value obtaining unit obtains, as the first threshold value, an average value of the plurality of eye opening degrees calculated by the eye opening degree calculating unit or a predetermined value set in advance.
Further, the fatigue state detecting device of the present invention is characterized in that,
the first set threshold is 0.2 and the second set threshold is 0.4.
A fatigue state detection method according to the present invention is a method for detecting a fatigue state of a driver, the method including:
a detection image acquisition step of acquiring a plurality of static detection images or dynamic detection images of a specified duration of time in an area near a driver as detection images;
an eye opening degree calculation step of calculating an eye opening degree in each detection image for the plurality of detection images acquired in the image acquisition step;
a first threshold value obtaining step of obtaining a first threshold value for judging whether the eye is open;
a first judgment step of comparing the degree of opening of the eyes in each detection image calculated by the eye opening/closing degree calculation step with a first threshold value to judge whether the eyes in each detection image are open or not;
a counting operation step of counting the number of the detection images judged as the eyes are open by the first judgment step and calculating the ratio of the number to the total number of the detection images;
and a fatigue state determination step of comparing the ratio calculated in the count calculation step with a first set threshold value, determining that the driver is in a severe fatigue state when the ratio is equal to or less than the first set threshold value, and determining that the driver is in a mild fatigue state when the ratio is greater than the first set threshold value and equal to or less than a second set threshold value that is greater than the first set threshold value.
As described above, although the preferred embodiments of the present invention have been described, the above embodiments are presented as examples only and do not limit the scope of the present invention, and for example, the image size, the eye opening degree calculation method, the parameter of the pre-stored feature, and the like described above are merely examples, and other numbers may be used depending on the actually required environment, accuracy, and the like.
For example, the pilot of the present invention may be a vehicle pilot, a pilot of an aircraft or a ship, or an operator of other equipment.

Claims (5)

1. A fatigue state detection device for detecting a fatigue state of a driver, comprising:
an image acquisition unit that acquires a plurality of static detection images or a plurality of dynamic detection images of a predetermined time period in a region near a driver as detection images;
an eye opening degree calculation unit configured to calculate an eye opening degree in each of the plurality of detection images acquired by the image acquisition unit;
a first threshold value acquisition unit that acquires a first threshold value for determining whether or not an eye is open;
a first judging section that compares the degree of eye openness in each of the detection images calculated by the eye openness calculating section with a first threshold value to judge whether or not the eye in each of the detection images is open;
a counting operation part for counting the number of the detection images judged as the eyes are opened by the first judgment part and calculating the ratio of the number to the total number of the detection images;
and a fatigue state determination unit that compares the ratio calculated by the count calculation unit with a first set threshold value, determines that the driver is in a severe fatigue state when the ratio is equal to or less than the first set threshold value, and determines that the driver is in a mild fatigue state when the ratio is greater than the first set threshold value and equal to or less than a second set threshold value that is greater than the first set threshold value.
2. The fatigue state detecting device according to claim 1,
the first threshold acquisition unit includes:
a target image acquisition unit that acquires a plurality of static target images or a moving target image of a predetermined time period in a region near a driver as target images;
an image processing unit that detects the iris boundary and the positions of the upper and lower eyelids in each target image, and calculates the ratio of the area of the region surrounded by the positions of the upper and lower eyelids and the iris boundary to the entire iris area as the degree of eye openness in the target image;
and a calculation unit that calculates an average value of the degrees of opening and closing of the plurality of eyes obtained by the image processing unit as a first threshold value.
3. The fatigue state detecting device according to claim 1,
the first threshold value obtaining unit obtains, as the first threshold value, an average value of the plurality of eye opening degrees calculated by the eye opening degree calculating unit or a predetermined value set in advance.
4. The fatigue state detecting device according to claim 1,
the first set threshold is 0.2 and the second set threshold is 0.4.
5. A fatigue state detection method for detecting a fatigue state of a driver, comprising:
a detection image acquisition step of acquiring a plurality of static detection images or dynamic detection images of a specified duration of time in an area near a driver as detection images;
an eye opening degree calculation step of calculating an eye opening degree in each detection image for the plurality of detection images acquired in the image acquisition step;
a first threshold value obtaining step of obtaining a first threshold value for judging whether the eye is open;
a first judgment step of comparing the degree of opening of the eyes in each detection image calculated by the eye opening/closing degree calculation step with a first threshold value to judge whether the eyes in each detection image are open or not;
a counting operation step of counting the number of the detection images judged as the eyes are open by the first judgment step and calculating the ratio of the number to the total number of the detection images;
and a fatigue state determination step of comparing the ratio calculated in the count calculation step with a first set threshold value, determining that the driver is in a severe fatigue state when the ratio is equal to or less than the first set threshold value, and determining that the driver is in a mild fatigue state when the ratio is greater than the first set threshold value and equal to or less than a second set threshold value that is greater than the first set threshold value.
CN202011499151.8A 2020-12-10 2020-12-10 Fatigue state detection device and fatigue detection method Pending CN114627450A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011499151.8A CN114627450A (en) 2020-12-10 2020-12-10 Fatigue state detection device and fatigue detection method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011499151.8A CN114627450A (en) 2020-12-10 2020-12-10 Fatigue state detection device and fatigue detection method

Publications (1)

Publication Number Publication Date
CN114627450A true CN114627450A (en) 2022-06-14

Family

ID=81896589

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011499151.8A Pending CN114627450A (en) 2020-12-10 2020-12-10 Fatigue state detection device and fatigue detection method

Country Status (1)

Country Link
CN (1) CN114627450A (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120219189A1 (en) * 2009-10-30 2012-08-30 Shenzhen Safdao Technology Corporation Limited Method and device for detecting fatigue driving and the automobile using the same
CN104809445A (en) * 2015-05-07 2015-07-29 吉林大学 Fatigue driving detection method based on eye and mouth states
CN104881955A (en) * 2015-06-16 2015-09-02 华中科技大学 Method and system for detecting fatigue driving of driver
CN106846734A (en) * 2017-04-12 2017-06-13 南京理工大学 A kind of fatigue driving detection device and method
CN108720851A (en) * 2018-05-23 2018-11-02 释码融和(上海)信息科技有限公司 A kind of driving condition detection method, mobile terminal and storage medium
CN110276231A (en) * 2018-03-15 2019-09-24 厦门歌乐电子企业有限公司 A kind of detection device, the device and method of eyes opening degree

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120219189A1 (en) * 2009-10-30 2012-08-30 Shenzhen Safdao Technology Corporation Limited Method and device for detecting fatigue driving and the automobile using the same
CN104809445A (en) * 2015-05-07 2015-07-29 吉林大学 Fatigue driving detection method based on eye and mouth states
CN104881955A (en) * 2015-06-16 2015-09-02 华中科技大学 Method and system for detecting fatigue driving of driver
CN106846734A (en) * 2017-04-12 2017-06-13 南京理工大学 A kind of fatigue driving detection device and method
CN110276231A (en) * 2018-03-15 2019-09-24 厦门歌乐电子企业有限公司 A kind of detection device, the device and method of eyes opening degree
CN108720851A (en) * 2018-05-23 2018-11-02 释码融和(上海)信息科技有限公司 A kind of driving condition detection method, mobile terminal and storage medium

Similar Documents

Publication Publication Date Title
US9286515B2 (en) Doze detection method and apparatus thereof
CN104616438B (en) A kind of motion detection method of yawning for fatigue driving detection
JP5680667B2 (en) System and method for identifying driver wakefulness
US7620216B2 (en) Method of tracking a human eye in a video image
KR101159164B1 (en) Fake video detecting apparatus and method
CN110211335B (en) Method and device for detecting a microsleep of a driver of a vehicle
US8538063B2 (en) System and method for ensuring the performance of a video-based fire detection system
EP1732028A1 (en) System and method for detecting an eye
Anitha et al. A two fold expert system for yawning detection
CN108009495A (en) Fatigue driving method for early warning
KR20190083155A (en) Apparatus and method for detecting state of vehicle driver
CN113033521B (en) Perimeter dynamic early warning method and system based on target analysis
KR20090044957A (en) Theft and left baggage survellance system and meothod thereof
CN108460319B (en) Abnormal face detection method and device
CN115346060A (en) Picture abnormity identification method and device, electronic equipment and storage medium
EP3477941B1 (en) Method and controller for controlling a video processing unit based on the detection of newcomers in a first environment
CN112052779B (en) Access control system with palm vein recognition and iris recognition
Mašanović et al. Driver monitoring using the in-vehicle camera
CN114627450A (en) Fatigue state detection device and fatigue detection method
CN111860437A (en) Method and device for judging fatigue degree based on facial expression
KR101877873B1 (en) System and method for fear mentality analysis
Liu et al. Design and implementation of multimodal fatigue detection system combining eye and yawn information
JPH11161798A (en) Vehicle driver monitoring device
JP7212251B2 (en) Eye opening/closing detection device, occupant monitoring device
KR101582454B1 (en) Method for cognition of movement object in photographing image and system for prevention of vehicle boarding accident

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination