CN203885510U - Driver fatigue detection system based on infrared detection technology - Google Patents
Driver fatigue detection system based on infrared detection technology Download PDFInfo
- Publication number
- CN203885510U CN203885510U CN201420191486.7U CN201420191486U CN203885510U CN 203885510 U CN203885510 U CN 203885510U CN 201420191486 U CN201420191486 U CN 201420191486U CN 203885510 U CN203885510 U CN 203885510U
- Authority
- CN
- China
- Prior art keywords
- infrared
- eyes
- driver
- image
- face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 61
- 238000005516 engineering process Methods 0.000 title claims description 24
- 238000012545 processing Methods 0.000 claims abstract description 13
- 210000003128 head Anatomy 0.000 claims description 22
- 238000003909 pattern recognition Methods 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 abstract description 27
- 238000000034 method Methods 0.000 abstract description 16
- 206010016256 fatigue Diseases 0.000 description 56
- 210000001747 pupil Anatomy 0.000 description 16
- 230000000694 effects Effects 0.000 description 14
- 210000000744 eyelid Anatomy 0.000 description 10
- 230000000007 visual effect Effects 0.000 description 10
- 230000006870 function Effects 0.000 description 9
- 238000004458 analytical method Methods 0.000 description 7
- 238000012544 monitoring process Methods 0.000 description 7
- 238000011160 research Methods 0.000 description 7
- 238000013461 design Methods 0.000 description 6
- 238000005286 illumination Methods 0.000 description 6
- 238000011161 development Methods 0.000 description 5
- 230000001815 facial effect Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000001179 pupillary effect Effects 0.000 description 4
- 230000011514 reflex Effects 0.000 description 4
- 210000001525 retina Anatomy 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 241001282135 Poromitra oscitans Species 0.000 description 2
- 206010048232 Yawning Diseases 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 230000004418 eye rotation Effects 0.000 description 2
- 230000004438 eyesight Effects 0.000 description 2
- 230000002045 lasting effect Effects 0.000 description 2
- 238000007620 mathematical function Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 210000003195 fascia Anatomy 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000004899 motility Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
Landscapes
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Eye Examination Apparatus (AREA)
Abstract
The utility model relates to a driver fatigue detection system, and belongs to the technical field of detection equipment. The driver fatigue detection system comprises a CCD (Charge Coupled Device) camera and a CCD camera infrared light supply, wherein the CCD camera is connected to a digital signal processor (DSP) through a video input decoder and used for recognizing the mode of a detected image and then processing; the CCD camera infrared supply comprises two groups of infrared diodes which are uniformly distributed on a coaxial circular ring in the same plane; the two groups of infrared diodes are switched freely and can be used for emitting infrared lights being 850nm and 950nm in wavelength, respectively. The driver fatigue detection system is characterized in that the mouth shape is judged on the basis of judging the area of human eyes, so as to lower down the probability of neglected or false judging. According to the driver fatigue detection system, the method of differentiating an odd frame image and an even frame image is carried out to accurately detect the positions and characteristics of eyes and mouth shape, in order to accurately detect the level of fatigue of a driver; the Kalman filter and the Mean-shift algorithm are utilized to continuously detect the positions and characteristics of the eyes and mouth shape; the short-time tracking strategy is performed for tracking the positions of the eyes and the mouth shape, in order to realize the forecasting function.
Description
Technical field
This utility model relates to a kind of Study in Driver Fatigue State Surveillance System, belongs to checkout equipment technical field.
Background technology
The research method of driver fatigue detection at present can be divided into two large classes:
A) from driver's unique characteristics, by certain equipment, obtain driver's physiological parameter feature or visual signature, utilize driver different at the feature mode of normal condition and fatigue state, adopt corresponding mode identification technology classification to differentiate, thereby detect whether there be tired generation;
B) according to the behavior performance of vehicle, indirectly judge whether driver produces fatigue.In this class technology, by sensor, obtain vehicle various parameters in the process of moving, according to the abnormal conditions in Vehicle Driving Cycle process, as whether vehicle surpasses road mark line, speed, whether exceed the speed limit, whether the distance between vehicle is too near etc., judge whether driver has tired generation.
At present fatigue detecting system is many, effect is reasonable have following several:
A) DAS of Australian International University of Japan exploitation, commercially obtains and uses at present.Use is arranged on the tracing of human eye system monitoring driver on fascia, can monitoring driving person show, and utilizes the grip feedback of steering wheel, road surface tracing deviation is fed back to driver simultaneously.Road surface tracking equipment object is to detect the vehicle unlawful practices such as pavement marker or road edge of nuzzling up suddenly, and this system provides warning detecting when abnormal, and has the braking function of steering wheel.The degree that the seat vibration warning simultaneously adopting departs from road side is relevant, to encourage them to correct deviated route.
B) European Union completed AWAKE engineering in 2004.The significant condition adopting comprises that eyelid movement, grip change and follow the tracks of on road surface, and the braking action such as apply the brakes and steering wheel position, and these methods are combined and resist traffic risk.
C) the Copilot monitoring system of Carnegie Mellon university exploitation.By Perelose, measure eyelid movement, monitoring system is little and handy, and a kind of effective research tool is provided.
D) the FaeeLAB III system monitoring driving behavior of Seeing machine seminar exploitation, can detect the situations such as tired and energy dispersion.By a pair of video camera, obtain video image, from left to right coupling draws the three-dimensional position of each feature.Adopt least square optimization location head three-dimensional position, FaceLAB software parallel is processed eye gaze data, and eye gaze direction is determined according to eye gaze vector in iris center, location, calculates eyes and opens and frequency of wink, monitoring eyelid.It is very effective that FaceLAB is proved to be in drive simulating, and have good effect on low light, head grand movement and visual direction are followed the tracks of, even if driver is with sunglasses also can detect.Can find out, whether system utilizes the comprehensive characteristics such as behavior performance of various visual signatures and vehicle to go to detect tired generation, and some type of alarm of the also employing simultaneously having reminds driver to drive with caution.At home this respect is related at present fewerly, only have a small amount of achievement in research.
The problem and the development trend that exist
Because the driving environment differences such as driver's individual variation and light, road surface are large, current fatigue detecting algorithm is substantially all based on drive simulating environment, next step will develop on true driving environment, make fatigue detecting technology be widely applied to commercial field, reduce fatigue driving and the vehicle accident that causes.No matter be from driver's unique characteristics or from vehicle behavior aspect, the characteristic type that can directly obtain is limited, and the surface character data of directly extracting are many and have a redundancy.Therefore to excavate fatigue characteristic on the one hand, with advanced signal processing method, extract and can characterize tired characteristic parameter, to adopt signal fused processing method on the other hand, a plurality of fatigue characteristic parameters are combined driver fatigue situation is detected, overcome the impacts such as space, illumination, improve real-time, the accuracy of detection algorithm.Study in Driver Fatigue State Surveillance System should have tired decision making function.Because different people has different tired performance characteristics, fatigue detecting system should have intelligence, has self study, inference function.At the initial stage of driving, system is trained system according to the related data that obtains driver, and the fatigue characteristic that can draw driver is selected the most applicable his detection method, can adapt to each driver.
Current public fatigue driving data set relatively lacks, and detection system method of evaluating performance disunity is difficult to the performance of various fatigue detecting technologies to carry out Quantitative Comparison.For quantitatively, more various detection techniques objectively, promote the development of fatigue detection method, need to strengthen the construction to fatigue driving vedio data storehouse, integrated system evaluating method and standard.
Fatigue detecting equipment mainly concentrates on the detection to driver's eye both at home and abroad at present, does not realize the judgement of lip-syncing type, therefore easily occurs the situation of failing to judge and misjudging; In addition, fatigue driving equipment can only detect eye position and eye feature in real time both at home and abroad at present, can not realize the forecast function to eye position and eye feature, therefore driver's fatigue state is not realized to forecast function; Fatigue detecting equipment takes the method for odd even two two field picture pixel comparison to detect eye position and feature both at home and abroad at present, and detection speed is slow, and accuracy of detection is low.
Summary of the invention
The purpose of this utility model is to solve the deficiency that above-mentioned prior art exists, and a kind of Study in Driver Fatigue State Surveillance System based on infrared detection technology is provided.
Native system, by various fatigue detection methods are compared to analysis, has proposed the Study in Driver Fatigue State Surveillance System design of the information such as a set of comprehensive driver's eyes fatigue characteristic, head pose, countenance feature, driver's direction of gaze.Based on human eye, the reflectance of two kinds of wavelength (850nm/950nm) Infrared is had the feature of notable difference, system adopts infrared light supply to gather image, takes odd even two two field pictures to carry out the method for difference, accurately detects eyes.Utilize Kalman wave filter and Mean-shift algorithm to take to adopt continuous detecting, tracking strategy in short-term, eye position is followed the tracks of, predicted, analyze the information of eyes and calculate the countenance feature that eye feature, head pose and mouth states can show, by to the shallow degree of these feature-sets is tired, moderate is tired and overtired criterion provide the warning of different stage, realize the function of fatigue detecting.
A kind of Study in Driver Fatigue State Surveillance System based on infrared detection technology of the present utility model, include the CCD photographic head based on infrared detection technology, CCD photographic head is connected to digital signal processor DSP infrared detection image by video input decoder, detected image is carried out to pattern recognition and processing, digital signal processor is connected with alarm, by judging that the switching of driver's eyes judges that whether driver is tired, and further can be by judging whether eigenvalue and the positional value of eyes and face reaches setting numerical value, system reminds driver to take a good rest and driving safety by alarm generation alarm sound.
The above-mentioned CCD photographic head based on infrared detection technology, infrared light supply is comprised of two groups of inner and outer ring infrared diodes, is evenly distributed on an annulus of isoplanar concentric, freely switches between the two, and the visual field limited range of video camera is reduced;
Described two groups of infrared diodes are launched respectively the infrared light of 850nm wavelength, 950nm wavelength, realize the detection of infrared detection technology to eyes.
(1) principle of infrared detection technology to eye detection
Human eye is different to the infrared light reflection range degree of different wave length, and when 850nm wavelength, retina can reflect 90% incident illumination, when 950nm, retina can only reflect 40% incident illumination, and within the scope of 880 ± 80nm, other parts of people's face are basically identical for infrared degree of reflection.Infrared and the 950nm of 850nm wavelength is infrared to be irradiated respectively the image that people's face obtains and carries out Difference Calculation, can accurately detect eyes.The optimum size that is arranged on the light-emitting diodes pipe ring of lens surface can only be an empirical value.If a circle diode of outside is opened, will produce dark pupil effect, a circle diode of the inside is opened and will be produced bright pupil effect.
(2) positioning analysis of eyes and tracing algorithm
A) detection of eyes
Infrared detection principle based on above-mentioned completes the detection of eyes, after Infrared irradiation people face, utilizes CCD camera collection people face video image.The reflected image Difference Calculation that collects two wavelength, can detect eyes.Due to effect of noise, the image that difference obtains need to carry out pretreatment, first carry out rectangular histogram equalization processing, the gray level that number of pixels in image is many is carried out to broadening, the gray level that number of pixels is few is reduced, the histogram transformation of original image is become to equally distributed form, be then converted into bianry image.
B) eyes tracing algorithm
Mean-Shift algorithm and Kalman wave filter are combined, adopt continuous detecting, tracking strategy in short-term, when Kalman filter detection is to human eye, directly adopt its result detecting, record present image and position of human eye simultaneously; If can't detect, just with image and the position of human eye of last record, Mean-Shift algorithm is initialized, and use Mean-Shift algorithm to match search human eye area on present image, if Kalman wave filter still can't detect in successive image, carry out the tracking of Mean-Shift algorithm always.
(3) eyelid movement parameter
People enters after fatigue, and eyes there will be opens the features such as the speed of closing is slack-off, eye rotation frequency lowers, eyelid has closed trend, One's eyesight is restrained.Extract the characteristic information of eyelid, utilize the average closure speed algorithm of eyes closed PERCLOS average time algorithm and eyes to calculate driver's fatigue data.
PERCLOS principle
Shared time scale when PERCLOS refers within regular hour eyes closed.
In specific experiment, there are P70, P80, three kinds of modules of EYEMEA (EM):
P70: more than 70% percentage of time of eyes closed area;
P80: more than 80% percentage of time of eyes closed area, this index is the most frequently used; Wherein P80 is considered to can reflect people's degree of fatigue.
Fig. 5 is the measuring principle figure of PERCLOS value.In figure, to be eyes closed open degree curve over time with opening to curve in process, can obtain certain degree of eyes closed of required measurement or open the lasting time according to this curve, thereby calculate PERCLOS value.T in figure
1for eyes are opened time of closed 20% completely; t
2for eyes are opened time of closed 80% completely; t
3for eyes are opened the time of opening 20% completely next time; t
4for eyes are opened the time of opening 80% completely next time.By measuring t
1to t
4value just can calculate the value of PERCLOS:
In formula, f is the percentage rate of shared a certain special time of eyes closed time
For P80 metering system, when PERCLOS value f>0.15, think that driver is in fatigue state.
(4) human face posture analysis
Driver's facial pose utilizes the positional information of eyes to make assessment
Δ x=x in formula
2-x
1, Δ y=y
2-y
1, (x
2, y
2) be the coordinate position of left and right eyes.
When face be partial to left to time, θ > is 8 °;
When face is during at dead ahead, | θ |≤8 °;
When face is partial to right, θ <-8 °;
(5) eye gaze Direction estimation and tracking
People's the information of watching attentively comprises the direction of gaze of face and the direction of gaze of eyes, and than the direction of gaze of eyes, the direction of gaze visual field of face is larger, therefore people's face is watched attentively as broad sense and is watched attentively for a long time, and eyes are watched attentively as narrow sense and watched attentively for a long time.In design, consider these two and watch information attentively.Broad sense is watched attentively through head pose information and is calculated, and narrow sense is watched the eye pupil geometric parameter of taking through video camera attentively and calculated.Because the shape of pupil and direction are along with head pose changes, therefore, broad sense is watched information attentively and finally also with the geometric parameter of pupil, is calculated.If Δ x and Δ y are the displacement deflection parameters of pupil-reflective effect; R is the ratio of pupillary reflex image ellipse major axis and minor axis; θ is oval deflection; g
x, g
yit is the coordinate function of pupillary reflex light image.Δ x and Δ y have reflected the functional relationship between reflected image and through hole, are that narrow sense is watched information attentively.R represents that face rotates in anti-phase plane, and when people's face is at dead ahead time, ratio is 1, and when face moves up and down, ratio becomes large or diminishes.Angle θ is facial movement direction in camera objective optical axial plane, (g
x, g
y) the interior pupil center-of-mass coordinate of plane.
Obtain above six parameters, utilize extensive recurrent neural networks to eye gaze directional structure vectorical structure mathematical function, calculate driver's the information of watching attentively, if driver is overtime to certain direction fixation time, judge fatigue state.
(6) countenance analysis
Utilize the status information of mouth to describe countenance feature.Eyes detected, people's face area is also determined thereupon.The eyes coordinates position detecting is divided into upper and lower two parts by people's face, then adopt iterative threshold values selection algorithm to carry out threshold values processing to the latter half of people's face, thereby make up owing to can not get the problem of face integrity profile in lip and the little binary image causing of colour of skin difference.The image of taking is converted to gray level image, and threshold values is carried out to automatization's threshold values processing, can obtain the profile information in face region.
Because the gray value of lip is more shallow than people face, thereby after binary conversion treatment, there will be hole region, " cavity " and the impact of noise spot that also there will be nostril to cause simultaneously.Obtain, after the profile information in face region, image is carried out to denoising, utilize minimum boundary rectangle to express the profile of hole region that face produces, finally utilize classical Harris corner detection approach to search the corners of the mouth.This utility model detects angle point to carry out in original gray level image, and only detects face region.The specific algorithm of Corner Detection is as follows:
R＝det(M)-k(tr
2(M))
In formula
I wherein
u(x, y), I
v(x, y), I
uv(x, y) is respectively gray scale that picture point (x, y) locates at partial derivative and the second-order mixed partial derivative of u and v direction; K is constant; The mark that tr (M) is Metzler matrix.
The upper lower lip height that the degree that face opens is opened by face and the ratio of left and right corners of the mouth width are determined.Utilize the yawning frequency computation part driver fatigue of face degree.
The utlity model has following innovative point
1, the judgement that has added again lip-syncing type on the basis of judgement human eye area, has reduced the probability of failing to judge or misjudging.
2, odd even two two field pictures are carried out to the method for difference, accurately detect position and the feature of eyes, mouth type, realize the accurate detection to driver fatigue degree.
3, native system adopts Kalman wave filter and Mean-shift algorithm, position and the feature of eyes and mouth type is realized to continuous detecting, and adopt tracking strategy in short-term, eyes and mouth type position is followed the tracks of, and realized forecast function.
Accompanying drawing explanation
Fig. 1: the principle schematic of bright pupil (above) and dark pupil (below) effect;
Fig. 2: infrared light supply is arranged schematic diagram;
Fig. 3: PERCLOS principle schematic;
Fig. 4: the fundamental diagram of this utility model Study in Driver Fatigue State Surveillance System;
Fig. 5: the hardware configuration schematic diagram of this utility model Study in Driver Fatigue State Surveillance System.
The specific embodiment
Referring to accompanying drawing, provide this utility model specific embodiment, this utility model is described further.
Embodiment 1
The Study in Driver Fatigue State Surveillance System of the present embodiment based on infrared detection technology, include the vehicle-borne CCD photographic head 1 based on infrared detection technology, CCD photographic head 1 is picked-up driver's facial image continuously, and ceaselessly follow the tracks of the motion of driver's eyes, CCD photographic head is connected to digital signal processor 5 infrared detection image by video input decoder 4, detected image is carried out to pattern recognition and processing, by judging that the switching of driver's eyes judges that whether driver is tired, further by judging whether eigenvalue and the positional value of eyes and face reaches setting numerical value, by alarm 6, there is alarm sound and remind driver to take a good rest and driving safety in system.
The above-mentioned CCD photographic head 1 based on infrared detection technology, infrared light supply is comprised of first group of internal ring infrared diode 2 and second group of outer shroud infrared diode 3, be evenly distributed on an annulus of isoplanar concentric, freely switch between the two, the visual field limited range of video camera is reduced;
Described two groups of infrared diodes are launched respectively the infrared light of 850nm wavelength, 950nm wavelength, realize the detection of infrared detection technology to eyes.
CCD photographic head based on infrared detection technology, detected image is sent to digital signal processor DM642, native system software adopts MATLAB development environment to develop fatigue driving instrument and detects software, by simulink software conversionization C code, on the computer of CCS development environment, C code is write to emulator 510, and further conversion turns to assembler language and writes in hardware development plate DM642, detected image is carried out to pattern recognition and processing, when the eigenvalue of eyes and face and positional value reach while setting numerical value, will there is alarm sound and remind driver to take a good rest and driving safety in system.
The full name of DM642 TMS320DM642, up-to-date fixed DSP in the C6000 of ShiTI company series DSP, its core is C6416 type high-performance digital signal processor, there is extremely strong handling property, motility and the programmability of height, periphery is integrated simultaneously very complete equipment and the interface such as audio frequency, video and network service, the high-speed dsp applications such as consumer electronics product that are specially adapted to machine vision, medical imaging, Network Video Surveillance, digital broadcasting and process based on digital video/image.Take TVP5150 as video input decoder, take PCM1801 as audio frequency input Acquisition Circuit, the multi-channel video capturing that the TMS320DM642 type DSP of take is core processor is held concurrently and is compressed processing PCI, is applied to build high stability, high robust and multimedia digital monitoring system.
TMS320DM642 adopts the DSP core of second filial generation high-performance, advanced very long instruction word veloci T1.2 structure and the parallel mechanism of enhancing, peripheral integrated very complete audio frequency, video and network communication interface.
Concrete technical solution is analyzed as follows.
1, technical scheme
(1) principle of infrared detection technology to eye detection
Human eye is different to the infrared light reflection range degree of different wave length, and when 850nm wavelength, retina can reflect 90% incident illumination, when 950nm, retina can only reflect 40% incident illumination, and within the scope of 880 ± 80nm, other parts of people's face are basically identical for infrared degree of reflection.Infrared and the 950nm of 850nm wavelength is infrared to be irradiated respectively the image that people's face obtains and carries out Difference Calculation, can accurately detect eyes.The optimum size that is arranged on the light-emitting diodes pipe ring of lens surface can only be an empirical value.If a circle diode of outside is opened, will produce dark pupil effect, a circle diode of the inside is opened and will be produced bright pupil effect.(seeing Fig. 1,2)
(2) positioning analysis of eyes and tracing algorithm
A) detection of eyes
Infrared detection principle based on above-mentioned completes the detection of eyes, after Infrared irradiation people face, utilizes CCD to gather people's face video image.Ccd video camera operation principle is based on active infrared camera technique, light sensor in video camera detects visible ray and is not enough to reach while gathering the requiring of image, control module in video camera will automatically be opened solid luminescent infrared lamp and be carried out floor light, within the scope that the spectrum of infrared-emitting diode infrared lamp can be experienced in common CCD camera, and because human eye is insensitive to infrared light, so the impact that when thermal camera can eliminate night running, illumination deficiency is brought gathering video, can not disturb normally travelling of driver again.
The reflected image Difference Calculation that collects two wavelength, can detect eyes.Due to effect of noise, the image that difference obtains need to carry out pretreatment, first carry out rectangular histogram equalization processing, the gray level that number of pixels in image is many is carried out to broadening, the gray level that number of pixels is few is reduced, the histogram transformation of original image is become to equally distributed form, be then converted into bianry image.
B) eyes tracing algorithm
This utility model combines Mean-Shift algorithm and Kalman wave filter, adopt continuous detecting, tracking strategy in short-term, while being Kalman filter detection to human eye, directly adopt its result detecting, record present image and position of human eye simultaneously; If can't detect, just with image and the position of human eye of last record, Mean-Shift algorithm is initialized, and use Mean-Shift algorithm to match search human eye area on present image, if Kalman wave filter still can't detect in successive image, carry out the tracking of Mean-Shift algorithm always.
(3) eyelid movement parameter
People enters after fatigue, and eyes there will be opens the features such as the speed of closing is slack-off, eye rotation frequency lowers, eyelid has closed trend, One's eyesight is restrained.Extract the characteristic information of eyelid, utilize the average closure speed algorithm of eyes closed PERCLOS average time algorithm and eyes to calculate driver's fatigue data.
PERCLOS principle
Shared time scale when PERCLOS refers within regular hour eyes closed.
In specific experiment, there are P70, P80, three kinds of modules of EYEMEA (EM):
P70: more than 70% percentage of time of eyes closed area;
P80: more than 80% percentage of time of eyes closed area, this index is the most frequently used; Wherein P80 is considered to can reflect people's degree of fatigue.
Fig. 3 is the measuring principle figure of PERCLOS value.In figure, to be eyes closed open degree curve over time with opening to curve in process, can obtain certain degree of eyes closed of required measurement or open the lasting time according to this curve, thereby calculate PERCLOS value.T in figure
1for eyes are opened time of closed 20% completely; t
2for eyes are opened time of closed 80% completely; t
3for eyes are opened the time of opening 20% completely next time; t
4for eyes are opened the time of opening 80% completely next time.By measuring t
1to t
4value just can calculate the value of PERCLOS:
In formula, f is the percentage rate (seeing Fig. 3) of shared a certain special time of eyes closed time
For P80 metering system, when PERCLOS value f>0.15, think that driver is in fatigue state.
(4) human face posture analysis
Driver's facial pose utilizes the positional information of eyes to make assessment
Δ x=x in formula
2-x
1, Δ y=y
2-y
1, (x
2, y
2) be the coordinate position of left and right eyes.
When face be partial to left to time, θ > is 8 °;
When face is during at dead ahead, | θ |≤8 °;
When face is partial to right, θ <-8 °;
(5) eye gaze Direction estimation and tracking
People's the information of watching attentively comprises the direction of gaze of face and the direction of gaze of eyes, and than the direction of gaze of eyes, the direction of gaze visual field of face is larger, therefore people's face is watched attentively as broad sense and is watched attentively for a long time, and eyes are watched attentively as narrow sense and watched attentively for a long time.In design, consider these two and watch information attentively.Broad sense is watched attentively through head pose information and is calculated, and narrow sense is watched the eye pupil geometric parameter of taking through video camera attentively and calculated.Because the shape of pupil and direction are along with head pose changes, therefore, broad sense is watched information attentively and finally also with the geometric parameter of pupil, is calculated.If Δ x and Δ y are the displacement deflection parameters of pupil-reflective effect; R is the ratio of pupillary reflex image ellipse major axis and minor axis; θ is oval deflection; g
x, g
yit is the coordinate function of pupillary reflex light image.Δ x and Δ y have reflected the functional relationship between reflected image and through hole, are that narrow sense is watched information attentively.R represents that face rotates in anti-phase plane, and when people's face is at dead ahead time, ratio is 1, and when face moves up and down, ratio becomes large or diminishes.Angle θ is facial movement direction in camera objective optical axial plane, (g
x, g
y) the interior pupil center-of-mass coordinate of plane.
Obtain above six parameters, utilize extensive recurrent neural networks to eye gaze directional structure vectorical structure mathematical function, calculate driver's the information of watching attentively, if driver is overtime to certain direction fixation time, judge fatigue state.
(6) countenance analysis
This utility model utilizes the status information of mouth to describe countenance feature.Eyes detected, people's face area is also determined thereupon.The eyes coordinates position detecting is divided into upper and lower two parts by people's face, then adopt iterative threshold values selection algorithm to carry out threshold values processing to the latter half of people's face, thereby make up owing to can not get the problem of face integrity profile in lip and the little binary image causing of colour of skin difference.The image of taking is converted to gray level image, and threshold values is carried out to automatization's threshold values processing, can obtain the profile information in face region.
Because the gray value of lip is more shallow than people face, thereby after binary conversion treatment, there will be hole region, " cavity " and the impact of noise spot that also there will be nostril to cause simultaneously.Obtain, after the profile information in face region, image is carried out to denoising, utilize minimum boundary rectangle to express the profile of hole region that face produces, finally utilize classical Harris corner detection approach to search the corners of the mouth.This utility model detects angle point to carry out in original gray level image, and only detects face region.The specific algorithm of Corner Detection is as follows:
R＝det(M)-k(tr
2(M))
In formula
I wherein
u(x, y), I
v(x, y), I
uv(x, y) is respectively gray scale that picture point (x, y) locates at partial derivative and the second-order mixed partial derivative of u and v direction; K is constant; The mark that tr (M) is Metzler matrix.
The upper lower lip height that the degree that face opens is opened by face and the ratio of left and right corners of the mouth width are determined.Utilize the yawning frequency computation part driver fatigue of face degree.
2. the feasibility of technical scheme is discussed
Pattern recognition is now very ripe, and project adopts the algorithm based on pattern recognition to carry out system design, has certain reliability on algorithm.In design, first from Arithmetic of Face Image Recognition via, carry out coding and build visual detecting system, simplify experiment difficulty; Video camera is directly accessed to computer, and detection trace information can be transplanted in dsp chip after having tested easily, has feasibility and maturity.
(1). the infrared beams of specific wavelength, along the irradiation optical axis people face of video camera, can obtain bright pupil effect.If but departed from optical axis, the light of reflection cannot enter video camera will cause dark pupil effect.Infrared light supply arranges and just can have bright pupil effect along the optical axis of lens in theory.But in fact this point is difficult to, and infrared light supply can limit the visual field of photographic head.This utility model consists of infrared light supply two groups of infrared diodes, be evenly distributed on an annulus of isoplanar concentric, freely switches between the two, and the visual field limited range of video camera is reduced.
(2). to the detection of driver fatigue, be mainly the contact measurement method based on driver's physiological signal, driver behavior behavior, vehicle-state at present.Although the bio-signal acquisition accuracy based on driver is higher, affects driver's driving; Based on vehicle-state, measurement has unreliability and non-operability.By comparison, the contactless detection device based on face characteristic does not have Body contact, measures in real time, driver is not had to nocuity radiation with driver.In design by effective combination of multiple fatigue detection method, the comprehensive detection of realization to information such as driver's eyes feature, head pose, countenance feature, eye gaze directions, the fatigue state of finally driver's fatigue being made to different brackets is divided and is made corresponding early warning and judges.
(3). utilize iterative threshold values selection method to carry out threshold values processing in second region of people's face, well improved unintelligible, the connective bad defect of corners of the mouth profile that traditional binarization method causes.
This utility model object and meaning:
1. along with the developing rapidly of Modern Traffic, vehicle accident is more and more serious, and fatigue driving has become the key factor of traffic safety hidden danger.Under the pressure of security consideration, fatigue driving causes common people's attention, and various countries' research worker is devoted to research in this respect one after another, and has obtained certain achievement.And China's research starting is in this regard late, research of the present utility model can advance China's research and development in this regard.
2. the tired measuring technique of feature more than is under the support of infrared light supply, extract accurately driver's eye feature, detect driver's eyelid movement characteristic parameter (blink speed, open the time of closing one's eyes, pupil geometric properties), head pose, eye gaze directional information, thereby calculate fatigue strength; Extract in addition the characteristic parameter of driver's face and measure the countenance feature of driver face under fatigue state.Driver's fatigue detecting is carried out to the strategy that combination features detects, driver's fatigue conditions is made in real time and accurately being detected, will bring huge meaning for traffic safety.
Claims (2)
1. the Study in Driver Fatigue State Surveillance System based on infrared detection technology, it is characterized in that including the CCD photographic head (1) based on infrared detection technology, CCD photographic head is connected to infrared detection image the digital signal processor (5) that detected image is carried out to pattern recognition and processing by video input decoder (4), digital signal processor (5) is connected with alarm (6);
The infrared light supply of the described CCD photographic head based on infrared detection technology is comprised of first group of internal ring infrared diode (2) and second group of outer shroud infrared diode (3), is evenly distributed on an annulus of isoplanar concentric.
2. according to a kind of Study in Driver Fatigue State Surveillance System based on infrared detection technology described in claim 1, it is characterized in that described two groups of infrared diodes launch respectively the infrared light of 850nm wavelength, 950nm wavelength, realize the detection of infrared detection technology to eyes.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201420191486.7U CN203885510U (en) | 2013-12-24 | 2014-04-18 | Driver fatigue detection system based on infrared detection technology |
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201320858305.7 | 2013-12-24 | ||
| CN201320858305 | 2013-12-24 | ||
| CN201420191486.7U CN203885510U (en) | 2013-12-24 | 2014-04-18 | Driver fatigue detection system based on infrared detection technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN203885510U true CN203885510U (en) | 2014-10-22 |
Family
ID=51711967
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201420191486.7U Expired - Lifetime CN203885510U (en) | 2013-12-24 | 2014-04-18 | Driver fatigue detection system based on infrared detection technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN203885510U (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104224204A (en) * | 2013-12-24 | 2014-12-24 | 烟台通用照明有限公司 | Driver fatigue detection system on basis of infrared detection technology |
| CN108430321A (en) * | 2015-12-23 | 2018-08-21 | 皇家飞利浦有限公司 | Equipment, system and method for the vital sign for determining people |
| CN111754728A (en) * | 2019-03-29 | 2020-10-09 | 京东方科技集团股份有限公司 | Detection assembly, detection device and detection method for fatigue driving |
-
2014
- 2014-04-18 CN CN201420191486.7U patent/CN203885510U/en not_active Expired - Lifetime
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104224204A (en) * | 2013-12-24 | 2014-12-24 | 烟台通用照明有限公司 | Driver fatigue detection system on basis of infrared detection technology |
| CN104224204B (en) * | 2013-12-24 | 2016-09-07 | 烟台通用照明有限公司 | A kind of Study in Driver Fatigue State Surveillance System based on infrared detection technology |
| CN108430321A (en) * | 2015-12-23 | 2018-08-21 | 皇家飞利浦有限公司 | Equipment, system and method for the vital sign for determining people |
| CN108430321B (en) * | 2015-12-23 | 2023-01-10 | 皇家飞利浦有限公司 | Device, system and method for determining vital signs of a person |
| CN111754728A (en) * | 2019-03-29 | 2020-10-09 | 京东方科技集团股份有限公司 | Detection assembly, detection device and detection method for fatigue driving |
| US11300446B2 (en) | 2019-03-29 | 2022-04-12 | Beijing Boe Optoelectronics Technology Co., Ltd. | Optical detection method and optical detection apparatus for fatigue state, and optical detection device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104224204B (en) | A kind of Study in Driver Fatigue State Surveillance System based on infrared detection technology | |
| CN102324166B (en) | Fatigue driving detection method and device | |
| CN101950355B (en) | Method for detecting fatigue state of driver based on digital video | |
| CN101593425B (en) | Machine vision based fatigue driving monitoring method and system | |
| Jo et al. | Vision-based method for detecting driver drowsiness and distraction in driver monitoring system | |
| CN101639894B (en) | Method for detecting train driver behavior and fatigue state on line and detection system thereof | |
| CN100462047C (en) | Safe driving auxiliary device based on omnidirectional computer vision | |
| EP1320830B1 (en) | Facial image processing system | |
| CN104013414A (en) | Driver fatigue detecting system based on smart mobile phone | |
| CN105769120A (en) | Fatigue driving detection method and device | |
| CN105719431A (en) | Fatigue driving detection system | |
| CN105286802B (en) | Driver Fatigue Detection based on video information | |
| Batista | A drowsiness and point of attention monitoring system for driver vigilance | |
| CN104574817A (en) | Machine vision-based fatigue driving pre-warning system suitable for smart phone | |
| CN202257856U (en) | Driver fatigue-driving monitoring device | |
| CN105844128A (en) | Method and device for identity identification | |
| CN104616438A (en) | Yawning action detection method for detecting fatigue driving | |
| Flores et al. | Driver drowsiness detection system under infrared illumination for an intelligent vehicle | |
| CN103714659A (en) | Fatigue driving identification system based on double-spectrum fusion | |
| CN104881956A (en) | Fatigue driving early warning system | |
| CN203885510U (en) | Driver fatigue detection system based on infrared detection technology | |
| CN100589135C (en) | Automatic door bell and its implementation | |
| CN203677103U (en) | CCD (Charge Coupled Device) camera based on infrared detection technology | |
| Lang et al. | The study of driver fatigue monitor algorithm combined PERCLOS and AECS | |
| CN105894732A (en) | Fatigue driving monitoring system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20190129 Address after: 264006 No. 69 Changjiang Road, Yantai Development Zone, Shandong Province Patentee after: YANTAI TAIMING LIGHTING Co.,Ltd. Address before: 264006 No. 69 Changjiang Road, Yantai Development Zone, Shandong Province Patentee before: YANTAI GENERAL LIGHTING Co.,Ltd. |
|
| TR01 | Transfer of patent right | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 264006 No. 69 Changjiang Road, Yantai Development Zone, Shandong, China Patentee after: Taiming Intelligent Technology Development Co.,Ltd. Address before: 264006 No. 69 Changjiang Road, Yantai Development Zone, Shandong, China Patentee before: YANTAI TAIMING LIGHTING Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder | ||
| CX01 | Expiry of patent term |
Granted publication date: 20141022 |
|
| CX01 | Expiry of patent term |