CN101968918A - Feedback type fatigue detecting system - Google Patents
Feedback type fatigue detecting system Download PDFInfo
- Publication number
- CN101968918A CN101968918A CN 201010526783 CN201010526783A CN101968918A CN 101968918 A CN101968918 A CN 101968918A CN 201010526783 CN201010526783 CN 201010526783 CN 201010526783 A CN201010526783 A CN 201010526783A CN 101968918 A CN101968918 A CN 101968918A
- Authority
- CN
- China
- Prior art keywords
- fatigue
- driver
- fatigue detecting
- voice
- detecting unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides a feedback type fatigue detecting system which is characterized by comprising a sensing module, a fatigue detecting module and a sound alarm and feedback module, wherein the sensing module is used for collecting an image optical signal and/or a sound signal from a driver; the fatigue detecting module is connected with the sensing module and is used for judging the fatigue state of the driver according to the collected image optical signal and/or the sound signal; and the sound alarm and feedback module is connected with the fatigue detecting module and is used for sending out a sound alarm signal and a sound interactive signal according to the fatigue state of the driver and feeding the related information of the sound interactive signal back to the fatigue detecting module as a reference standard for judging the fatigue state of the driver. The feedback type fatigue detecting system has high accuracy for fatigue detection of the driver and can effectively avoid generating false alarm and leakage alarm.
Description
Technical field
The present invention relates to vehicle safety and Monitoring and Controlling field, particularly a kind of reaction type fatigue detecting system.
Background technology
Fatigue driving is one of important hidden danger of current traffic safety.The driver is when fatigue, and therefore its perception to surrounding environment, situation judgement and the ability of controlling of vehicle all had in various degree decline be easy to take place traffic hazard.Detection early warning technology about safe conditions such as driver's fatigue and attention dispersion, owing to it is subjected to the attention of various countries' height in the development prospect aspect the traffic accident prevention, the researchist has carried out many-sided research at physiology and operational feature according to the driver when tired.The detection method of driver's fatigue state can roughly be divided into based on driver's operation behavior, based on car status information, based on driver's physiological signal, based on four kinds of detection methods of driver's physiological reaction feature.
Driver's fatigue state recognition technology based on driver's operation behavior is meant by driver's operation behavior and infers driver's fatigue state.For example detect, and detected data are handled, thereby draw the service data of driver's bearing circle and the relation between the fatigue data by bearing circle operation to the driver.
Based on the fatigue state recognition technology of car status information, be meant and utilize vehicle driving trace to change and lane line such as departs from the fatigue state that the vehicle driving information is inferred the driver.This method is the same with fatigue state recognition technology based on driver's operation behavior, all based on the vehicle conventional device, need not add too much hardware device, and can not cause interference to driver's normal driving, therefore has higher utility.
But, driver's operation behavior and car status information are except outside the Pass the fatigue state with the driver has, also be subjected to the influence of factors such as personal habits, travel speed, road environment, operative skill, therefore based on driver's operation behavior with based on the fatigue detecting method of car status information, it exists and detects the higher shortcoming of error rate.
Detection method based on driver's physiological signal starts from physiology the earliest.Correlative study shows that the physical signs of driver under fatigue state can depart from the index of normal condition.Therefore can judge whether the driver enters fatigue state by driver's physical signs.At present comparatively ripe detection method comprises the measurement of EEG signals EEG to the driver, core signal ECG etc.For example when entering fatigue state, the delta ripple among the EEG signals EEG and the activity of theta ripple can increase substantially, and the activity of alpha ripple has and increase by a small margin, and core signal ECG can significantly clocklike descend.Therefore by just can know driver's fatigue state to the measurement of EEG signals EEG, core signal ECG.
Though the detection method based on driver's physiological signal is higher to the accuracy of fatigue judgement, but physiological signal often needs to adopt contact type measurement, need wear the helmet all the time during such as measurement EEG signals EEG, thereby it is bigger to individual degree of dependence, a lot of limitation is arranged when being actually used in driver's fatigue detecting, even checkout equipment can cause interference to driving, therefore the application of this detection method difficulty very.
Be meant that based on driver's physiological reaction feature the eye movement characteristic of utilizing the driver, head movement characteristic etc. infer driver's fatigue state, it is to use maximum a kind of detection methods at present.For example the patent of invention of China Patent No. 200610031817.0 discloses a kind of driver fatigue monitoring system and method based on image processing and information fusion technology, its principle is as follows: at first by the continuous digital image signal of camera collection, detect people's face position, and then the position and the size of definite eyes, pupil, face; Determine that again the driver's eyes closure time accounts for the size of rule, eye gaze direction and time four fatigue characteristics that the number percent (PERCLOS) of T.T., yawning frequency, head rocks, and the four measuring value is converted to degree of fatigue; Last exploit information integration technology merges the information of four fatigue characteristics of driver, judges the current fatigue conditions of driver; When degree of fatigue surpasses setting value, alarm will start warning.This system is by merging a plurality of fatigue characteristics, thereby reaches the effect that reduces rate of false alarm and rate of failing to report.
But, this fatigue state detection method based on driver's physiological reaction feature all needs to gather digital image signal, and tend to be subjected to the bigger influence of objective environment factor when will gather digital image signal, for example night, insufficient light or strong solar light irradiation all can affect to image acquisition.And for example some driver habits are worn sunglasses, then can't collect the image information of eye, thereby cause occurring false alarm or fail to report the police.
In sum, there is the high problem of error rate in existing fatigue detecting system.
Summary of the invention
The purpose of this invention is to provide a kind of reaction type fatigue detecting system, have the high problem of error rate to solve existing fatigue detecting system.
The present invention proposes a kind of reaction type fatigue detecting system, it is characterized in that, comprises sensing module, fatigue detecting module and audio alert and feedback module.Sensing module is used to gather image optical signalling and/or the voice signal that comes from the driver.The fatigue detecting module links to each other with sensing module, is used for judging driver's fatigue state according to the image optical signalling and/or the voice signal that collect.Audio alert and feedback module, link to each other with the fatigue detecting module, be used for sending audio alert signal and interactive voice signal according to driver's fatigue state, and the relevant information of interactive voice signal fed back to the fatigue detecting module, with voice signal and/or image optical signalling, as the normative reference of judging driver's fatigue state to respond in conjunction with the driver.
According to the described reaction type fatigue detecting system of preferred embodiment of the present invention, sensing module comprises image sensing cell, is used to gather certainly in driver's image optical signalling.The fatigue detecting module comprises the fatigue detecting unit of identification maneuver, it links to each other with feedback module with image sensing cell and audio alert respectively, be used for the interactive voice information of sending according to audio alert and feedback module, and the image optical signalling of combining image sensing unit driver's action in a period of time of collecting, calculate driver's fatigue state value.
According to the described reaction type fatigue detecting system of preferred embodiment of the present invention, the fatigue detecting unit of identification response action calculates its fatigue state value by identification driver's the action of shaking the head, specifically according to following computing formula:
Wherein, x is the horizontal range between two centers of driver, and y is the vertical ranges of two centers of driver to the mouth center, and g is a characteristic quantity, and P1 is the fatigue state value based on action.
According to the described reaction type fatigue detecting system of preferred embodiment of the present invention, sensing module comprises the voice sensing unit, is used to gather the voice signal that comes from the driver.The fatigue detecting module comprises the fatigue detecting unit of computing voice energy, it links to each other with feedback module with voice sensing unit and audio alert respectively, be used for the interactive voice information of sending according to audio alert and feedback module, and the voice signal that the driver sends in a period of time that collects in conjunction with the voice sensing unit, calculate driver's fatigue state value.
According to the described reaction type fatigue detecting system of preferred embodiment of the present invention, the fatigue detecting unit of computing voice energy is specifically according to following computing formula:
Wherein, N is a current time, and M is a hits, and Xn is the amplitude of voice signal, and EN is the speech energy of driver's current time, and Eo is the energy of driver's normal voice, and P2 is the fatigue state value based on speech energy.
According to the described reaction type fatigue detecting system of preferred embodiment of the present invention, sensing module comprises the voice sensing unit, is used to gather the voice signal that comes from the driver.The fatigue detecting module comprises the fatigue detecting unit of recognizing voice content, it links to each other with feedback module with voice sensing unit and audio alert respectively, be used for the interactive voice information of sending according to audio alert and feedback module, and the content of the voice signal that sends of the driver who collects in conjunction with the voice sensing unit, calculate driver's fatigue state value.
According to the described reaction type fatigue detecting system of preferred embodiment of the present invention, sensing module comprises image sensing cell and voice sensing unit.The fatigue detecting module comprises the fatigue detecting unit of identification maneuver, the fatigue detecting unit of computing voice energy and the fatigue detecting unit of recognizing voice content, the fatigue detecting unit of identification maneuver links to each other with image sensing cell, and the fatigue detecting unit of computing voice energy links to each other with the voice sensing unit respectively with the fatigue detecting unit of recognizing voice content.Integrated unit, it links to each other with feedback module with the fatigue detecting unit of identification maneuver, the fatigue detecting unit of computing voice energy, the fatigue detecting unit and the audio alert of recognizing voice content respectively, be used for the fatigue state that each fatigue detecting unit calculates is merged, and finally calculate driver's degree of fatigue.
According to the described reaction type fatigue detecting system of preferred embodiment of the present invention, integrated unit is specifically according to following computing formula:
Wherein, Y is driver's a degree of fatigue, and n is the number of the fatigue detecting unit of participation, and the n value is 4 here, the fatigue state that Xi calculates for each fatigue detecting unit is got 1 when tired, otherwise is got 0, Ii gets 1, otherwise gets 0 for the indieating variable that enables of each fatigue detecting unit when enabling.
According to the described reaction type fatigue detecting system of preferred embodiment of the present invention, the fatigue detecting module also further comprises the fatigue detecting unit of face image, and it links to each other with image sensing cell, is used for calculating its fatigue state value according to driver's face image.
According to the described reaction type fatigue detecting system of preferred embodiment of the present invention, what the fatigue detecting unit of face image calculated is the percent of shared time of driver's eyes closed in a period of time, and it is specifically according to following computing formula:
Wherein, f is the percent of shared time of driver's eyes closed in a period of time, and t1 is that eyes open for closed 20% time fully; T2 is that eyes open for closed 80% time fully; T3 is that eyes are opened 20% the time of next time opening fully; T4 is that eyes are opened 80% the time of next time opening fully.
With respect to prior art, the invention has the beneficial effects as follows:
1, system of the present invention confirms by specific action and the such reaction type mechanism of particular problem answer affirmation, carries out fatigue identification, can judge driver's fatigue state more exactly, effectively avoids the generation of false-alarm, false dismissal situation.
2, system of the present invention is by merging the output of a plurality of fatigue detecting unit, and draws final testing result, can eliminate the flase drop that causes because of uncertain factors such as environment may, have very high accuracy.
3, the present invention fully uses media of communication this nature of voice, that the driver is disturbed minimum under driving condition, produces interaction with the driver, can reach the effect of the alleviation driving fatigue of nature.
Description of drawings
Fig. 1 is a kind of Organization Chart of reaction type fatigue detecting system of the present invention;
Fig. 2 is first kind of example structure figure of reaction type fatigue detecting system of the present invention;
Face's location synoptic diagram that Fig. 3 adopts when shaking the head action for the present invention's identification;
The characteristic quantity oscillogram that Fig. 4 adopts when shaking the head action for the present invention's identification;
Fig. 5 is second kind of example structure figure of reaction type fatigue detecting system of the present invention;
Fig. 6 is the third example structure figure of reaction type fatigue detecting system of the present invention;
Fig. 7 is the 4th kind of example structure figure of reaction type fatigue detecting system of the present invention;
Fig. 8 is the measuring principle figure of the fatigue detecting unit of face image of the present invention to the PERCLOS value;
Fig. 9 is the 5th kind of example structure figure of reaction type fatigue detecting system of the present invention;
Figure 10 is the 6th kind of example structure figure of reaction type fatigue detecting system of the present invention.
Embodiment
Main thought of the present invention is to make full use of this media of communication of voice, make system and driver produce interaction, thereby not only can effectively avoid false-alarm (false alarm, the driver is in fatigue state, system has but produced alarm) and false dismissal (driver is in fatigue state, system does not but produce alarm), improve the correct recognition rata of fatigue detecting system, and can play the effect of alleviating driving fatigue.
See also Fig. 1, it is for a kind of structural drawing of reaction type fatigue detecting system of the present invention.It comprises sensing module 100, fatigue detecting module 200 and audio alert and feedback module 300.Fatigue detecting module 200 links to each other with feedback module 300 with sensing module 100 and audio alert respectively.
Before system enters work, need carry out initialization earlier.Because the present invention image and the voice signal that are based on being collected in the driver carry out analytical calculation, judge its fatigue state, thereby need to gather in advance under the normal conditions (non-fatigue state), driver's image and voice messaging are with as with reference to standard.The eyes size of for example measuring the driver and the energy of position, normal voice signal etc.Initialized process is simple, only needs the positive face of driver facing to fatigue detecting equipment, finishes once simple dialogue and gets final product.
During work, sensing module 100 can be gathered and come from driver's image optical signalling and/or voice signal, and is transferred to fatigue detecting module 200.The information that fatigue detecting module 200 is sent according to sensing module 100 is judged driver's fatigue state, and testing result is sent to audio alert and feedback module 300.Audio alert and feedback module 300 can send audio alert signal and interactive voice signal according to driver's fatigue state.When sending the interactive voice signal, audio alert and feedback module 300 can feed back to fatigue detecting module 200 with the relevant information of interactive voice signal, as the normative reference of judging driver's fatigue state.Simultaneously, sensing module 100 can be gathered once more and come from driver's image optical signalling and/or voice signal, and is transferred to fatigue detecting module 200.The next information of that fatigue detecting module 200 can be come according to sensing module 100 transmission and audio alert and feedback module 300 feedbacks is evaluated again to driver's fatigue state, and give audio alert and feedback module 300 with result transmission, with grade and the kind that determines the audio alert that it sends.So cycle detection realizes the real-time monitoring to driver's fatigue strength.
With several representative embodiment, specify the present invention below.
Embodiment one
See also Fig. 2, this reaction type fatigue detecting system, its sensing module 100 comprises image sensing cell 101, its fatigue detecting module 200 comprises the fatigue detecting unit 201 of identification maneuver, and the fatigue detecting unit 201 of identification maneuver links to each other with feedback module 300 with image sensing cell 101 and audio alert respectively.
In the present embodiment, the input of image sensing cell 101 is facial image optics signals, and output is digitized facial image.Image sensing cell 101 can adopt near infrared camera image sensor, and under the night vision environment, this camera still can normally obtain facial image.System outputs in the fatigue detecting unit 201 of identification maneuver after collecting facial image by image sensing cell 101.
The input of the fatigue detecting unit 201 of identification maneuver has two: come from continuous people's face digital picture of image sensing cell 101, and the action command information of audio alert and feedback module 300 feedbacks.The identification of specific action is realized in the fatigue detecting unit 201 of identification maneuver, confirms whether human pilot has finished corresponding action according to voice suggestion.For example, 300 promptings of audio alert and feedback module: " please do the action of shaking the head ", and command information is fed back to the fatigue detecting unit 201 of identification maneuver.People's face video of the 201 pairs of image sensing cells in fatigue detecting unit, 101 inputs of identification maneuver is analyzed at this moment, is confirmed to be the driver and whether has finished the action of shaking the head.
Be that example specifies computation process now with the action of shaking the head.The fatigue detecting unit 201 of identification maneuver at first needs the mouth center of Fine Mapping people face and the coordinate of eye center in computation process.As shown in Figure 3, X is two horizontal ranges between the center, and Y is the vertical ranges of two centers to the mouth center, by the accurate location of human face, and recording feature amount g=X/Y.Finish in the action of shaking the head the people so, the X/Y feature can present wave form varies pattern as shown in Figure 4.
Height point expression people face in the waveform is in the positive face state of level, at this moment the character numerical value maximum; Low spot represents that people's face is in the state of side face in the waveform, this moment the character numerical value minimum.In a period of time, search characteristics maximal value gmax and minimum value gmin, calculate the fatigue state value P1 based on action then:
Determine by experience, usually, as P1 0.3 the time, can think that the driver has finished the level that the cooperates action of shaking the head, and promptly is in non-fatigue state.Otherwise then be in fatigue state, and send corresponding voice alarm by audio alert and feedback module 300.
Embodiment two
See also Fig. 5, this reaction type fatigue detecting system, its sensing module 100 comprises voice sensing unit 102, its fatigue detecting module 200 comprises the fatigue detecting unit 202 of computing voice energy, and the fatigue detecting unit 202 of computing voice energy links to each other with feedback module 300 with voice sensing unit 102 and audio alert respectively.
The design starting point of present embodiment is to attempt to constitute interactive voice with the driver, and judges whether to fail alternately whether the driver is in fatigue state.Wherein the fatigue detecting unit 202 of computing voice energy is used for judging by the speech energy that calculates the driver driver's fatigue state, its input be the voice signal that voice sensing unit 102 is gathered, and from the feedback information of audio alert and feedback module 300.
System can use level and smooth square of energy in realizing, its computing formula is:
Implication directly perceived is that current time N with preceding M signal amplitude Xn square summation constantly of current time, just can be used as a kind of sign of signal energy EN.Because driver's voice responsive duration of the expectation in the system is all shorter, in the enforcement, get and generally get 0.5 second, like this according to sample frequency, just can calculate the size of M.
Because the actual environment for use more complicated of system, thereby can use relative value to weigh driver's fatigue state.In initialization procedure, when the energy of the normal voice that measures human pilot is then to remember the fatigue state value based on speech energy
Rule of thumb, when
＜0.3 o'clock, can think the response that does not detect the driver, therefore, it is in fatigue state decidable.
What need replenish is to use here
Value is meant as criterion, and is maximum in certain observation period
Value.This observation period is that the system prompt driver sends certain voice.For example, 300 couples of drivers of audio alert and feedback module have said a joke, can investigate in a period of time after joke has been said
Perhaps 300 couples of drivers of audio alert and feedback module have carried out an enquirement, can regard the observation period as in a period of time after the enquirement.
Certainly, except adopting speech energy, also can use other index to weigh.Calculate instantaneous energy etc. after for example using wave-shape amplitude, use time-frequency conversion.
Embodiment three
See also Fig. 6, this reaction type fatigue detecting system, its sensing module 100 comprises voice sensing unit 102, its fatigue detecting module 200 comprises the fatigue detecting unit 203 of recognizing voice content, and the fatigue detecting unit 203 of recognizing voice content links to each other with feedback module 300 with voice sensing unit 102 and audio alert respectively.
The same with the fatigue detecting unit 202 of computing voice energy, the input of the fatigue detecting unit 203 of recognizing voice content also is the voice signal gathered of voice sensing unit 102 and from the feedback information of audio alert and feedback module 300.Judge whether to fail alternately whether human pilot is in fatigue state too.Different is that the formation base of present embodiment is based on the mutual of voice content.
The fatigue detecting unit 203 of recognizing voice content utilizes speech recognition technology, the voice messaging that the response of comparison system expectation and audio alert and feedback module 300 collect, if contrast both unanimities, think that then the driver is not in fatigue state, otherwise then be in fatigue state.
For example, audio alert and feedback module 300 provide an enquirement: " numeral 5 is bigger than numeral 2, is it right? " when human pilot answer " to ", the fatigue detecting unit 203 of recognizing voice content just thinks that it is not in fatigue state, otherwise, think that then it is in fatigue state; Similar problem, " area code in Shanghai is 021, is it right? in addition ", " build of elephant is bigger than tiger, is it right? ", " than 5 big what are than 7 little numerals? " Deng.
Consider that the system applies environment may be comparatively noisy, that in the system implementation interaction content is designed is comparatively simple.Include only: " to ", " mistake " and " 0 to 9 numeral ".Wherein, " to " and " mistake " also designed synonym, " yes ", " not to ", " quite right " or the like.
Embodiment four
See also Fig. 7, this reaction type fatigue detecting system, its sensing module 100 comprises image sensing cell 101.Its fatigue detecting module 200 comprises the fatigue detecting unit 201 of identification maneuver, the fatigue detecting unit 204 and the integrated unit 205 of face image.The fatigue detecting unit 201 of identification maneuver and the fatigue detecting unit 204 of face image link to each other with image sensing cell 101 and integrated unit 205 respectively, integrated unit 205 links to each other with feedback module 300 with audio alert, and audio alert and feedback module 300 are reversely connected to the fatigue detecting unit 201 of identification maneuver.
During work, image sensing cell 101 can be transferred to the fatigue detecting unit 201 of identification maneuver and the fatigue detecting unit 204 of face image respectively with the image optical signalling that collects, calculate driver's fatigue state respectively by the fatigue detecting unit 201 of identification maneuver and the fatigue detecting unit 204 of face image, and together input to integrated unit 205.The fatigue state value that integrated unit 205 can calculate both merges, and audio alert and feedback module 300 are exported in final fatigue identification.Audio alert and feedback module 300 send corresponding audio alert or interactive voice signal according to the tired identifying information that integrated unit 205 sends, and simultaneously the interactive voice signal feedback are given the fatigue detecting unit 201 of identification maneuver.
In the present embodiment, what the fatigue detecting unit 204 of face image was imported is digitized facial image, and output is the fatigue state value.It realizes the detection to the PERCLOS value of facial image.Its calculating principle is by the accurate location of human face, to calculate shared ratio of eyes closed time in a certain period, in order to weigh people's degree of fatigue.Fig. 8 is the measuring principle figure of the 204 pairs of PERCLOS values in fatigue detecting unit of face image of the present invention.Curve is eyes closed and open the degree of opening in process curve over time among the figure, can obtain certain degree of eyes closed of required measurement or open the lasting time according to this curve, thereby calculate the PERCLOS value.T1 is that eyes open for closed 20% time fully among the figure; T2 is that eyes open for closed 80% time fully; T3 is that eyes are opened 20% the time of next time opening fully; T4 is that eyes are opened 80% the time of next time opening fully.By measuring t1 just can calculate PERCLOS to the value of t4 value:
In the formula, f is the percent of shared a certain special time of eyes closed time.It is generally acknowledged, as PERCLOS value f 0.15 the time, think that the driver is in fatigue state.
In the present embodiment, system does not directly use the criterion of f value as degree of fatigue, also in conjunction with the output of the fatigue detecting unit 201 of identification maneuver, judges jointly whether the driver is in fatigue state.
Embodiment five
See also Fig. 9, this reaction type fatigue detecting system, its sensing module 100 comprises voice sensing unit 102.Its fatigue detecting module 200 comprises the fatigue detecting unit 202 of computing voice energy, the fatigue detecting unit 203 and the integrated unit 205 of recognizing voice content.The fatigue detecting unit 202 of computing voice energy and the fatigue detecting unit 203 of recognizing voice content link to each other with voice sensing unit 102 and integrated unit 205 respectively, integrated unit 205 links to each other with feedback module 300 with audio alert, and audio alert and feedback module 300 are reversely connected to the fatigue detecting unit 202 of computing voice energy and the fatigue detecting unit 203 of recognizing voice content.
Present embodiment is that embodiment two is combined application with embodiment three, voice sensing unit 102 can be transferred to the fatigue detecting unit 202 of computing voice energy and the fatigue detecting unit 203 of recognizing voice content respectively with the voice signal that collects, calculate driver's fatigue state respectively by the fatigue detecting unit 202 of computing voice energy and the fatigue detecting unit 203 of recognizing voice content, and together input to integrated unit 205.The fatigue state value that integrated unit 205 can calculate both merges, and audio alert and feedback module 300 are exported in final fatigue identification.The tired identifying information that audio alert and feedback module 300 send according to integrated unit 205, send corresponding audio alert or interactive voice signal, give the fatigue detecting unit 202 of computing voice energy and the fatigue detecting unit 203 of recognizing voice content with the interactive voice signal feedback simultaneously.
Embodiment six
Present embodiment is that embodiment four and embodiment five combinations are used.See also Figure 10, the sensing module 100 of this system comprises image sensing cell 101 and voice sensing unit 102.Fatigue detecting module 200 comprises the fatigue detecting unit 201 of identification maneuver, the fatigue detecting unit 204 of face image, the fatigue detecting unit 202 of computing voice energy, the fatigue detecting unit 203 and the integrated unit 205 of recognizing voice content.Image sensing cell 101 links to each other with the fatigue detecting unit 201 of identification maneuver and the fatigue detecting unit 204 of face image respectively.Voice sensing unit 102 links to each other with the fatigue detecting unit 202 of computing voice energy and the fatigue detecting unit 203 of recognizing voice content respectively.The fatigue detecting unit 202 of the fatigue detecting unit 201 of identification maneuver, the fatigue detecting unit 204 of face image, computing voice energy and the fatigue detecting unit 203 of recognizing voice content all link to each other with integrated unit 205.Integrated unit 205 links to each other with feedback module 300 with audio alert.Audio alert and feedback module 300 are reversely connected to the fatigue detecting unit 201 of identification maneuver, the fatigue detecting unit 202 of computing voice energy and the fatigue detecting unit 203 of recognizing voice content respectively.
The system of present embodiment is divided into two identification branches, and one is image recognition, makes one to be speech recognition.Two branches can work alone, and also can work simultaneously, and two branches finally merge by integrated unit 205, and recognition result is exported to audio alert and feedback module 300 the most at last.
More than all fatigue detecting unit, the user all can initiatively close.For example, the driver is wearing the situation of sunglasses, can initiatively the fatigue detecting unit 204 of face image be closed.Four fatigue detecting unit can be exported the driver respectively and whether be in tired judgement under the condition that enables, and by voting mechanism, merging becomes final fatigue identification output.When certain fatigue detecting unit does not enable, think that this module is absent from voting.
The input of four fatigue detecting unit of note is respectively Xi, and value Xi is 1 when thinking fatigue, otherwise the Xi value is 0; Enabling indieating variable is, when value is 1, and this fatigue detecting cell enable, otherwise this fatigue detecting unit does not enable, i.e. abstention.Integrated unit 205 final output Y are:
Therefore, the span of Y is between 0 to 1.Possible numerical value is 0,0.25,0.5,0.75 and 1, have five ranks altogether.For each output Y, indieating variable
With can be used as the confidence level index.
Big more, illustrate that the fatigue detecting unit that participates in ballot is many more, vice versa.
The input of audio alert and feedback module 300 is the recognition results that merge, and audio alert can adopt different output with feedback module 300 for different fusion results.Generally speaking, it has three types output:
(1) general voice reminder.The Typical Disposition that this class is reminded is as shown in table 1:
Table 1
Audio alert and feedback module 300 also can be considered confidence level simultaneously, and when confidence level higher (the fatigue detecting element number that promptly enables is more), the multiplicity of voice output increases.This class voice reminder is unidirectional output, do not have and the driver between interactive with exchange.
(2) action command is reported to the police.Usually, when driver's degree of fatigue is judged by system 〉=0.5 the time, audio alert and feedback module 300 can provide the action command warning at random.The report to the police starting point of design of this class is that the requirement driver moves with correct response, confirm its not be in fatigue state.Otherwise system can think that it is in fatigue state.
At present, in the system implementation case, two kinds of actions of more employing, " please do the action of shaking the head " and " please do nodding action ".In principle, can also be easy to expansion and use other action command, for example, " please use hand combing hair once ".But, because the singularity of driving environment when designing this class action, reduce the interference to driving behavior itself as far as possible.
(3) voice dialogue is reported to the police.This class is reported to the police and is divided into two classes, promptly
A, the audio alert of expectation coupling answer;
B, the audio alert that still content is not had particular requirement is responded in expectation.
For the category-A audio alert, typical problem is a digital issue, the perhaps simple matter of right and wrong.Here for the example in some problem bases: " elephant is bigger than tiger, is it right? ", " area code in Shanghai is 021, is it right? ", " than 5 big what are than 7 little numerals? ", " in the time of thunderstorm, usually can lightning, is that right? "In order to control the accuracy of speech recognition well, thereby the answer of problem can be controlled at be in the not sum digital scope.
For the category-B audio alert, system checks on one's answers and no requirement (NR).Typical conversation content is to say a joke, think hard zig zag, guessing riddles etc.
Various types of voice is reported to the police enable the fatigue detecting unit and feedback signal as shown in table 2:
Table 2
As can be seen from Table 2, audio alert and feedback module 300 can adopt the form of coding to each fatigue detecting unit feedack, and generally speaking, there are following three types of feedback informations as shown in table 3 in system:
Table 3
System is after the voice alarm of sending the answer of expectation coupling, and except being non-coding (011 and 012) and the digital answer coding (020 to 029), expectation speech response (030) feedback information exists simultaneously.
System of the present invention by with driver's interactive voice, reach and detect accuracy rate more accurately, can effectively avoid the generation of false-alarm, false dismissal.More than disclosed only be several specific embodiment of the present invention, but the present invention is not limited thereto, any those skilled in the art can think variation, all should drop in protection scope of the present invention.
Claims (10)
1. a reaction type fatigue detecting system is characterized in that, comprising:
One sensing module is used to gather the image optical signalling and/or the voice signal that come from the driver;
One fatigue detecting module links to each other with this sensing module, is used for judging driver's fatigue state according to the image optical signalling and/or the voice signal that collect;
One audio alert and feedback module, link to each other with this fatigue detecting module, be used for sending audio alert signal and interactive voice signal according to driver's fatigue state, and the relevant information of interactive voice signal fed back to this fatigue detecting module, with voice signal and/or image optical signalling, as the normative reference of judging driver's fatigue state in conjunction with driver's response.
2. reaction type fatigue detecting system as claimed in claim 1 is characterized in that,
This sensing module comprises an image sensing cell, is used to gather certainly in driver's image optical signalling;
This fatigue detecting module comprises the fatigue detecting unit of an identification maneuver, it links to each other with feedback module with this image sensing cell and this audio alert respectively, be used for the interactive voice information of sending according to audio alert and feedback module, and the image optical signalling of driver's action in a period of time that collects in conjunction with this image sensing cell, calculate driver's fatigue state value.
3. reaction type fatigue detecting system as claimed in claim 2 is characterized in that, the fatigue detecting unit of this identification maneuver calculates its fatigue state value by identification driver's the action of shaking the head, specifically according to following computing formula:
Wherein, x is the horizontal range between two centers of driver, and y is the vertical ranges of two centers of driver to the mouth center, and g is a characteristic quantity, and P1 is the fatigue state value based on action.
4. reaction type fatigue detecting system as claimed in claim 1 is characterized in that,
This sensing module comprises a voice sensing unit, is used to gather the voice signal that comes from the driver;
This fatigue detecting module comprises the fatigue detecting unit of a computing voice energy, it links to each other with feedback module with this voice sensing unit and this audio alert respectively, be used for the interactive voice information of sending according to this audio alert and feedback module, and the voice signal that the driver sends in a period of time that collects in conjunction with this voice sensing unit, calculate driver's fatigue state value.
5. reaction type fatigue detecting system as claimed in claim 4 is characterized in that, the fatigue detecting unit of this computing voice energy is specifically according to following computing formula:
Wherein, N is a current time, and M is a hits, and xn is the amplitude of voice signal, and EN is the speech energy of driver's current time, and Eo is the energy of driver's normal voice, and P2 is the fatigue state value based on speech energy.
6. reaction type fatigue detecting system as claimed in claim 1 is characterized in that,
This sensing module comprises a voice sensing unit, is used to gather the voice signal that comes from the driver;
This fatigue detecting module comprises the fatigue detecting unit of a recognizing voice content, it links to each other with feedback module with this voice sensing unit and this audio alert respectively, be used for the interactive voice information of sending according to this audio alert and feedback module, and the content of the voice signal that sends of the driver who collects in conjunction with this voice sensing unit, calculate driver's fatigue state value.
7. reaction type fatigue detecting system as claimed in claim 1 is characterized in that,
This sensing module comprises an image sensing cell and a voice sensing unit;
This fatigue detecting module comprises the fatigue detecting unit of an identification maneuver, the fatigue detecting unit of a computing voice energy, the fatigue detecting unit and an integrated unit of a recognizing voice content, the fatigue detecting unit of this identification maneuver links to each other with this image sensing cell, and the fatigue detecting unit of this computing voice energy all links to each other with this voice sensing unit with the fatigue detecting unit of this recognizing voice content;
This integrated unit, it links to each other with feedback module with the fatigue detecting unit of this identification maneuver, the fatigue detecting unit of this computing voice energy, fatigue detecting unit and this audio alert of this recognizing voice content respectively, be used for the fatigue state that each fatigue detecting unit calculates is merged, and finally calculate driver's degree of fatigue.
8. reaction type fatigue detecting system as claimed in claim 7 is characterized in that, this integrated unit is specifically according to following computing formula:
，
Wherein, Y is driver's a degree of fatigue, and n is the number of the fatigue detecting unit of participation, the fatigue state that Xi calculates for each fatigue detecting unit is got 1 when tired, otherwise is got 0, Ii gets 1, otherwise gets 0 for the indieating variable that enables of each fatigue detecting unit when enabling.
9. as claim 2,3,7,8 each described reaction type fatigue detecting system, it is characterized in that,
This fatigue detecting module also further comprises the fatigue detecting unit of a face image, and it links to each other with this image sensing cell, is used for calculating its fatigue state value according to driver's face image.
10. as each described reaction type fatigue detecting system of claim 9, it is characterized in that what the fatigue detecting unit of this face image calculated is the percent of shared time of driver's eyes closed in a period of time, it is specifically according to following computing formula:
Wherein, f is the percent of shared time of driver's eyes closed in a period of time, and t1 is that eyes open for closed 20% time fully; T2 is that eyes open for closed 80% time fully; T3 is that eyes are opened 20% the time of next time opening fully; T4 is that eyes are opened 80% the time of next time opening fully.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010105267839A CN101968918B (en) | 2010-11-01 | 2010-11-01 | Feedback type fatigue detecting system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010105267839A CN101968918B (en) | 2010-11-01 | 2010-11-01 | Feedback type fatigue detecting system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101968918A true CN101968918A (en) | 2011-02-09 |
| CN101968918B CN101968918B (en) | 2012-05-23 |
Family
ID=43548065
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010105267839A Expired - Fee Related CN101968918B (en) | 2010-11-01 | 2010-11-01 | Feedback type fatigue detecting system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101968918B (en) |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102868951A (en) * | 2012-09-03 | 2013-01-09 | 陈火 | Sensing type early-warning earphone reminder |
| CN103009669A (en) * | 2012-12-11 | 2013-04-03 | 金丰（中国）机械工业有限公司 | Punch operator fatigue judgement system and punch operator fatigue judgement method |
| CN103679065A (en) * | 2012-09-15 | 2014-03-26 | Abb技术股份公司 | Safety device for a technical installation or a technical process |
| CN104097642A (en) * | 2013-04-09 | 2014-10-15 | 福特全球技术公司 | Method and system for controlling a vehicle during an autonomous control mode |
| CN104828095A (en) * | 2014-09-02 | 2015-08-12 | 北汽福田汽车股份有限公司 | Method, device and system of detecting driving status of driver |
| CN104952209A (en) * | 2015-04-30 | 2015-09-30 | 广州视声光电有限公司 | Driving prewarning method and device |
| CN105303770A (en) * | 2014-06-03 | 2016-02-03 | 现代摩比斯株式会社 | Sleepy warning device with wearable display for driver and method |
| CN106062869A (en) * | 2014-03-25 | 2016-10-26 | 夏普株式会社 | Interactive household-electrical-appliance system, server device, interactive household electrical appliance, method whereby household-electrical-appliance system performs interaction, and non-volatile computer-readable data recording medium having, stored thereon, program for executing said method on computer |
| CN106657648A (en) * | 2016-12-28 | 2017-05-10 | 上海斐讯数据通信技术有限公司 | Mobile terminal for preventing fatigue driving and realization method thereof |
| CN107038433A (en) * | 2017-05-15 | 2017-08-11 | 柳州译海网络科技有限公司 | A kind of fingerprint speech recognition weariness working warning device |
| CN107103294A (en) * | 2017-04-20 | 2017-08-29 | 上海耐相智能科技有限公司 | A kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone |
| WO2018014598A1 (en) * | 2016-07-18 | 2018-01-25 | 中兴通讯股份有限公司 | Method and terminal for alerting fatigued driver |
| CN108099915A (en) * | 2017-12-25 | 2018-06-01 | 芜湖皖江知识产权运营中心有限公司 | A kind of fatigue driving control system for identifying applied in intelligent vehicle |
| CN108099816A (en) * | 2017-12-25 | 2018-06-01 | 芜湖皖江知识产权运营中心有限公司 | A kind of vehicle audio input control method for intelligent travel |
| CN108162755A (en) * | 2017-12-25 | 2018-06-15 | 芜湖皖江知识产权运营中心有限公司 | A kind of fatigue driving identification control method applied in intelligent vehicle |
| CN108831460A (en) * | 2018-06-15 | 2018-11-16 | 浙江吉利控股集团有限公司 | A kind of interactive voice control system and method based on fatigue monitoring |
| CN110303884A (en) * | 2019-07-10 | 2019-10-08 | 初速度（苏州）科技有限公司 | A kind of anti-fatigue-driving method, system and device |
| CN111243233A (en) * | 2018-11-28 | 2020-06-05 | 现代自动车株式会社 | Apparatus, system and method for managing fatigue driving |
| CN111354162A (en) * | 2020-03-07 | 2020-06-30 | 深圳市松恩电子科技有限公司 | Monitoring system for health fatigue and prevention of driver |
| CN112396805A (en) * | 2019-07-31 | 2021-02-23 | Oppo广东移动通信有限公司 | Fatigue driving reminding method, device, terminal and storage medium |
| CN112434563A (en) * | 2020-11-04 | 2021-03-02 | 芜湖职业技术学院 | Driving fatigue relieving system and method |
| CN116092267A (en) * | 2023-01-29 | 2023-05-09 | 浙江吉利控股集团有限公司 | Vehicle fatigue driving reminding method, device, equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH11306462A (en) * | 1998-04-24 | 1999-11-05 | Sony Corp | State monitoring device |
| US6313749B1 (en) * | 1997-01-04 | 2001-11-06 | James Anthony Horne | Sleepiness detection for vehicle driver or machine operator |
| DE10039551A1 (en) * | 2000-08-09 | 2002-02-21 | Vinzenz Von Holle | Sleep prevention device for automobile driver provides acoustic signal within seconds of driver falling asleep |
| US20080231461A1 (en) * | 2007-03-20 | 2008-09-25 | Julian Sanchez | Method and system for maintaining operator alertness |
-
2010
- 2010-11-01 CN CN2010105267839A patent/CN101968918B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6313749B1 (en) * | 1997-01-04 | 2001-11-06 | James Anthony Horne | Sleepiness detection for vehicle driver or machine operator |
| JPH11306462A (en) * | 1998-04-24 | 1999-11-05 | Sony Corp | State monitoring device |
| DE10039551A1 (en) * | 2000-08-09 | 2002-02-21 | Vinzenz Von Holle | Sleep prevention device for automobile driver provides acoustic signal within seconds of driver falling asleep |
| US20080231461A1 (en) * | 2007-03-20 | 2008-09-25 | Julian Sanchez | Method and system for maintaining operator alertness |
Cited By (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102868951A (en) * | 2012-09-03 | 2013-01-09 | 陈火 | Sensing type early-warning earphone reminder |
| CN103679065A (en) * | 2012-09-15 | 2014-03-26 | Abb技术股份公司 | Safety device for a technical installation or a technical process |
| CN103679065B (en) * | 2012-09-15 | 2018-11-23 | Abb瑞士股份有限公司 | Safety device for technical equipment or technical process |
| CN103009669A (en) * | 2012-12-11 | 2013-04-03 | 金丰（中国）机械工业有限公司 | Punch operator fatigue judgement system and punch operator fatigue judgement method |
| CN103009669B (en) * | 2012-12-11 | 2014-12-17 | 金丰（中国）机械工业有限公司 | Punch operator fatigue judgement system and punch operator fatigue judgement method |
| CN104097642B (en) * | 2013-04-09 | 2018-10-26 | 福特全球技术公司 | Method and controller for controlling the vehicle with the control of interim autonomous vehicle |
| CN104097642A (en) * | 2013-04-09 | 2014-10-15 | 福特全球技术公司 | Method and system for controlling a vehicle during an autonomous control mode |
| CN106062869A (en) * | 2014-03-25 | 2016-10-26 | 夏普株式会社 | Interactive household-electrical-appliance system, server device, interactive household electrical appliance, method whereby household-electrical-appliance system performs interaction, and non-volatile computer-readable data recording medium having, stored thereon, program for executing said method on computer |
| CN106062869B (en) * | 2014-03-25 | 2020-03-27 | 夏普株式会社 | Interactive household appliance system, household appliance equipment, interaction method thereof and server device |
| CN105303770A (en) * | 2014-06-03 | 2016-02-03 | 现代摩比斯株式会社 | Sleepy warning device with wearable display for driver and method |
| CN104828095A (en) * | 2014-09-02 | 2015-08-12 | 北汽福田汽车股份有限公司 | Method, device and system of detecting driving status of driver |
| CN104828095B (en) * | 2014-09-02 | 2018-06-19 | 北京宝沃汽车有限公司 | Detect the method, apparatus and system of driver's driving condition |
| CN104952209A (en) * | 2015-04-30 | 2015-09-30 | 广州视声光电有限公司 | Driving prewarning method and device |
| WO2018014598A1 (en) * | 2016-07-18 | 2018-01-25 | 中兴通讯股份有限公司 | Method and terminal for alerting fatigued driver |
| CN107633657A (en) * | 2016-07-18 | 2018-01-26 | 中兴通讯股份有限公司 | The based reminding method and terminal of a kind of fatigue driving |
| CN106657648A (en) * | 2016-12-28 | 2017-05-10 | 上海斐讯数据通信技术有限公司 | Mobile terminal for preventing fatigue driving and realization method thereof |
| CN107103294A (en) * | 2017-04-20 | 2017-08-29 | 上海耐相智能科技有限公司 | A kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone |
| CN107038433A (en) * | 2017-05-15 | 2017-08-11 | 柳州译海网络科技有限公司 | A kind of fingerprint speech recognition weariness working warning device |
| CN108099915A (en) * | 2017-12-25 | 2018-06-01 | 芜湖皖江知识产权运营中心有限公司 | A kind of fatigue driving control system for identifying applied in intelligent vehicle |
| CN108162755A (en) * | 2017-12-25 | 2018-06-15 | 芜湖皖江知识产权运营中心有限公司 | A kind of fatigue driving identification control method applied in intelligent vehicle |
| CN108099816A (en) * | 2017-12-25 | 2018-06-01 | 芜湖皖江知识产权运营中心有限公司 | A kind of vehicle audio input control method for intelligent travel |
| CN108831460A (en) * | 2018-06-15 | 2018-11-16 | 浙江吉利控股集团有限公司 | A kind of interactive voice control system and method based on fatigue monitoring |
| CN111243233A (en) * | 2018-11-28 | 2020-06-05 | 现代自动车株式会社 | Apparatus, system and method for managing fatigue driving |
| CN110303884A (en) * | 2019-07-10 | 2019-10-08 | 初速度（苏州）科技有限公司 | A kind of anti-fatigue-driving method, system and device |
| CN110303884B (en) * | 2019-07-10 | 2020-11-06 | 初速度（苏州）科技有限公司 | Fatigue driving prevention method, system and device |
| CN112396805A (en) * | 2019-07-31 | 2021-02-23 | Oppo广东移动通信有限公司 | Fatigue driving reminding method, device, terminal and storage medium |
| CN112396805B (en) * | 2019-07-31 | 2022-10-25 | Oppo广东移动通信有限公司 | Fatigue driving reminding method, device, terminal and storage medium |
| CN111354162A (en) * | 2020-03-07 | 2020-06-30 | 深圳市松恩电子科技有限公司 | Monitoring system for health fatigue and prevention of driver |
| CN112434563A (en) * | 2020-11-04 | 2021-03-02 | 芜湖职业技术学院 | Driving fatigue relieving system and method |
| CN116092267A (en) * | 2023-01-29 | 2023-05-09 | 浙江吉利控股集团有限公司 | Vehicle fatigue driving reminding method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101968918B (en) | 2012-05-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101968918B (en) | Feedback type fatigue detecting system | |
| CN108052888B (en) | A kind of driver's replacement system | |
| Dong et al. | Driver inattention monitoring system for intelligent vehicles: A review | |
| CN106956591B (en) | A kind of system driving permission for judging driver | |
| CN104146722B (en) | Driving fatigue detecting and grading early warning method based on head signals | |
| CN101583313B (en) | Awake state judging model making device, awake state judging device, and warning device | |
| CN205041401U (en) | Equipment is worn detection device and is had and wears guardianship device that detects function | |
| CN105615878B (en) | A kind of fatigue driving eeg monitoring method | |
| CN113743471B (en) | Driving evaluation method and system | |
| CN104183091A (en) | System for adjusting sensitivity of fatigue driving early warning system in self-adaptive mode | |
| CN104757981A (en) | Method and device for high-sensitively receiving and transmitting integrated infrared detection of driver's fatigue | |
| CN106691443A (en) | Electroencephalogram-based wearable anti-fatigue intelligent monitoring and pre-warning system for driver | |
| CN107031653A (en) | A kind of driver identity authoring system for being used to share automobile | |
| CN101877051A (en) | Driver attention state monitoring method and device | |
| CN105869356A (en) | Railway driver work state evaluation intervening system and method | |
| CN104361716A (en) | Method for detecting and reminding fatigue in real time | |
| CN103198618A (en) | Mobile phone capable of preventing fatigue driving and warning method | |
| CN109875583A (en) | A kind of fatigue driving detecting system and method based on AR technology | |
| Alam et al. | Active vision-based attention monitoring system for non-distracted driving | |
| Kumar et al. | Detecting distraction in drivers using electroencephalogram (EEG) signals | |
| CN117475413A (en) | Fatigue driving detection method and system based on image recognition and BP neural network | |
| CN112168190B (en) | Real-time driving pressure monitoring system and method | |
| Deshmukh et al. | Driver fatigue detection using sensor network | |
| Li et al. | A driving attention detection method based on head pose | |
| Zhang et al. | Low-power listen based driver drowsiness detection system using smartwatch |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120523 Termination date: 20131101 |