WO2006000166A1 - Method and device for detecting operator fatigue or quality - Google Patents

Abstract

The method of operator fatigue detection from operator muscular activity is performed by detecting at least one parameter affected by operator muscular activity, which parameter is assessed using fatigue assessment rules obtained using a data mining method from a corresponding parameter of at least one operator for whom the extent of fatigue is known. The device to perform the fatigue detection method comprises a programmable unit with a pre­programmed fatigue detection model - that is to say fatigue assessment rules obtained by measuring a signal of at least one of parameters generated by operator muscular activity for at least one fatigued operator and at least one alert operator -, a detector or detectors to measure or measure and process the signal of the parameter or parameters used by the model, connected to the input of the programmable unit to assess operator fatigue, while the programmable unit output is connected to a fatigue signaling device.

Description

Method and Device for Detecting Operator Fatigue or Quality

Field of Invention

The invention deals with a method of detecting person's inability to perform an activity in which the person's vigilance and/or thoroughness and/or specific skills and/or continuous assessing and solving situations occurred are required. A vehicle driver, plant operator or similar operators can be mentioned as examples of those persons. In addition, the invention comprises a device, which employs the above-mentioned method, to detect a person's ability or inability to perform the above described activities. Naturally, the device can be used to detect an ability of other persons - such as guard service personnel or process control personnel - as well. The subject matter of the invention is particularly suitable for use in the automotive industry where it can prevent car accidents arisen from an operator micro sleep or delayed response caused by, for example, fatigue, alcohol, drugs, narcotics, etc. This invention can be used for a detection of an operator's ability to perform certain activities as well.

State of the Art

At present, there are frequently solved problems dealing with quality of operators and fatigue of operators performing tasks that require them to be capable of responding instantly and properly. Among different types of tasks and operators, the fatigue of drivers is one of the most frequently solved problems. Due to fatigue, a large number of serious traffic accidents occur throughout the world everyday. Those accidents often result in the most grievous consequences - death of the driver or another person. According to different authors, proportion of the car accidents caused by fatigue is very high. Many experts say that fatigue could be a major factor in over 50% of all road accidents; see http://www.driverfatigue.50megs.com/. Early studies evidently underestimated the actual influence of fatigue. Later sophisticated analysis proved large direct impact of fatigue on car accidents. For example, the early studies by Knipling et al. {Knipling, R.R. and W. W. Wierwille, (eds.) Vehicle-Based Drowsy Driver Detection: Current Status and Future Prospects, published in Proceeding of IVHS America Fourth Annual Meeting, Atlanta, Georgia, in April 17 — 20, 1994 and Knipling, R.R. and J.S. Wang, Revised Estimates of the U.S. Drowsy Driver Crash Problem Size Based on General Estimates System Case Reviews" presented at the 39th Annual Conference of "Association for the Advancement of Automotive Medicine", Chicago, 1995) include significant underestimation of percentages of accidents caused by fatigue. According to the above mentioned authors, fatigue causes only from 1.2 to 1.6% of the car accidents in the USA and 3.2% of grievous (fatal) accidents. According to a detailed analysis drawn up by Home and Reyner (Home, J. and L. Reyner, Vehicle Accidents Related to Sleep: A Review., published in Occupational and Environmental Medicine inl999. Vol. 56 No. 5: pages 289-294, in 1999), these numbers are going to increase to 10-25%. Lynne of NHTSA (Lynne, L., Wake-up Call Aimed at Drowsy Drivers: Message is 'Drive Alert, Arrive Alive', published in the JAMA journal, Vol. 276, No. 15, page 1209) reports that the driver falling asleep during driving causes 20% of the passenger car accidents and 14% of truck accidents. Besides driver fatigue, there are frequently solved problems dealing with fatigue detection of other persons, who can be together referred to as "operators". Hereinafter, the term "operator" or "operators" is understood to include operator or operators of different plants, drivers, pilots and any similar persons who are required to be alert, persistent, attentive and vigil in performing their activities. Fatigue substantially affects the operator ability to instantly and properly respond to a change and various situations occurred. The operator ability to perform a given activity is affected by many factors. Among those factors, the operator quality given, for example, by his/her experience, long-term skills, and permanent physical and psychical capabilities is of a great importance. An experienced operator will respond to a given situation in a manner different from that of an inexperienced operator responding to the same situation. The ability to instantly and properly respond is affected adversely by operator fatigue in any event. The problems of fatigue have already been an intensively studied field for some decades. However, there has not been any mass-manufactured device suitable for detecting operator fatigue yet. Academic and applied research on fatigue comprises the following areas: a) Detection of physiological characteristics of fatigue using different devices. For example, the following characteristics are being monitored: brain function characteristics are measured using EEG, heart function is measured using ECG, eye blinks are measured using EOG, response time is monitored, skin resistance is measured, eye movements and face expression are analysed and other characteristics and parameters are measured. EEG is among the most studied physiologic fatigue indicators. The use of EEG to detect fatigue is reported by, for example, LaI, S.K.L. and A.Craig : Electroencephalography Activity Associated with Driver Fatigue- Implications for a Fatigue Countermeasure Device, Journal of Psychophysiolhy 2001, LaI, S.K.L. et al. Development of an algorithm for an EEG-based driver fatigue countermeasure- Journal of Safety Research, 2003. In addition, patents US 6,625,485 and US 6,575,902 deal with a fatigue detection using EEG. Another possibility consists in measuring ECG, which provides a simple and transparently measurable signal and is used in many studies. However, the ECG signal is not suitable for field applications. At present, eye monitoring seems to be the most studied and promising manner of measurement. Within the monitoring, the following are studied: problems of eye pupil size as well as eye blinking and movement characteristics. One of fatigue indicators is PERCLOS, described, for example, in studies by Dinges, D.F., et al. „ Evaluation of Techniques for Ocular Measurement as an Index of Fatigue as the Basis for Alertness Measurement", published by NHTSA in 1998, pages 1-113, and Knipling, R. R. and P. Rau, PERCLOS: ,,A Valid Psychophysiological Measure of Alertness as Assessed by Psychomotor Vigilance", published by FHWA in 1998, pages 1-4. The PERCLOS is defined as a proportion of the time when the eye is 80-% shut. The technique of detecting fatigue using that indicator is described, for example, in studies by Tijerina, L., et al, A Preliminary Assesment of Algorithms for Drowsy and Inattentive Driver Detection on the Road, 1999, US DOTNHTSA. page 42, and Hamlin, R.P., Three-in-one Vehicle Operator Senior. 1995, Northrop Grumman Corp., page 13. However, each of those methods of detecting requires sensors to be attached to the operator or installed in his/her vicinity. Presence of the sensors in operator's vicinity can cause inconvenience for the operator in performing his/her task. Moreover, the sensors must be, in most cases, placed in an accurate position; failings that frequently happens results in a decrease in the detection accuracy. Therefore, those instruments are not suitable for a large-scale detection of person fatigue. Some attempts have been done to eliminate operator's inconvenience. Those which monitor the operator's eyes in particular are described, for example, in studies by Eriksson, M. and NP. Papanikolopoulos ,βye-Tracking for Detection of Driver Fatigue", published in Proc. of the IEEE Conference on Intelligent Transportation Systems in 1997, and Eriksson, M. and NP. Papanikolopoulos ..Driver Fatigue: a Vision-Based Approach to Automatic Diagnosis", published in Transportation Research Part C: Emerging Technologies, 2001, Vol. 9 No. 6, pages 399-413. These studies try to eliminate the operator's inconvenience by employing a contactless device. This device most frequently consists of a miniature video camera. However, this device requires the driver, for example, to be seated at a particular height or to have his/her head in a particular position and includes a very difficult image analysis process to enable the eye movement to be monitored in a 3D space, hi addition, those devices are very expensive and their reliability may be insufficient, due to variable environmental conditions such as a change in intensity of the daylight or evening light. b) Monitoring of environmental factors such as ambient temperature, noise, vibrations and the environment inside the vehicle cabin. However, those indicators do not provide enough reliable information to be mass used, for example, in transportation. c) Monitoring of the workload related to shiftwork, labor time, overtime work, etc. There are many studies dealing with a sleep deprivation, particularly, of shift workers; these are, for example, Rhyanes, L.; Mάrquez, M., Diaz, C, Monarez, D. 1999: ,,Sleep and the Effect of Shift Workers ", New Mexico High School, page 7, Ahsberg, E., 1998, ..Perceived Fatigue Related to Work"., Department of Psychology, University of Stockholm, page 32, and Philip, P. et ah, 2002, „ Work and Rest Sleep Schedules of 227 European Truck Drivers", Sleep Medicine, Vol. 3, No. 6, pp. 507-511. However, that area of research is not suitable for, for example, a detection of vehicle operator fatigue. It is largely a statistical research on fatigue effects or a fatigue research to determine an organization policy, service schedule, etc. d) Detection of operator or vehicle movement characteristics (such as steering wheel movements, vehicle position in relation to the road, etc. This area also includes a detection of fatigue due to dynamic qualities of the operator. Fatigue symptoms prove themselves in man's abilities to control or adjust the device, system or plant in question. It is common knowledge that a tired operator responds to stimuli in a manner different from that of an alert operator. The difference between the responses of an alert and fatigued operator can be found in various areas: 1) The movement dynamics itself. That is caused by a reduced innervations of neuromuscular discs. It can be demonstrated on eye muscles. 2) Delayed response time. That is common knowledge, that delayed response time is related to transmitting a signal, processing the signal in CNS and response to it. It means that a transmission delay occurs between the stimulus and the response to it in the eye-hand or eye-leg system. Transmission delay increases with increased fatigue. Transmission delay depends, among other factors, on the length of the nerve impulse transmission; that means that the delay of the eye-hand system is lower than that of the eye-leg system. The lowest transmission delay arises from the movement of the eye itself (for example, in watching an object or responding to stimuli). 3) A slower assessment of situations, errors in making decisions, errors in finding the right solution. This area is generally known from various studies. The fatigue effect on assessment could be investigated using designated tests (such as Letter Cancellation Test). 4) Micro sleep and processes of initial stages of sleep. The operator tries to make his/her work easier and exert his/her brain as low as possible. That results in a higher tolerance to control errors. If a vehicle control is in question, the problem particularly consists in keeping the vehicle on the road. The operator fatigue may result in driving along the roadside (road verge) or, at the worst, in leaving the road or driving in the opposite direction. As regards steering wheel movements, that situation is represented by an absence of typical control movements of low amplitude and an occurrence of short compensatory movements of a high velocity (that means that the driver is "waking up" and aligning the vehicle position). In this situation, the muscular tone is similar to that of a sleep state. 5) Alertness active maintaining. The operator performs useless movements (for example, the operator shuffles in his/her seat to prevent himself/herself from falling asleep). If fatigue is present, each of these 5 mechanisms affects the manner of adjustment - control - to a different extent. Besides the above-mentioned Item 3, which is associated with the cognitive component of control, the other items produce a response in vehicle movement dynamics. Therefore, the approach in which a man is regarded as a controller is well-founded in researching man's control qualities. In addition, the research of control qualities of a man - operator - has had a long-term tradition; one of the first publications to deal with the effect of fatigue in operator movements is the work „ Psychological Aspects of Stick and Rudder Controls in Aircraft" by J. Orlansky, published in Aeronautical Engineering Review in January 1949,pages22-31 . In common situations, the driver behaves (by compensating the road surface bumpiness and irregularities, keeping the speed and movement trajectory within the limits required and responding to traffic situations) like a controller. The vehicle movement control is carried out through a movement of the driver's upper and lower extremities. Therefore, the central fatigue symptoms are transferred to the vehicle through intentional and unintentional movements of the driver. The effect of fatigue in the transfer to the system or plant controlled is a general quality of man - operator - and applies to any activity based on tracking down or compensation task. Control of a complicated mechanism (such as a vehicle) can be divided into three hierarchic levels (refer to Rasmussen, J.: „ Skills, Rules and Knowledge, Signals, Signs and Symbols and other Distinctions in Human Performance models ", IEEE trans. SMC, Vol. 13, No. 2, pp. 257-266, 1983): 1. Executive level, at which the learned stereotypes, executed by the operator more or less mechanically, are used in particular, hi case of driving a vehicle, it is actually considered to be the basic activity of the driver, who tries to keep the vehicle aligned with the axis of the appropriate traffic lane. It is a classic feedback control. 2. Co-ordination level - control using rules. A rather higher hierarchic level, which modifies the values required for the feedback loops at the lowest executive level. The function of the rules can be represented by traffic rules. Those rules must cover the entire problem area. The rules make it possible for the driver to make an unambiguous decision on a further control strategy in any situation. 3. Organizational level, which corresponds to a knowledge-based control. The operator is expected to move in a problem area, in which he/she has incomplete information, and/or some situations may occur in which the rules get into a conflict. In those events, when the selection of a suitable strategy is based on a multi-criterional decision in conditions of uncertainty, knowledge of a wide context is necessary. The patents and patent applications of the state of art can be divided into following categories: a) The fatigue detection is based entirely on muscular activity or is based on muscular activity as one of measured physiological parameters. For example document US 6,547,728 with the title "Device for measuring organism condition" deals with monitoring of non-sensory biological activity and is related mainly to circadian activity monitoring. No usable evaluation of fatigue is presented; the simple diagnostic is intended to general public as simple device for crude guess of tiredness as component of circadian activity. Document US 6,497,658 with the title "Alarm upon detection of impending sleep state" uses monitoring of EMG and other physiological processes. This method is not robust as the bispectral index is used for fatigue estimation. The fatigue detection process is disturbing for the person, therefore invention is not usable for monitoring operator fatigue during his / her routine activity. The fatigue detection is performed trough muscular activity indirectly; the muscular activity is not explicitly mentioned. The fatigue is detected from movement of the vehicle or its parts; the measured parameter is indirectly influenced by muscular activity of the driver. Here belong patents dealing with steering wheel monitoring for fatigue assessment, sleepiness or other indispositions, vehicle speed an acceleration monitoring, etc. The main shortcomings of all of these patents are lack of suitable methods for extraction of information on fatigue from the measured signal. For fatigue assessment are therefore used simple criteria. For example Steering Attention Monitor produced by Electronic Safety Products, Inc. which use corrective movements monitoring. Principles of fatigue detection in bellow mentioned patents can be characterised by lack of generalisation capabilities and sensitivity to environmental factors, therefore are commercially unusable. For example document US 6,756,903 with the title "Driver alertness monitoring system" is dealing with fatigue estimation based on distinguishing between driver initiated movements and non driver initiated movements, which are caused by machine itself or by its interactions. In spite of the fact that the description of the device is well done, the practical applicability of the invention is questionable. The fatigue index is calculated as proportion between driver initiated movements and non driver initiated movements, this does not correspond to reality well. Also the movements of the vehicle or its parts as steering wheel can not be simply classified as driver initiated or not. Document US 6,424,265 with the title "Magnetic steering wheel movement sensing device" is dealing mainly with the mechanical construction of the device. The fatigue estimation is performed by using simple rules applied to measured signal. Another document US 5,900,819 with the title "Drowsy driver detection system" is dealing with measuring the movement of the axle, as lateral axle acceleration, fore-aft acceleration, vehicle speed etc. for fatigue estimation. The invention uses simple ad-hoc rules for fatigue estimation. Document US 5,798,695 with the title "Impaired operator detection and warning system employing analysis of operator control actions" is based on tracking task principle and power spectrum array (PSA) analysis of sine waves. The parameters as vehicle's lateral acceleration or jitter are used as indicators for sleepiness onset detection. The disadvantage of the solution based on a few described parameters which are used for fatigue detection is lace of robust for credible fatigue evaluation in changing ambient environment. Document US 5,745,031 with the title "Safety driving system" uses steering wheel monitoring, position of vehicle monitoring and road monitoring system for fatigue detection. The fuzzy inference system is used for fatigue assessment. The invention is based on simple assumptions about driver behaviour. No advanced mathematical or signal processing techniques are used. The invention is not suitable for the practical use. c) Fatigue detection from movements not connected to driving. Here belong the fatigue detection methods based on unintentional movements and shuffling. As examples we can mention following inventions: US 5,835,008 with the title "Driver, vehicle and traffic information system." using various sensors for monitoring position of driver body foods and only a simple evaluation of driver's intentions. According to the document US 6,392,550 "Method and apparatus for monitoring driver alertness", the fatigue is estimated from driver's seated posture, using pressure sensors mounted in the seat. Changes in the pattern of pressure are analysed trough neural network processing technique however this invention is complicated and not very reliable. The document US 6,661,345 with the title "Alertness monitoring system" is based on using doppler sensors recording subject movements for fatigue detection. The invention is based on wrong assumption: "When people are awake, they fidget and move. When they are drowsy, this motion slows down, changes character and may stop". The opposite is true. The sleepy person tends to make unintentional movements to prevent the onset of sleepiness. d) Methods of fatigue detection based on monitoring various physiological parameters as EEG, EOG, ECG etc. The measuring of physiological signal in routine environment cause either operator discomfort or serious complications caused by environmental factor interference. In all cases, the patents based on estimation of fatigue from operator's physiological data have limited applicability. Document US 6,265,978 with the title "Method and apparatus for monitoring states of consciousness, drowsiness, distress, and performance" uses a wrist band where various sensors for blood flow, body temperature, pulse, EMG and SPR (skin potential response) are monitored. This solution is obviously uncomfortable for the tested person. Another inventions US20040044293A1 "Vigilance monitoring system", US 5,813,993 "Alertness and drowsiness detection and tracking system", US 6,575,902 "Vigilance monitoring system", WO0018471A1 : "Alertness and drowsiness detection and tracking system", US 6,511,424 "Method of and apparatus for evaluation and mitigation of micro sleep events" use various ways of EEG measurement and analysis to provide a fatigue estimate. However because of the EEG measurements requires electrodes attached to person's head, these inventions are suitable for laboratory or medical environment only.

Therefore, fatigue symptoms can be expected to prove themselves in the driver dynamics in driving and changes in the quality of all control processes. Classic methods do not provide a practically exploitable device to determine (in a reliability required) whether or not a driver has shown fatigue signs yet or what is the extend of driver's fatigue, also classic methods cannot, accordingly determine, the driver's ability to perform the activity in question.

Subject of Invention Fatigue is a complex physiological process resulting in effects such as a lethargy, reduced vigilance and changes in autonomous and endocrine functions of human organism. It should be pointed out that the term fatigue in this invention should be understood as including any other reason related to the operator himself/herself, which prevents him/her from fully concentrating on the control activity that should be performed by him/her and, therefore, adversely affects his/her ability to perform that activity. In terms of this invention, the term ,,fatigue" therefore includes, for example, overworking and sleep deprivation as well as use of alcohol, drugs or narcotics or other substances affecting the operator ability to fully concentrate on the control activity he should be performing, as well as operator's physical or mental problems and other factors. The term "fatigue" in ordinary describes a very common phenomenon. For purpose of this invention the "fatigue" comprises and can be defined as: - awareness of a decreased capacity for physical and/or mental activity due to an imbalance in the availability, utilization, and/or restoration of resources needed to perform activity - a state of weariness related to reduced motivation a transitional state between wakefulness and sleep physical state of disturbed homeostasis due to work or stress, which manifest in loss in efficiency and a general disinclination to work - a feeling of weariness and inability to mobilize energy Onset of fatigue is associated with increased anxiety, decreased short term memory, slowed reaction time, decreased work efficiency, reduced motivational drive, decreased vigilance, increased variability in work performance, increased errors and omissions which increase when time pressure, diminishing of information processing and sustained attention. The term "fatigue" used in the invention is to be understood to comprise also any term mentioned below so for purposes of this invention we can consider the following terms characterizing fatigue as synonyms. They are: exhaustion, lack of motivation, tiredness, boredom, sleepiness, feeling tired and listless, apathy, indifference, inertia, lethargy, stolidity, vacancy, drowsiness, depletion, feeling weary, feeling tired, strained or sleepy, being tired, being sleepy, being drained, being worn out, being spent, overworked. Also, fatigue can be suitably understood as opposite to following terms: vigilance, alertness, watchfulness, and wakefulness. Any of these terms as for example lack of vigilance, lack of alertness, can be also suitably treated as replacement of word fatigue in accordance with this invention. The term "operator fatigue" in accordance with further aspect of this invention also includes operator quality as well. The term "operator quality" means operator experience, training undergone and long-time physical and mental abilities to perform a given activity. In accordance with the invention, the operator quality can be treated as a part of fatigue or in a direct connection with fatigue. In that case, a good quality driver of a motor vehicle, for example, may be, to a certain extent of fatigue, classified as fit by the device according to the invention while a worse quality driver of the same extent of fatigue may be classified as unfit. In accordance with the invention, the two terms - "fatigue" and "quality" - can be treated both as a whole - in respect of fatigue in particular - and separately. In case of separate treatment the resulting device can evaluate the operator based only on his/here fatigue. Other resulting device can evaluate the operator based only on his/here quality. The two devices can be used also together to perform complex evaluation. The most suitable technique and a particular practical definition of the term "fatigue" and/or "quality" is suitably selected in a co-operation with experts of the given field of operator activity. The below mentioned explication is focused on an implementation of the device to detect and evaluate fatigue in accordance with the invention. The procedure of creating the device to detect and evaluate quality is almost identical and consists in replacing the inputs and input variables in respect of fatigue by those in respect of quality as well as replacing the target variable characterizing the extent of fatigue by that characterizing the extent of quality. I many cases input variables can be the same for both fatigue detection and quality evaluation, different will be the target variable and the detection/evaluation models. Fatigue results in many physiological and psychological consequences and manifests. One of the consequences of fatigue is, among other consequences, a reduction in the frequency of impulses stimulating body motor units and a reduction in the number of active body motor units, which affects the dynamics of the entire muscular activity. Another consequence of fatigue is a response time extension (delayed response time). That response time extension is transferred, along with the above mentioned complex changes in the operator movement dynamics, to movements of the controlled parts of the system. So it can be said that fatigue results in specific changes, in the operator's activities being thus transferred to the mechanism controlled by the operator. Therefore, information on operator fatigue can be detected wherever control is carried out through an operator muscular activity. In case the operator is a motor vehicle driver the fatigue affects for example the following activities: 1) steering wheel movements 2) vehicle longitudinal and transverse acceleration 3) gear (gear-box) control manner 4) vehicle braking manner 5) accelerator pedal control manner

It is obvious that the operator's fatigue also affects other activities performed by the operator. A detailed description of the activities which might be affected by operator behavior is not included in the scope of this invention, however, every expert can draw up that description for a given area of the activity performed by an operator. The character of the above mentioned changes caused by fatigue or control behavior modifications is very complex. Due to noise, disturbances, failures of the controlled mechanism itself and other factors, the differences in the behavior of the mechanism controlled by the operator of a various extent of fatigue are very difficult to distinguish. Owing to this invention, those complex effects of operator fatigue or differences in operator quality are made possible to be detectable even after having been transformed several times during the transfer and regardless of being included in such "noised" data. For those purposes, the invention provides a sufficiently powerful instrument. The instrument based on the invention uses at least one of data mining methods or even a combination of some of those methods preferably applied on a segmented, split into time intervals, and transformed signal derived from operator physical activity performed through operator's muscles. Herein, the term "data mining" preferably means mathematical or predictive modeling methods and most preferably including its capability to generalize. The term also includes specific data mining methods like, for example, non parametric approaches, visual analysis, etc. For purposes of this invention, the mathematical or predictive modeling methods preferably mean mechanisms to form assessment or decision- making rules from data. For purposes of this invention, the term "rules" is a term comprising any number of rules involved in the fatigue or quality evaluation including one. For purposes of this invention, the term "rules" should be understood to the widest possible extent and includes, for example, mechanisms of evaluating data using trained neural networks. The term "data mining" comprises preferably techniques, instruments and procedures which make possible the transfer of the data and information obtained to a form which enables the mathematical or predictive modeling methods to be deployed directly. For purpose of this invention the term "model" means any mathematical, statistical, predictive, decision making, parametric, nonparametric model or mechanism, for instance linear regression, decision tree, neural network, a simple rule, system of simple or complex rules, non parametric model comparing data with data, etc. The subject of this invention comprises applications of data mining and predictive modeling methods to determine the extent of operator fatigue during performing the operator's routine activity.

The invention uses information on at least one parameter, preferably, on more useful parameters or even on all parameters of the activity to be investigated. That information is used together with the data mining method to create a model of fatigue detection (hereinafter referred to as the "model"). The model captures the relation between the characteristics shown by the operator during his/her activity and the extent of his/her fatigue. The term "parameter" preferably means measurable activity performed or in other words influenced by the operator. As additional parameters can be also used measurable environmental and other factors influencing the operator in performing his activity. In case of the motor vehicle driver, the term parameter comprises or includes a steering wheel movement, accelerator pedal movement, brake pedal movement, gear (gear box) lever movement, vehicle velocity, vehicle longitudinal and transverse acceleration as well as other movements and characteristics. The model created is preferably focused on generalizing for purpose of an evaluation of fatigue of any operator who does not need to be a member of a model group of operators. The model group of operators is a group of operators, which is used to create the model, set its parameters, and generate its rules. Created model and rules are used for distinguishing between different levels of operator's fatigue on the basis of operator's characteristics derived through the movement and behavior of controlled devices and systems. Then, the generalized model can be used to detect fatigue of an operator belonging to the model group as well as fatigue of an operator who does not belong to the model group. In fact, the model is a set of decision-making rules designed to evaluate operator fatigue or a possible decision-making on whether the operator is capable or is not capable of performing a given activity. The model is preferably used to real-time evaluation of the fatigue of an arbitrary operator directly during performing his/her routine activity without interfering anyhow in the operator activities. The same procedure is applied if quality is detected and evaluated; the differences between a good quality operator and a worse quality one can also be supposed to be able to be detected based on the same parameters. Technically speaking, the method of operator fatigue and quality detection based on the invention uses the below listed steps - processes. In addition, the processes can be mutually combined, joined, modified or suitably adjusted. These are the following: 1) Process of processing the signals of the model group of operators 2) Process of data pre-processing and variable creation 3) Process of model creation, training and adaptation 4) Process of model implementation and on-line i.e. real-time fatigue or quality evaluation In a preferred method of fatigue or quality detection, the above listed processes consists of sub processes, which can also be connected to each other in a manner other than that described herein. In addition, processes or sub processes of signal transformation, signal filtration, variable transformation, and variable creation can be in some embodiments skipped or omitted if it is possible to scan data from detectors and similar devices in a suitable form; also processes or sub processes of classical model creation can be in some embodiments replaced by non parametric identification model comparing data with data.

The first step consists in processing the signal of the model group of operators. The input signal is mainly generated by the activity of the operator, e.g. controlling a system including any kind of vehicle, planes, ships etc., a plant or a device in question. According to one embodiment of the invention the input signal preferably includes data on the external factors due to which the operator may behave else, without being fatigued or without being of a low quality, than in their absence. Examples of the input signal generated by the activity of an operator of a plant, system or device are the following: scanning - sensing the position, speed or acceleration of that plant, system or device or its parts, scanning - sensing the movement activity of the operator himself/herself, etc. Then, the external factors vary depending on the operator activity performed. hi motor car driving, the external factors may include, for example, atmospheric changes, road condition, conditions of the environment in which the vehicle is being driving, conditions of the environment inside the vehicle and other factors.

A technical design itself in respect of scanning - sensing and recording the input signal is not included in the scope of this invention. According to a preferred embodiment of the invention, the signal detection/recording is required to make it possible to measure within a corresponding frequency range, a range to 50 Hz is largely sufficient, with the most important information often occurs within a frequency range up to 5 Hz. For example, information on fatigue or quality is included even in the low- frequency component of a range from 0.01 to 0.5Hz. If an input includes the position measurement, the accuracy upper limit may range between 10^"4 and 10^"3m for plants or devices directly connected to the operator, such as a steering wheel, gear lever or pedal, etc. The method of signal processing based on the invention advantageously does not result in any special requirements for obtaining or storing the input signal or those for the linearity of the measurement device range. However, it is useful for the functionality of the resulting device that the input signal used for creating the model, is obtained in a manner as similar as possible to the signal used in the resulting implemented device. Or it is useful that the input variables used in creating the model are identical to the input variables entering the model created in the resulting device implemented or are as similar to those variables as possible. That means it is suitable that the characteristics, non-linearities and defects of the signal derived from the plant, system or device, which goes to the resulting decision-making mechanism implemented based on this invention, are similar to those of the signal used to create the model. Various principles can be used for a construction of a technical embodiment of the device according to this invention. For example, signal detectors, sensors, probes or pickups of various types can be implemented depending on the plant, system or device controlled. These input devices can, to a different extent, pre-process the signal and create variables. To measure position, it could be suitable to use, for example, photoelectric or inductive probes. Swiveling angles, e.g. such as steering-wheel swiveling angles, can be measured using potentiometers or, for example, Hall probe, which was used in the below described invention application example. To measure speed rev counters can be used, to measure acceleration, accelerometers can be used, etc. At present, there are many detectors commercially available for measuring and processing electric and non-electric quantities. The technical development in this field is very rapid. Therefore, a designer is given a free hand in implementing the mechanism of recording the input signals derived from operator activity. As stated above, the external parameters, which affect the operator behavior, could be recorded suitably as well. For purposes of clearness, the following examples of those parameters may be mentioned: load of the motor vehicle controlled, atmospheric effects such as black ice or wet road and similar effects. Those parameters may vary depending on the operator activity performed. To achieve a satisfactory final solution of individual tasks, it could be suitable to include all useful data of all aspects of the performed activity. In evaluating operator quality or operator fatigue, it does not need to be nearly sufficient to measure a single parameter only. For example, if motor vehicle driver fatigue or quality is evaluated, it may not be sufficient to measure steering wheel movement only. Therefore, to ensure the detection is reliable, it can be particularly suitable in certain embodiments to record at least some of other parameters produced by the operator activity or affecting it. In the event of a motor car driver, those parameters are, for example, the following: steering wheel movement, vehicle speed, vehicle lateral and/or longitudinal acceleration and driver's behavior affecting factors such as the terrain in which the vehicle occurs, climatic conditions (such as rain, snow and black ice), vehicle load and other possible data which anyhow affect the driver's behavior in a given situation. The above listed examples of individual parameters are only mentioned as exemplary examples and nowise limit the extent of possible data, which can be measured to evaluate fatigue or quality in performing the method based on this invention. It is suitable to cover operator behavior in various situations which may occur during the operator's activity; in the event of a motor vehicle driver, those situations may include: various speeds, drive in a town, along a motorway or cart road and similar situations. In addition, in certain embodiments it may be advantageous to find the above mentioned data for various operator types, e.g. such as for an experienced and inexperienced driver, for various driver age groups, etc. Therefore, it can be particularly advantageous that the data obtained are processed and evaluated in a complexity and mutual connection. That is made possible due to the innovative idea consisting in the use of the data mining techniques - particularly the predictive modeling or some other data mining techniques or a combination of several techniques used at the same time. In that first stage, data produced by operators of a model group of operators, for whom the fatigue or quality is known in advance, are collected. Term "model group of operators" means a group of operators for whom the extent of their fatigue or quality is known in advance. The data from the activity of those operators are obtained to create a fatigue detection model, or a quality estimation model. That process is described below. For individual operators, each time interval of the activity in question is associated with a predetermined fatigue or quality extent, which could be suitably determined on an individual basis. In one preferred embodiment the fatigue extent is determined, for example, based on a number of hours elapsed from the last operator's sleep, number of hours of an uninterrupted work of the operator, an estimate based on an analysis of physiological effects of fatigue or an estimate based on an analysis of face expression, an expert estimate of the functions of which the parameters consist in the operator's sleep history, etc. In certain embodiments it is preferred for the final determination of fatigue to consider the experience of an individual operator and determine the final value of fatigue in co-operation with an expert and/or the operator himself/herself. If, for example, a motor car driver quality is detected and evaluated, operator quality can be determined in respect of the number of kilometers driven or with taking his/her talent and abilities into consideration, etc. In addition, in performing the method based on this invention, it is preferred to work with those data in which any possibility of intentionally affecting the result by the operator is prevented. In measuring data of a fatigued or low quality operator, that operator can "mobilize his/her strength" for a time and behave like an alert or high quality one. After a time, however that ability will pass off and fatigue will start, or low quality of the operator will manifest itself, to affect his/her behavior. It is preferred to divide the measurement data into time intervals, for example, into 1-, 3- or 5-minute intervals of performing the activity in question - refer to below. The total time for which the data are measured as well as the duration of individual intervals into which the time is divided may vary depending on the activity performed so that they nowise limit this invention. The signal measured is filtered, transformed and modified to create an initial data basis. The purpose of the signal filtration is, particularly, to remove undesirable frequencies, noise and useless signal components. Some of the commonly used filters, such as Butterworth, Chebyshev or elliptic filter, can be used. In addition, the signal can be filtered using a weighted moving average or a suitable non-causal filter, such as a non-causal median filter for example. Technically speaking, any suitable filter can be used. The filter selection depends on the measuring device, measurement type - whether, for example, a speed measurement or steering wheel angle measurement is to be used, sampling frequency, type of the vehicle or plant controlled or adjusted by the operator, etc.. Then, the signal is advantageously undersampled or oversampled to suppress the high-frequency component of the signal measured. Now, a description of examples of dividing the signal into time intervals follows. After the filtration of the signal, or prior to it, depending on the filter type, the signal obtained could be suitably divided into time intervals. As mentioned above, the intervals can be of a 1 -minute duration or more or even less. The duration of time intervals is advantageously selected depending on a particular task or situation. The variables related to, or derived from, the signal of a given time interval represent a row in a table. It is preferred to include the extent of operator fatigue or quality at the given time interval as an additional variable in the given row referred to below. An example of dividing the signal into time intervals can be individual minutes of a driver's drive, called driver-minutes, see the invention implementation example. Then, it is preferred to continue by performing a signal transformation. The purpose of the signal transformation is to obtain more information so that it is possible to create a more suitable basic set of variables. However, the signal transformation can be replaced, according to another preferred embodiment by a filtration. Signal transformation in this invention is of a supportive nature and can be in some embodiments simply omitted or skipped. Signal filtration in this invention is also of a supportive nature and can be also omitted or skipped in some embodiments. The most frequently used transformations are Fourier and cosine transformations, which are used to decompose the signal to several frequency bands, for example, the steering wheel movement signal is decomposed according to one preferred embodiment to the following frequency bands: 0-O.lHz, 0.1-0.4Hz, 0.4-0.8Hz, 0.8-1.5Hz and 1.5-2Hz. However, that dissection can be performed in a different manner as well, e,g, the following transformations can be also used: Walsh-Hadamard transformation, Haar transformation, Hartley transformation, wavelet transformation, etc. In addition, in some embodiments the transformations can be combined in various manners. The essence or aim of the above mentioned transformations is to transform the signal into known waveforms or a form easier to work with. For Fourier analysis, those are sine waves of a various frequency while for wavelet analysis, those are "wavelets" which, moreover, are deformed in various manners. In other embodiments it is preferred to perform the signal transformation after the signal has been divided into time intervals. The next step consists in the preferred embodiment in an inverse transformation, for example, an inverse cosine transformation, with 5 derived signals of the above mentioned frequencies are obtained for the above mentioned example. To transform the signal, it is preferred to employ "expert knowledge", i.e. knowledge of an expert in the given field. That is for the purpose that it could be very convenient to know the fatigue effects and influence on muscular activity or the effect of operator's quality on his/her performance and working method. In certain embodiment it could be advantageous to know possible effects of fatigue or quality on the signal obtained.

The process of the signal transformation and variable generation may be very various. Instead of a cosine transformation, for example, Fourier transformation or filtration can be used in certain embodiments. Instead of the above mentioned 5 frequency bands a different number of ranges can be also used. Another type of the transformation is a filtration. The original signal is passed through suitable filters so that derived signals are obtained using a different manner, which differ from each other by their spectra. Further signal processing can be performed in the above mentioned manner. It is preferred when the transformation is carried out with considering the manner of obtaining the signal so that the next step - generating a suitable basic set of variables - is made possible to be performed.

Generating Variables

The goal of generating a basic set of variables is to provide input data to create a predictive or mathematical model. It is preferred to create variables from both original signal and signal passed through various transformations and divided into segments, sections or intervals. From that signal, obtained are those variables which can be supposed to be of a higher information value in respect of the target variable than the original signal. To generate a basic set of variables, "expert knowledge", i.e. knowledge of an expert is preferably used. It means that it may be very useful in some embodiments to have a knowledge of fatigue or quality consequences and effects on muscular activity available. In addition, it is very preferred to have a conception of possible effects of fatigue or operator quality. Basically said, the better the variables are generated and the better is the expert who performs generating, the better the resulting model can be. However, as mentioned above or below, that expert can be also replaced is some embodiments by a large quantity of various data related anyhow to the operator activity problems in question, and, subsequently, by selecting the best of the derived variables depending on their connection to the target variable, i.e. operator's quality or fatigue, investigated using data mining techniques. From among all the data, it is preferred to only select those which have been assessed to relate to the problem being solved. A manner of performing that process is described below.

Variable examples are the following: a) Variables based on the energy of the signal Energy of signal of individual frequency bands Proportion of energies of two or more signals of different frequency bands (for example proportion of energy in frequency band 0.5-0.8Hz to energy in frequency band 1-1.5 Hz) Proportion of energies of two signals of different frequency bands, related to the total signal energy. Other variables

b) Variables based on average mutual information: Average mutual information of signals of different frequency bands Mutual information of signals obtained from different pickups (vehicle longitudinal and transverse acceleration - after filtration) Other variables

c) Variables based on general statistical properties of the signal: Average deviation Standard deviation within a given time segment Skewness, Kurtosis Number of passes through zero Other variables

Other variables can be derived from the spectrum, signal wavelet transformation, derived signals, etc.

If, for example, an operator using a computer mouse follows an object on a monitor, the following examples of variables can be used: Average distance between the cursor and the mouse for the last 0.1, 0.3, 0.5, 1 and 2 seconds Average absolute value of the mouse movement speed for the last 0.1, 0.3, 0.5, 1 and 2 seconds The main frequencies of the spectrum obtained from the last 20 seconds of movement Frequency response recorded in form of response to pre-determined frequencies of 0.01, 0.05, 0.1, 0.2, 0.5, 1, and 1.5 Hz (its amplitude and phase slip). Cursor and mouse acceleration for the last 0.1, 0.3, 0.5, 1 and 2 seconds and absolute value of acceleration The quantity of the variables generated can be arbitrary large; some of the variables can look meaningless at first sight. It can be generally said that the goal of this step is to create large sets of variables and rows, which expresses,, contains, or describes the connection between the operator behavior and the target value i.e. the extent of operator fatigue. Those connections can be fully unknown or causally unexplainable, even subsequently. Generally it is preferred when the final output of the previous step consists of data which characterize the activity of individual operators in individual time intervals. So, those measured time intervals correspond to a different extent, known beforehand of fatigue or quality of individual operators. In a preferred embodiments a particular output consists in, for example, a table of predictive modeling which could be named, in some places below, as "predictive modeling table", an example of which is shown, in connection with fatigue detection, in Fig. 1. It is a table containing input variable columns and an output variable column. The input variable values have been derived from the signals of individual operators (drivers) in individual time intervals; therefore, they reflect the operator activity during the interval in which the activity is performed by the operator. The output variable i.e. the output target value describes a corresponding extent of operator fatigue in each time interval (identified as FATIGUE in the figure). Then, the table row consists of particular values of "input" variables characterizing the activity and the corresponding "target" variable value characterizing fatigue in a particular time interval of the activity of an operator. Here, it is important to point out that a large quantity of input variables can be generated for each-row of the table (i.e. for each time interval to which an extent of fatigue - the target variable value - is assigned). If operator quality is detected and evaluated, the target variable will be operator quality; the whole procedure will be almost identical; input variables may be identical as well. While quality differs mainly from person to another fatigue is much more variable target which changes for the same person within short periods of time - e.g. hours. An example of possible input variables generated is described below. The predictive modeling table makes it possible to use extraordinarily effective top commercial software programs such as SAS Enterprise Miner, SPSS, STATISTICA Data Miner or Oracle Darwin. These programs have been intensively developed for decades for the purpose of solving similarly complex problems in the fields of top technologies, banking, business intelligence and industry as effectively as possible. Existing techniques and methodologies used to detect fatigue are limited as they process information contained in a signal that has not been transformed or data that have not been pre-processed sufficiently or suitably. Due to the given limiting conditions, the output of those methods is a naive detection model or a single method, by which its authors try to solve various complex problems associated with individual model creation stages. Naturally, the efficiency of that approach cannot compare with a deployment of top, time-tested and long-time developed high-tech environments and combinations of their extraordinarily effective methods, complementing each other, in individual model creation stages. Unlike that of naive and simple methods, the use of those advanced systems and their top methods results in forming a sufficiently effective model of fatigue detection. The employment of those advanced environments is particularly allowed due to converting the signal (using previous stages of signal transformation) into a predictive modeling table, dividing the signal into time intervals and variable generating.. The use of a predictive modeling table, which may be enhanced by using a top data mining environment, makes it possible to really use all available parameters derived from the operator activity (steering-wheel movement, pedal movement, vehicle movement, etc.), and to realistically consider the benefit of those parameters. Another advantage consists in the possibility of selecting an optimum combination of those parameters, for the information benefit resulting from a combination may be too insignificant to deal with. In addition, the information benefit of a parameter may be included in another parameter or a combination of more parameters. In addition, a parameter may first seem to be interesting, however, in the end it may lead to useless or even misleading information in respect of the real fatigue detection. It is evident that the absence of that information which is provided by the approach based on the invention typically results in focusing on unimportant aspects that do not lead to the goal even if a great effort is made. That is more than typical of the fatigue detection problems. The next step comprises variable pre-processing. In that process, it is typical to deploy some of specific data mining techniques such as mechanisms of missing value replacement, mechanisms of extreme value elimination, linear and non- linear transformations, mechanisms of ^' dummy variable creation and similar mechanisms. A particular example consist in eliminating extreme values and measurement errors using different statistics or regression algorithms; usage of ,,fuzzy" logic, application of principal component analysis, creating dummy variables in working with a nominal or categorical variable, logarithmic or logistic transformation and similar processes. As an example of replacing the missing values of a given variable, is their replacement by an average value of the variable itself or a typical value in connection to values of other variables in the given row of the table (in the given time interval). The replacement of missing values by different typical values is preferably carried out by means of decision or classification trees or random forests. It is very advantageous when the table being prepared is consistent. Variable transformation examples are as follows: - calculating the variable logarithm dividing the variable by its standard deviation narrowing the variable range to an interval (-1,1) other transformations - Variable transformation in this invention is of a supportive nature and can be in some embodiments simply omitted or skipped. The purpose of that transformation is to achieve that the variables have a suitable statistical distribution and other characteristics suitable for subsequent processing. In transforming the variables, it is very advantageous to know the methodology of predictive modeling. In certain embodiments of the invention it is very advantageous to know the requirements for input data for various types of predictive models. It is also very advantageous to know statistical methods and procedures. The variables or combinations of the variables of which the values do not relate in useful manner to the target variable (operator fatigue or quality) investigated are preferably excluded in a suitable manner. Useful variables or the combinations of variables, which sufficiently express the target variable, can be additionally complemented, combined or transformed. The process of variable pre-processing and transformation improves in the preferred embodiment the efficiency of predictive model training and the model's resulting accuracy in estimating the target value, the operator's fatigue or quality. Therefore that process is preferred, but not necessary. Using the process, other variables can be obtained as well. It is preferred when the output of this stage consists in, for example, a classic predictive modeling table adapted to an instant deployment of a particular technique of data mining, mathematical modeling or predictive modeling, see the below described mechanism. The use of a predictive modeling table, which may be enhanced by using a top data mining environment, makes it possible to select the methods that best suit the individual stages of fatigue model development and, particularly, to select, combine and evolve the best generalizing model. The advantage consists in the possibility of finding the right direction and, then, focusing on making the detection model more effective. The absence of that possibility leads to a uselessly strenuous development of a model that is insufficiently effective or may only be useful for solving a partial problem of one of many crucial aspects of those complex problems. The use of a predictive modeling table, which may be enhanced by using a top data mining environment, makes it possible to process a necessary extensive set of input variables and to select the best variables that may form the base for a sufficiently effective model of fatigue detection. The advantage consists in the possibility of generating that quantity of variables, for example, thousands of variables, which will sufficiently cover all combinations of seemingly useful inputs. This is another, very essential initial prerequisite for the success of the whole project. In the absence of that possibility, which is only allowed due to the technique being submitted, the problems cannot be approached objectively - with a focus on the goal to be reached and the creation of a sufficiently effective model; the problems may only be approached based on restricted predetermined hypotheses dealing with a very limited part of available information, which leads to an insufficiently effective model.

The next step preferably includes creating, training and adapting the model. During this stage, models capturing, and possibly expressing, the relation between the characteristics shown by the operator and the extent of operator fatigue or quality are created. Those models afe^~~^~ focused on generalization for purposes of evaluating fatigue or quality of a general operator who does not need to be a member of the model group. For purposes of this invention, the term "generalization" preferably means model's ability to evaluate fatigue or quality of new operators how are not members of the model group of operators. The values of individual input variables describing operator's activity are put in context with the fatigue extent, or quality level in case of quality estimation, known beforehand, of a given operator in a given time interval. These values of the input variables with the corresponding values of fatigue extent (or quality level) output variables are preferably used to create the rows of the initial predictive modeling table. These values come from the model group of operators. Within individual data mining techniques, appropriate configuration and adaptation processes are performed to reach the best possible estimate, based on input variables, of the fatigue or quality extent investigated. In this stage, it is very advantageous to employ various advanced data mining techniques such as the latest methods of cluster analysis, regression algorithms, expert systems, classification or decision trees, artificial neural networks, genetic algorithms, fuzzy logic and similar methods. As a particular example, the following, for example, is mentioned: use of a simple logistic regression if fatigue is represented by a binary target variable which values correspond only to two states: "Fatigued (Drowsy)" / "Not fatigued (Alert)". Here, regression parameters are set in such a way that the model adapted distinguishes as good as possible the values of input variables typical of the "Fatigued (Drowsy)" state from the values of input variables typical of the "Not fatigued (Alert)" state. Individual techniques, methods and instruments are tested, compared and combined in a suitable manner to achieve a higher efficiency. The final output of this stage is a model containing the best generalizing features and suitably the best estimate of the target variable - fatigue extent or quality level investigated. In accordance with the above description, the above mentioned estimate is based on the characteristics shown by a routine activity of the operators of the model group through the values of the input variables. According to one embodiment this step is performed by, for example, dividing the original data set into training, validation and test subsets. It is preferred to partly perform that division at the level of individual operators, e.g. drivers, but not at the level of time intervals. That means that a part of operators can be found in validation data only, another part in test data only and the other part in training data only. Out of the total data quantity, for example, 40% are formed by training data, 30 % by validation data and 30% by test data. However, those proportions can be different as well. For example, test data can be omitted and the original set can be split into 2 parts - training and validation ones -only, etc. Then, individual models can be suitably designed using a predictive modeling table divided into training, validation and test data. The model in question can be suitably trained to estimate the connection between the input variables characterizing operator activity and the output target variable characterizing fatigue or quality based on training data. Created model generalization can be suitably set up using continuous evaluation of the model output on validation data. Best model selection can be suitably performed by comparing the different models' estimation accuracy on test data.

The fourth step consists of an implementation, simulation and on-line evaluation (i.e. real¬ time evaluation of quality or fatigue). Provided that the previous steps have been performed successfully, the above described output model created within the previous step is able to generalize. Typical characteristics, based on an evaluation of the model group of operators, related to an extent of fatigue or quality can be connected with the same extent of fatigue or quality of an unknown operator showing similar characteristics under similar conditions, hi this manner, the above mentioned model provides a relevant estimate of fatigue or quality of an arbitrary unknown operator, who has not been characterized anyhow beforehand, based on a mere evaluation of the routine operation of the operator without interfering in his activities. Here, the routine operation is represented by input variables. In that manner, the substance of this invention preferably use, as one of the main components, the generalizing model which captures, and if possible expresses, the connection between an extent of fatigue or quality and the corresponding characteristics of the activities performed by the operator. Other components can be represented in certain embodiments by the modules detecting the operator signal, processing that signal, and transforming it to a form of the generalizing model input data. These components can also contain probes, detectors or pickups performing signal processing suitably into the form of input variables. Using on-line, i.e. real-time, database engineering, all the signal detection, processing, and transformation can be run in real time simultaneously with the operator routine activity and without affecting that activity through any interference. Therefore, the final output of those activities processing the signal to a form corresponding to model input variables is made in the shortest possible time. Then, the input data (variables) are immediately processed by the model to form the latest possible estimate of operator fatigue or quality directly during the activity performed by an operator and without affecting that activity through any interference. Therefore, operator quality or fatigue is estimated simultaneously with the operator activity. Then, the output component is advantageously used to form the directly last estimation of the fatigue or quality, hi one embodiment of the invention the output component can determine, in a probability p, whether the operator (driver) is fatigued/of high quality or not or what is the extent of his/her fatigue/quality. That determination can relate to a given time interval, for example, for the last minute of drive or for the last 10 minutes of drive, etc. As it is known that driver quality or driver fatigue is, within a relatively short driving period, for example, an interval of 30 minutes, similar in all consecutive time- intervals (driving-minutes), then, in case of a 100% accuracy of quality or fatigue estimation, the same or similar estimated value can be found in each of the intervals (driving-minutes) over the course of the period. However, as the predictive/decision-making mechanism is able to estimate quality or fatigue only in a probability, it is suitable to cumulate, sum, average or filter the quality or fatigue estimates of a number of the last consecutive measured intervals in a suitable manner to suitably perform the estimation in the certainty required. This can be performed, in the model itself, in the signalization device, or in a stand-alone module. An estimate formed that way, which may already be relatively stable and fixed, can be interpreted as operator quality or fatigue estimated in a given period of driving. It is possible to increase the certainty of the estimation of fatigue or quality, i.e. the estimation can be made more accurate and stable, in several manners, for example: a) Cumulative summation: Binary or continuous output for the last n measurements (i.e. estimations of target variable - fatigue or quality) is simply summed up and the resulting value is compared with a threshold value; if the threshold value is exceeded, an alarm, can be, fore example, activated. Equation to perform that calculation can be the following:

where y is an output, n is the number of values over which the cumulative summation is performed, />, is the predictive model output value for the i^th time interval (i.e. fatigue or quality level probability in the i"¹ time interval) and b is a threshold, which is of a negative value. If the given sum of, suitably, positive values />, exceeds the value of, suitably, negative b, the resulting positive value y will activate, for example, an alarm. b) Simple average of the last n values is calculated. The resulting value can be displayed on a screen or, if a predetermined threshold value is exceeded, an alarm can be initiated. c) Exponential weighting or exponential moving average is performed. In that event, the last values are of the highest weight. Weight of each value on the resulting evaluation exponentially drops with the age of that value. d) A statistical evaluation, tests or hypothesis testing based on the measured values /»; is performed. I.e., for example, testing of the hypothesis that the driver is fatigued/of low quality against the hypothesis that the driver is alert/of high quality, e) Other manners. It is suitable to note that the model/decision-making rule output can be a binary or continuous value and the last step output can also be a continuous value or it can be rounded to a categorical or binary value. Some of accumulation methods of the last estimates to form the final evaluation of operator fatigue and/or quality at a given time moment can be accompanied with information stating whether the operator is, in respect of the fatigue and/or quality extent found by the evaluation, capable of continuing in the activity or his/her activity indicates a high extent of inattention or another indisposition. Each output datum or evaluation is preferably accompanied with a confidential interval or a probability describing the relevancy of the given datum or evaluation. According to a preferred embodiment of the invention, the estimations or data found are recorded in a recording device for example, in a notebook or other memory device, for operator checking purposes and, later, those results can be used to assess and classify the causes of a possible accident. As stated above, the control process generally represents a complicated feedback mechanism, during which a system is adjusted -such as a vehicle position on the road, vehicle speed and direction - through muscular activity - control - with a visual feedback check carried out by eyes. The feedback loop includes visual perceptions (in the example of the vehicle it is namely position on the road), a process of transmitting those perceptions to the brain, an assessment of those perceptions and a response of the operator (in the example of the vehicle the driver) through his/her muscles; that means that nerve impulses are transferred to muscles and the results of that control are checked back by eyes. As for fatigue or quality, failures caused by fatigue or quality are introduced into the system; these failures may result in a fatal consequence in an extreme case (for example, in case the driver leaves the road with his/her vehicle, collides ion with a vehicle going in the opposite direction or similar events) the consequence could be deadly. When the system makes it possible to transfer information useful for control, then it makes it possible to transfer the failures as well. That means that the system failures caused by operator fatigue or his/her low quality are along with the useful signal, transformed by the system during the transfer and can be measured anywhere in the system circuit. The nature of those failures is described above in 5 items in the chapter dealing with the present state of the art.

Below, the process of useful information separation is described. The previous articles explained the mechanism of the fatigue or quality effect on the process of control, e.g. driving. There are basically two types of the main technical problems of fatigue or quality factor separation. These are the following:

1) The accurate quantitative or, often even qualitative effects of fatigue or quality on the operator's manner of control (driving) are not known 2) The problem of how to separate the effect of operator fatigue or quality from other effects (such as road quality, traffic density or weather).

In connection with item 1, any other analyses do not need to be necessary. To create a model to distinguish between different categories of fatigue or quality, it does not need to be necessary to know, in advance, model parameters as well as causal relationships between the effects of operator fatigue or quality and the model parameters. It is good to recognize that fatigue or quality effects are not only masked by noise but also have been transformed several times due to the transfer -propagation - within the system. In connection with item 2, the above mentioned process of signal transformation is solved. The goal of signal transformation is adapting the signal so that the effect of fatigue or poor quality is unmasked as much as possible. The main idea formulated in the largest possible generalization and the first approximation is the following: "Signal transformation is carried out to obtain the highest possible differences between records of fatigued and alert drivers or between records of high and low quality drivers". That means that a "particular characteristic" derived from the signal obtained for fatigued or low quality operators most differs as much as possible from that obtained for alert or high quality operators respectively. That difference can be found using statistic methods, with the above mentioned division of the measured data into training, validation and test ones can be suitably used. In that way, it can be ensured that the difference is not accidental or coincidental.

Herein, terms "training set" or "training data" mean the following: 1) A set of data identified as training data and used to create models. For those data, the target variable (corresponding extent of fatigue or quality) is known beforehand. 2) Subsets of the above defined set. In the process of model creation, the data for which the target variable (an extent of operator fatigue or quality) is known are preferably divided into training, validation and test sets. So, these are the training data themselves, which are preferably used to teach, adapt and find model parameters.

Similarly, term "test data" means the following: 1) Test data within the training data set used to test a model; that means that those data can be suitably used in the model development. 2) The test data not used in the model creation until the model implementation stage starts. These can be, for example, data obtained in a situation when the model is complete and tested before being implemented in practice or the model is tested "in operation"

To make possible recording and processing the signal in a suitable way, it is preferred to create a database table. A creation of that database table preferably means that the signal is divided, at a point of transformation, into time intervals, for example, of duration of 1 minute each, hi that event, 60 table rows are obtained from a 1-hour signal, which can be used next. It is preferred when the process of useful information separation initially consists of the following two stages: - Signal transformation using signal analysis methods - Database table variable transformation The signal transformation using signal analysis methods means a filtration and derived signal creation as well as cosine, Fourier and wavelet transformation or another suitable transformation, signal energy calculation and derived signal processing and combining. The database table transformations preferably represent combinations of primary variables created within the previous step, secondary variable creation and transformation and other processes. To separate various effects segmentation is preferably used. That means, for example, that created segments correspond to a drive on a dry or wet road, drive on a motorway, drive in a town, drive on side roads, drive with a minimum load or fully loaded vehicle, etc. The segment creation, in the drivers case, is preferably approached by using accompanying traffic information obtained in creating the basic data file or it can be approached based on the data themselves. Using the segmentation, a set of database tables is obtained, with a group of appropriate models that can be advantageously created for each table. It is preferred when the resulting overall model, according to a possible embodiment, of a complicated structure initially consists of a system of decision-making rules to assign a given situation into an appropriate segment. The process of variable generation is very important in terms of creating a functional model. According to one embodiment, a part of the process is solved "by brute force"; that means e.g. that tens of thousands of variables are generated and, then, variables with useful information are selected for next processing. These selected variables can be further improved. However, it is preferred when the existing knowledge of the problems being solved is included to the maximum possible extent in the process of variable creation. Variable creation is very wide field so that every expert surely comes upon many variables of that kind which may depend on the differences in control performed by a fatigued or alert or high or low quality operator.

In the event of fatigue detection, the process of transition from a rest to a movement e.g. of a leg, hand, etc., for a fatigued driver will be, based on an assumption, of a dynamics different from that for an alert driver. Similarly the dynamical properties of a high and low quality operator are supposed to be different. Although it is here an assumption only, that fact does not prevent the situations in which a transition from a rest to a movement occurs from being extracted from the signal as well as it does not prevent that transition from being described in a manner, for example, by an approximation of functions; then, an appropriate group of variables can be created for example, the parameters of the function used to approximate that transition. To perform the fatigue or quality detection according to this invention it is preferred in one embodiment to use those variables that show the strongest relation with the target variable - fatigue or quality.

It is possible to generate a lot of the above mentioned working hypotheses, hi addition, it is possible to generate a lot of variables of which the most suitable ones (which may amount to, for example, a 1%) are only used. The resulting situation is as follows: If there is knowledge of fatigue or quality problems, it could be suitable to generate several hundreds of variables which, based on an assumption, include variables of a predicating value. - If there is no knowledge of fatigue or quality, it is preferred to generate much more, for example 10000 - variables, from which the data pre-processing mechanism and variable selection mechanism can select several tens to one hundred of the most relevant variables. According to a preferred implementation of the method, it is obvious that these mechanisms can also be used if there is knowledge of the appropriate problems as it makes it possible to select really the most relevant variables. However, in that event, it does not need to be necessary to perform as many calculations, as all the variables are confronted with the target variable, as they are if there is no knowledge or the knowledge is not used for a reason. According to one embodiment it is preferred, that the determination of strength of the relation between the variables created and the target variable is based e.g. on the mutual information, correlation coefficient comparison, χ² test and other statistical techniques.

According to another aspect of the invention, here's another advantageous method of detecting an operator physical capability of performing an activity, vehicle driving in particular, of which the substance consists in the fact that the signal of at least one of the parameters generated by operator muscular activity such as, suitably, if driver fatigue or quality is detected, steering-wheel movement, accelerator pedal movement, brake pedal movement, clutch pedal movement, information on the vehicle movement and changes, caused by operator activity, in that movement, is scanned. Those data are compared with appropriate data typical of an alert or high quality and/or fatigued or low quality operator to assess whether the operator is high quality or alert or low quality fatigued.

A description of another also preferred detection method follows: a) at least one parameter is scanned for at least one alert/high quality operator and at least one fatigued/low quality operator, b) at least some selected data are transformed to increase the differences between specific values measured for the alert/high quality and fatigued/low quality operators. In this way many useful variables can be obtained c) using a transformation of the variables, a data set is obtained and stored in a suitable form, d) in real-time device for fatigue detection or quality estimation at least one identical parameter is measured for a particular operator, e) in real-time device for fatigue detection or quality estimation they the data are transformed in the same manner as that used for training data so as to obtain the same variables, f) in real-time device for fatigue detection or quality estimation the variables (data transformed) obtained by processing the same parameter or parameters for a particular operator are compared with the data stored in order to estimate whether the particular operator is alert or fatigued or his quality is high or low. It could be suitable if the above mentioned step c) is followed by creating a fatigue detection specific model or quality estimation specific model to which the variables mentioned in Item f) are compared to find the operator fatigue or quality in real-time. Non parametric models can be used. However, it can be very advantageous if the above mentioned step c) is followed by creating a fatigue or quality detection model which output provides direct operator fatigue or quality assessment in real-time. The inputs of this model are the variables mentioned in Item f). The model creation can be carried out using any suitable data mining method. This includes, for example, classical methods like linear regression, specific methods like nonparametric regression, or visual methods like 3D analysis.

The method based on this invention can be suitably employed in a real-time device to perform the fatigue detection, or quality estimation based on this invention. According to one also advantageous embodiment the device, for the real time fatigue detection or quality estimation, includes a programmable unit with a fatigue or quality detection programmed model, which includes fatigue or quality evaluation rules obtained by measuring the signal of at least one of the parameters generated by operator muscular activity for at least one fatigued or low quality operator and at least one alert or high quality operator. In case of fatigue estimation it can be the same operator in his/her period of fatigue and alertness. In addition, the device includes a detector or detectors to measure or to measure and process the signal of the parameter or parameters suitably into the form of input variables used by the model. The modules processing the signal into the shape of input variables can be parts of the detectors, separated units, or parts of the programmable unit. The detector(s) is (are) connected to the input of the programmable unit to evaluate operator fatigue or quality while the programmable unit output is connected to a fatigue or quality signaling device. For purposes of this invention, the programmable unit means any device, preferably able to process the signal into the form of input data or variables, which can include a fatigue or quality detection model or can be programmed by a fatigue or quality detection model created. That programming can be performed both before the detection starts, e.g. already during manufacturing, and after the detection starts e.g. after a vehicle is started. The programming unit is advantageously a computer, pocket computer, notebook, processor, embedded electronic system, etc. The programming unit advantageously includes a fatigue or quality detection/evaluation model created or, suitably, corresponding rules. The fatigue or quality signaling device includes advantageously any kind of suitable signaling devices comprising a visual signaling device e.g. a light indicator, display, etc., an acoustic signaling device e.g. a siren, buzzer, bell, speaker, etc., or even a complicated devices making it possible, for example, to safely put the controlled plant out of operation, stop the vehicle, train, etc. or call a help e.g. a senior operator, reserve operator, etc.. The fatigue or quality-signaling device is not included in the scope of this invention and can be designed by any expert for a given area of the activity investigated.

According to even another advantageous embodiment, the device consists of a computer memory which includes data to create a fatigue or quality detection model, a programmable unit adapted to be programmed using the model created that way and a detector or detectors to measure or measure and process the signal of the parameter or parameters used by the model. The detector(s) is (are) connected to the input of the programmable unit to evaluate operator fatigue or quality while the processor output is connected to a fatigue or quality signaling device. Data to create the fatigue or quality detection model can include both real data, based on which rules to evaluate fatigue are created, and those rules themselves. According to even further advantageous embodiment of the device according to the invention includes detectors for measuring or measuring and processing the signal of more parameters related to the operator or vehicle movement. According to another advantageous embodiment the device include detectors to measure or measure and process all available parameters. According to another aspect of the invention, a device to create fatigue or quality detection models is hereby being submitted. The device consists of a computer unit equipped with a program or software to create mathematical or predictive models and a computer memory with either parameters of at least one operator for whom the fatigue or quality extent is known beforehand or variables obtained from those parameters. In the computer memory, variables for a model group of operators can be suitably stored. A suitable form of the storage, in order to quickly build effective predictive models, can be classical predictive modeling tables or datasets. Then, the computer unit could be a server with a possible computer network. In certain embodiments of the invention it is possible to skip or omit signal and/or data processing. This includes signal and/or data transformation and filtration. In this case detectors and similar devices scan data directly in the form needed for the evaluation by the model. The invention makes it possible to evaluate the extent of fatigue itself, sleep deprivation, or another operator indisposition. The method is very reliable, due to the data mining technique used in particular, and brings a rapid evaluation, which, if performed in a suitable manner of cumulative evaluation, becomes the more reliable the longer is the time of operator activity examination. It is naturally possible to set a threshold of fatigue or quality values on one hand and, on the other hand, time intervals of operator's activity during which measured scanned values are summarized to enter the model in order to create the next, latest, estimation (evaluation). As stated above concerning the fatigue or quality signaling device, the fatigue or quality evaluation outputs can be diverse. They can warn directly the operator of the fact of fatigue occurred or, in case of danger, they can stop the operator activity by, for example, switching-off the vehicle ignition after several previous warnings if that drive continues in spite of the warning or the output can be led out to a position superior to the operator or, naturally, to any other place where a timely remedy can be arranged. In addition, the invention is advantageous due to the fact that it makes it possible to eliminate those effects on the results, which are caused by an instant non-standard behavior of operator, which may result from effects other than operator fatigue. The invention is also suitable for combinations of one or more of the mechanisms and procedures stated above. An operator sleep deprivation is naturally connected with his/her fatigue. However, it is obvious that, except for fatigue itself, operator behavior may be connected with other reasons of "fatigue" as well, such as an use of alcohol or drugs affecting the driver's vigilance, nerve disturbance preventing the driver from his/her concentrating on driving or many other factors, resulting in a driver's incapability of fully concentrating on the activity performed. An assessment of those reasons, regardless of being assessed separately or along with other factors, is possible by using the method and device based on the invention. It is obvious that an evaluation of any fatigue comprising, a sleep deprivation and any other reasons for an operator's incapability of reliably performing an appropriate operator activity as well, using the detection method or device according to this invention defined by the patent claims further, is included in the scope of application of this invention.

Another advantage of this invention consists in the fact that it makes it possible to replace a model group of operators by a single operator or a suitably adapted group of operators to assess a particular operator. That means an adaptation of the model group of operators or other mechanisms of the invention so that the best possible assessment of a particular operator or group of operators could be achieved.

Another advantage of this invention consists in the fact that it can be used in implementing or assessing tests of a person's fitness for his/her activity (,,fitness-for-duty tests") based on the characteristics shown, for example, on a simulator or another test device based on the invention (that is using the data mining techniques). The fitness of the operator can be tested before, during or after the given activity. It is an implementation of the patent on an auxiliary device, which may be other than that which is used by the operator in his/her routine activity. An assessment of operator fatigue, quality, or fitness using the above mentioned procedures and devices before, during or after starting the operator's routine activity is also an inherent part of the invention and its use, and that assessment is included in the scope of the invention protection as well. As for vehicle driving, the above mentioned signals of a model group, which are important for further processing, may be derived, for example, from the activity of the model group of operators on simulators, trainers and similar devices to simulate a motor vehicle driver activity. The model created based on appropriate data may also be useful to assess the fatigue of an operator of a real vehicle. That procedure is also included in the scope of the invention. Above mentioned details applies also to the device for operator quality evaluation which can be developed in the same way.

Below is a recapitulation of key factors that are among the invention advantages over the state of the art and methods detecting operator fatigue. Unlike the state of the art and related methods, these factors make it possible to really and effectively detect fatigue as well as to commercially use the device in accordance with the invention and substantiate its serial production:

A) Sufficiently effective and practically applicable fatigue detection implementation substantiating a commercial serial production of the device in accordance with the invention: Existing methods and techniques largely use a single parameter only, for example, steering- wheel movement is utilized in fatigue detection using simple methods. Other methods and techniques are still in the off-line study stage and test the importance of different hypotheses. Other methods and techniques are based on use of a single method, for example, a specific algorithm that uses a neural network or a specific regression method. Other methods and techniques describe a device to detect fatigue in real time without solving crucial aspects of the problems, for example, a development of a sufficiently effective method of the detection itself. Each of those methods and techniques only solves a part of one of many steps necessary to create an effective and commercially applicable instrument. That instrument cannot be created by putting different, often contrary methods and techniques cited in literature together. The method in accordance with on the invention differs from those methods in the fact that it makes it possible to conceptually connect the biological and physiological aspect of the problems to the technical aspect, with - being fully focused on the goal and result achieved, - using all available useful information obtained from operator muscular activity. That connection is allowed owing to the use of data mining techniques and their application according to the invention. That means an application of a complex progressive approach to data mining, which is based on dividing signal into time intervals, signal transformation, variable generation and predictive modeling table creation. That approach provides the following key advantages: 1) The predictive modeling table makes it possible to use extraordinarily effective top commercial software programs such as SAS Enterprise Miner, SPSS, STATISTICA Data Miner or Oracle Darwin. These programs have been intensively developed for decades for the purpose of solving similarly complex problems in the fields of top technologies, banking, business intelligence and industry as effectively as possible. Existing techniques and methodologies used to detect fatigue are limited as they process information contained in a signal that has not been transformed or data that have not been pre-processed sufficiently or suitably. Due to the given limiting conditions, the output of those methods is a naive detection model or a single method, by which its authors try to solve various complex problems, associated with individual model creation stages. Naturally, the efficiency of that approach cannot compare with a deployment of top, time-tested and long-time developed high-tech environments and combinations of their extraordinarily effective methods, complementing each other, in individual model creation stages. Unlike that of naive and pioneer methods, the use of those advanced systems and their top methods results in creating a sufficiently effective model of fatigue detection. The employment of those advanced environments is particularly allowed due to converting the signal (using previous stages of signal transformation) into a predictive modeling table, dividing the signal into time intervals and variable generating. 2) The use of a predictive modeling table, which may be enhanced by using a top data mining environment, makes it possible to really use all available parameters derived from the operator activity (steering-wheel movement, pedal movement, vehicle movement, etc.), and to realistically consider the benefit of those parameters. Another advantage consists in the possibility of selecting an optimum combination of those parameters, for the information benefit resulting from a combination may be too insignificant to deal with. In addition, the information benefit of a parameter may be included in another parameter or a combination of more parameters. In addition, a parameter may first seem to be interesting, however, in the end it may lead to useless or even misleading information in respect of the real fatigue detection. It is evident that the absence of that information which is provided by the approach based on the invention typically results in focusing on unimportant aspects that do not lead to the goal even if a great effort is made. That is more than typical of the fatigue detection problems. 3) The use of a predictive modeling table, which may be enhanced by using a top data mining environment, makes it possible to select the methods that best suit the individual stages of fatigue model creation and, particularly, to select, combine and evolve the best generalizing model. The advantage consists in the possibility of finding the right direction and, then, focusing on making the detection model more effective. The absence of that possibility leads to a uselessly strenuous development of a model that is insufficiently effective or may only be useful for solving a partial problem of one of many crucial aspects of those complex problems. 4) The use of a predictive modeling table, which may be enhanced by using a top data mining environment, makes it possible to process a necessary extensive set of input variables and to select the best variables that may form the base for a sufficiently effective model of fatigue detection. The advantage consists in the possibility of generating that quantity of variables, for example, thousands of variables, which will sufficiently cover all combinations of seemingly useful inputs. This is another, very essential initial prerequisite for the success of the whole project. In the absence of that possibility, which is only allowed due to the technique being submitted, the problems cannot be approached objectively - with a focus on the goal to be reached and the creation (formation) of a sufficiently effective model; the problems may only be approached based on restricted predetermined hypotheses dealing with a very limited part of available information, which leads to an insufficiently effective model. Other factors supporting the efficiency of the final detection model, such as a suitable transformation of variables, which increases the difference between data for a fatigued driver and those for an alert driver, missing data, removal of misleading extreme data and sensor errors, etc. 5) The total of the advantages provided by the invention makes it possible to use top commercial environments for a systematic and targeted evolution of a sufficiently effective and generalizing model that uses all useful information of parameters, the best analytical and predictive methods and an optimum set of variables. If one or more of these advantages is (are) absent, it will result in dispersing the means intended to develop the detection model in various directions as well as it will result in naive efforts leading to models that cannot be sufficiently effective. That situation can be documented by the existing absence of an effective, commercially applicable device to detect fatigue of, for example, vehicle drivers.

B) Sufficient generalizing ability making possible to sufficiently accurately evaluate the fatigue of an arbitrary unknown operator: The advantage consists in the ability to evaluate an arbitrary unknown operator, of whom the characteristics have not been used in the process of creating the fatigue detection model. Naturally, that is a real-time evaluation, performed using the device in accordance with the invention during operator routine activity. If this factor - submitted, described and highlighted - is absent, the model will not be able to evaluate the fatigue of new operators who have not been members of the model group, of whom the characteristics have been measured and analysed in a complex process of model creating. That absence evidently makes a mass commercial use of the fatigue detection mechanism impossible.

C) Robustness and reliability of the device to perform real-time fatigue detection in accordance with the intention: The advantage consists in the highest possible reliability and accuracy of operator fatigue real-time evaluation using the device in accordance with the invention. To achieve that goal, the technique being submitted which uses all useful information available, is necessary to be used; in addition, it is necessary to use the procedure that uses an accumulation of individual decisions. The advantages included in Clause A are very contributing as well. If this factor - highlighted - is absent, it will lead to a practical uselessness of the device, which would correspond to existing situation in the field.

D) Practicability of the device in accordance with the invention Another advantage of the invention consists in the fact that it is not a pioneer off-line study omitting some fatigue detection aspects which seem to be of a less importance but, in fact, they are complex and unconditionally necessary, however, it is a complex solution across all the phases of the problems. In fact, a substantial part of the above mentioned techniques, procedures, solutions and conclusions has been reached based on particular practical experiments (refer to examples of practical use performed on real data under prevailing realistic conditions). 1) Fatigue detection practicability a) The advantages mentioned in Clause A, the predictive modeling table enhanced by the analytical environment in particular, make it possible to use a revolutionary technique of solving the problems. It is that approach to predictive model creating which makes it possible to adapt all analytical and modeling means so that the fatigue detection is as effective as possible. Unlike existing studies based on an idea of using a particular method for solving the given problems, the invention being submitted makes it possible to use a reverse technique, which, on the contrary, adapts analytical and modeling achievements to the problems and, particularly, the fatigue detection efficiency. In spite of starting from use of a particular method, a particular type of neural network, for example, the technique, on the contrary, starts from the given problems and finds the most effective method, the best variant of that method and an optimum setting of that method, with all the above mentioned steps are performed for each particular stage of model creating (forming) so as to achieve a single goal - the best possible fatigue detection. That is practicable due to the use of a predictive modeling table to describe the system and the use of advanced analytical environment, making it possible to fast compare the efficiencies of all possible variants and combinations. The same applies both to the set of input variables and operator behavior parameters and characteristics. That means that, in spite of starting from particular parameters, characterizing operator activity, and particular variables, the technique, on the contrary, starts from all available parameters and variables and the effect of those parameters and variables on the fatigue detection values obtained. The main advantage of this item consists in the revolutionary approach focused on achieving the best possible efficiency of fatigue detection. This is ensured by considering all methods, parameters and variables as well as selecting the most effective combination of them in spite of using a single particular method or a few operator parameters only and predetermining a set of variables or signal in a particular spectrum. b) In addition, the fatigue detection practicability consists, for example, in focusing on the information which is directly associated with the very control of the given equipment. That is for the reason that some changes detected in the control loop unambiguously stem from the muscular activity characterizing a fatigued operator. Unlike the invention, other approaches such as eye movement (blinking) monitoring or actigraphy include a disadvantage consisting in a limited number of different independent inputs to support the method reliability and robustness. As regards those methods, the reliability of obtaining input data is disputable, which results from a rather low sensitivity of some devices to environmental factors, hi addition, the device in question may not obstruct the operator routine activity. For example, a camera to take eye movement (blinking) may directly or subconsciously bother the driver and its function depends not only on the above mentioned environmental factors (unstable lighting) but also on the behavior of the driver himself/herself (face movements outward the camera field of view, etc.). The techniques focused on monitoring those factors lead to the above mentioned naive methods lacking necessary prerequisites and advantages described in previous Clauses A, B and C. Similar advantages are associated with other approaches such as physiologic parameter detection; EEG and ECG monitoring, for example, is even much more inconvenient, for it requires a more difficult device and much more obstruction in the operator environment. c) As regards the operator-muscular-activity-monitoring-related techniques being submitted, it can also be stated that, for example, pedal movement detection has an advantage over vehicle movement detection, which consists in interpreting the operator activity directly - not in a mediated manner. That is for the reason that the vehicle movement can be affected, for example, by inclined road, wind, vehicle load and similar factors.

2) Practicability of a sufficiently effective model. As mentioned above, the suitable use of a predictive modeling table makes it possible, for example, to use top data mining environments. The advanced state and efficiency of the approach have been fully proven in solving no less complex problems in the fields of economy, top technologies, banking, industry, etc. In those fields, the deployment of the above mentioned software programs such as SAS Enterprise Miner, SPSS, CART, Oracle Darwin and other programs is regarded a standard, sufficient and, often, the only possible approach. 3) Practicability of the device in accordance with the invention. It is a device operating in real-time, directly during the operator activity and without any intervention in that activity or operator environment. The practicability is supported by the description of the complex mechanism - from movement sensors (such as steering-wheel movement, vehicle movement or pedal movement sensors) through signal transformation, signal processing and signal conversion to input variables of the built-in fatigue detection model to model output accumulating and fatigue signaling device - of the device in accordance with the invention. As mentioned above, the practicability of the device is made possible due to the present advanced technologies and online (real-time) database engineering as well.

The invention will be now described by way of examples with reference to the accompanying drawings.

Brief description of drawings

FIG. 1 represents a predictive modeling table, FIG. 2 describes principle of fatigue detection based on pursuit-tracking an on-screen cursor movement using a computer mouse, FIG. 3 represents a decision tree diagram, FIG. 4 shows a decision tree detail, FIG. 5 shows segmentation based on a fatigue probability modeled, FIG. 6 describes fatigue values modeled for unknown operator 1, FIG. 7 describes fatigue values modeled for unknown operator 2,, FIG. 8 describes a procedure of signal processing in fatigue detection based on steering- wheel movement, FIG. 9 describes process of predictive modeling in fatigue detection based on steering-wheel movement, FIG. 10 shows a diagram of neural networks used for fatigue detection based on steering- wheel movement and FIG. 11 represents a diagram of device for fatigue detection based on steering-wheel movement and vehicle speed

Examples of the invention

Below, to be more easily to understand, the invention is described by using two different examples of its possible use. These examples are related to fatigue detection. It is obvious that these two examples are not mentioned to anyhow limit the examples of the invention use, which may include all categories of human activity in which a response of the operator's muscles to a situation related to the activity performed by the operator occurs. Transforming these examples to quality estimation examples is trivial and consists mainly in replacing the fatigue extent target variable by a quality extent target variable.

The first example includes a simple demonstration of a fatigue detection based on a computer mouse movement displayed on a screen during pursuit-tracking a randomly moving object. The second example includes a fatigue detection based on steering-wheel movements, vehicle speed and terrain character during motor vehicle driving.

A simple example of the patent implementation in practice is an operator fatigue detector based on a task called the ,,pursuit tracking task". That task consists of tracking, using a mouse, an object (a cursor in this example) moving at a variable speed on a screen. The operator's task is to follow, using the mouse, the moving cursor and reach its overlapping. Principle: Due to operator fatigue, a failure in the eye-hand feedback mechanism occurs, which results in a response time increase, vigilance reduction and response accuracy reduction. Those effects adversely affect the accuracy of the moving cursor pursuit tracking. In addition other specifics in the cursor movement, which characterize the activity of a fatigued operator, occur. In case of quality estimation similar characteristic would distinguish performance of high quality operator from performance of low quality one.

A diagram of the pursuit tracking task principle is shown in Figure 2. The cursor moves on the screen as an object of the pursuit tracking. The object movement is random and the change in the movement speed is random as well, with the movement frequencies higher than 2.25 Hz are filtered and the sampling is carried out in 0.020-s intervals. Operators of different extents of fatigue try to follow, using a mouse, the moving cursor to reach its overlapping. The computer, besides the performance of the above mentioned functions, records a detailed run of all the activities.

An implementation of the assessment procedure to detect fatigue is carried out in the following steps:

Data of a model group of operators are processed to create a model. In this stage, activities of a group of operators of different, predetermined, extents of fatigue are recorded. In that event, the criterion for fatigue predetermination is a sleep deprivation extent (sleep deficit), evaluated as a time elapsed from the last time of waking up. In addition, that indicator can be chosen as a function, the parameters of which are a duration of the last sleep, time of waking up and time elapsed from the last time of waking up. Another possibility consists in an arbitrary determination of a fatigue threshold corresponding to an extent of sleep deprivation and an evaluation of fatigue extent on a binary basis (1: sleep-deprived, 0: alert). Another possibility consists in excluding the data in which the fatigue extent is close to the arbitrary threshold selected. That will improve the model discrimination ability and make the model training easier. If operator quality is evaluated, quality can be determined, for example, based on experience and driven kilometers. The following step includes a filtration of the output signal through a median filter and a generation of the variables, which are derived from the behavior of the input and output signals. Those variables include: current speed and acceleration of the object, distance between the mouse image and the cursor, average value of the distance for the last 0.1, 0.2 or 0.5 second, speed and average speed of the mouse image and the cursor, speed and acceleration absolute values and other data (55 variables in total). The variables represent columns of a "classic" predictive modeling table. Each row of the table represents values of those variables for one of the operators within a given time interval. To each row, a predetermined extent of fatigue of a given operator within a given time interval is assigned. In case of quality estimation the predetermined extend of quality of a given operator is assigned. These predetermined values form the last column of the table. The following step includes a variable selection and model creation, which correspond to the steps of data pre-processing and model creation, training and adaptation. In this stage, several predictive models are created such as linear regressions of different settings, several types of decision-making tree and several types of neural network configuration. The input for those models consists of selected subsets of the variables generated. The model, which provides the best fatigue estimate, is selected (due to the fatigue is known beforehand, it is possible to select the model which provides the best assessment and the best generalizing capabilities). The assessment is carried out based on the variables, which describe the operator activity. In this particular example, the selected model was a decision-tree working on the entropy reduction principle; a diagram of the model is shown in Fig. 3. The decision-making rules correspond to horizontal lines and the subsets correspond to squares or rectangles. The following applies to individual levels: the more down is the level, the better is the division of the original set to subsets in which fatigued or alert operators prevail. In Fig. 4, a detail of the decision-tree diagram is shown along with the rules and percentages of fatigued and alert operators in individual nodes. The number stated ahead of the slash represents a percentage of the samples in which the operators were alert and the number behind the slash represents a percentage of the samples in which the operators were fatigued. In this example, a 20-second time interval was selected to represent the sample. The variable according to which the decision is made is stated below the node and the appropriate decision-making rules are shown above the node. The variables used in the given part of the tree diagram are explained in the following table:

In Fig. 5, a comparison between an interception by the decision-tree diagram (dashed line), an ideal interception (dotted line) and random sampling (continuous line) is shown. The X-axis represents table rows (operator - time intervals of 20s) arranged depending on the extent of assessment credibility. The closer is the value to the left end the higher is the credibility that it is a tired (operator/time-interval) and the closer is the value to the right end the higher is the credibility that it is alert one.) As 23% of the samples of the training set originated from fatigued operators, the random sampling line is at that level. The dotted line shows the situation that would arise if all the samples were classified as failure free. The dashed line shows the interception of the best model selected.

The following step includes model testing on operators who were not included in the training set. The above mentioned model is able to generalize. That means that it is capable to generalize the relationship between the characteristics shown by an operator and his/her fatigue, obtained based on observing the model group of operators, for a new operator. During testing the model on two operators who were not included in the original sample of the model group, the results shown in Figures 6 and 7 were obtained. The dashed curve continuing by the black dotted curve represents operator fatigue during a day. After the limit of 0.35 is exceeded, the fatigue course is indicated by the black dotted curve. In this event, the fatigue modeled is dimensionless. The value of 0.35 corresponds to a 20- hour sleep deprivation. In some cases, it is possible to select that indicator as a function of which the parameters are the duration of the last sleep, the time of waking up, and the time elapsed from the last time of waking up.

Finally, an implementation of the model is performed. As the model is able generalize, the model code can be implemented in the pursuit tracking task component in the next step. Other components of a possible implementation will include the modules, which detect the operator signal, process that signal and transform it to a form of input data of the key component model. All the detection, processing and transformation of the signal can be, using advanced database engineering, carried out simultaneously with the operator activity, without any interference resulting in affecting that activity. The final output of those procedures to process the signal into a form corresponding to the model input variables is made in the shortest possible time. Then, those input variables are immediately processed by the model to create the latest possible estimate of operator fatigue, directly during the activity performed by the operator. In that way, it is possible to on-line assess the extent of operator fatigue and operator fitness for a duty; that assessment is, in fact, carried out in a constant time lag corresponding to the longest time interval required to create variables's values. In this example, the time lag was 20 seconds. This fatigue assessment procedure cannot be simply cheated. The procedure requires for the operator to be alert not only at a given moment but also continuously. A fatigued person is not able to keep himself/herself alert for a long time. The above mentioned example is a simple and easy to execute demonstration of the invention. The input signal is a cursor movement and a mouse movement on a screen. By filtering, errors caused by sampling and even by Windows internal timer are removed. As stated above, the variables are derived from speed, acceleration and distance between the mouse image and the cursor. The fatigue assessment using a predictive model can be easily implemented and displayed on the same screen or during the test.

The second example of the invention implementation includes one of the preferred ways of using the invention for an assessment of motor vehicle driver fatigue. That assessment is performed based on a detection of steering-wheel movement, vehicle speed and terrain features. Implementation of this procedure for the quality estimation case is similar and may consist only in replacing the fatigue extent target variable by a quality extent target variable. As stated above, fatigue effects can be detected based on a steering-wheel movement, vehicle speed and terrain features. The mechanism is the same as that described in the previous example or the chapter dealing with the invention substance.

In Fig. 8, a procedure of processing the signal derived from, for example, a steering-wheel angular displacement to create a predictive model is shown. First, input variable are created. The goal of this stage is to generate, for each predetermined fixed time interval of the activity performed by a member of a model group operator (60-s recording in this example), a sufficient number of variables which characterise the signal behavior and are known, based on expert knowledge, to indicate a state of fatigue. In this stage, a set of 264 variables is obtained, generated for each minute of driving, to which a sign 1 or 0 is assigned, depending on if the driver is or is not sleep-deprived. The steps to perform the procedure shown in Fig. 8 were as follows: 1) The signal is transformed to angular degrees and a resampling is performed - the sampling frequency is reduced to 50Hz (This step, as wall as all the following steps, can also be suitably performed in another way or can be omitted or replaced. The above mentioned procedures and values only represent a single example, however, there are many possibilities). 2) The signal of steering-wheel angular position and its derivation (that means steering- wheel movement velocity) is divided into minutes. The vehicle speed signal is divided into minutes as well. 3) A cosine transformation is performed. Subsequently, the signal transformed is divided into the following frequency bands: 0 - 0.055 Hz; 0.055 - 0.15 Hz; 0.15 - 0.25 Hz; 0.25 - 0.45 Hz; 0.45 - 0.65 Hz; 0.65 - 0.85 Hz; 0.85 - 15 Hz; 15 - 1.55 Hz; and over 1.55 Hz. Subsequently, an inverse cosine transformation is performed. 4) The first group of variables consists of the signal energy, energy of individual frequency bands related to the total energy and a mutual ratio of energies in individual frequency bands (the energy of a band of a lower frequency is always divided by the energy of a band of a higher frequency). 5) Another group of variables consists of the signal entropy, entropy of individual frequency bands, and mutual information of individual frequency bands computed for all combinations of frequency bands the signal is divide into. For entropy and mutual information a histogram, for splitting the values (which has a 39 discrete discreet groups), was used. . 6) Another group of variables consists of signal statistical characteristics, minimum, maximum, mean value, standard deviation, skewness and kurtosis. If the signal is of a zero value during the corresponding time interval, i.e. minutes of driving in this example. Zero was assigned to the last two values. 7) Other variables were derived from the signal properties in a given minute. Tthat means the number of intervals in which the signal ranges between zero and a given value, the longest interval in which the signal is within a given range, etc.. Another group of variables consists of steering-wheel acceleration statistical characteristics. Examples of the variables derived from steering-wheel movement are listed in the following table. It is naturally possible to create a large number of other variables, which could be used in a fatigue detection based on this invention.

TABLE OF SOME VARIABLES

Signal energy of frequency band from 0 to 0. IHz (the 1^st frequency band) Signal energy of frequency band from 0.1 to 0,25 Hz (the 2^nd frequency band) Signal energy of frequency band from 0.25 to 0,5 Hz (the 3^rd frequency band) Signal energy of frequency band from 0.5 to 0,75 Hz (the 4^th frequency band) Signal energy of frequency band from 0.75 to 1 Hz (the 5^th frequency band) Signal energy of frequency band from 1 to 1.25 Hz (the 6^th frequency band) Signal energy of frequency band from 1.25 to 1.5 Hz (the 7^th frequency band) Signal energy of frequency band from 1.5 to 1.75 Hz (the 8^th frequency band) Signal energy of frequency band >1.75 Hz (the 9^th frequency band) Total signal energy Signal energy of the 1^st frequency band (0-0.1Hz) divided by the total signal energy Signal energy of the 2^nd frequency band (0.1 - 0.25 Hz) divided by the total signal energy Signal energy of the 3^rd frequency band (0.25 - 0.5 Hz) divided by total signal energy Signal energy of the 4^th frequency band (0.5 - 0.75 Hz) divided by the total signal energy Signal energy of the 5^th frequency band (0.75 - 1) Hz divided by the total signal energy Signal energy of the 6^th frequency band (1 - 1.25 Hz) divided by the total signal energy Signal energy of the 7^th frequency band (1.25 - 1.5) Hz divided by the total signal energy Signal energy of the 8^th frequency band (1.5 - 1.75 Hz) divided by the total signal energy Signal energy of the 9^th frequency band (>1.75 Hz) divided by the total signal energy Ratio of the 1^st frequency band energy to the 2^nd frequency band energy Ratio of the 1^st frequency band energy to the 3^rd frequency band energy Ratio of the 1^st frequency band energy to the 4^th frequency band energy Ratio of the 1^st frequency band energy to the 5^th frequency band energy Ratio of the 1^st frequency band energy to the 6^l frequency band energy Ratio of the 1^st frequency band energy to the 7^th frequency band energy Ratio of the 1^st frequency band energy to the 8^th frequency band energy Ratio of the 1^st frequency band energy to the 9^th frequency band energy Ratio of the 2^nd frequency band energy to the 3^rd frequency band energy Ratio of the 2ⁿ frequency band energy to the 4' frequency band energy Ratio of the 2^nd frequency band energy to the 5^th frequency band energy Ratio of the 2^nd frequency band energy to the 6^th frequency band energy Ratio of the 2^nd frequency band energy to the 7^th frequency band energy Ratio of the 2^nd frequency band energy to the 8^th frequency band energy Ratio of the 2^nd frequency band energy to the 9^th frequency band energy Ratio of the 3^rd frequency band energy to the 4^th frequency band energy Ratio of the 3^rd frequency band energy to the 5^th frequency band energy Ratio of the 3^rd frequency band energy to the 6^th frequency band energy Ratio of the 3^rd frequency band energy to the 7^th frequency band energy Ratio of the 3^rd frequency band energy to the 8^th frequency band energy Ratio of the 3^rd frequency band energy to the 9^th frequency band energy Ratio of the 4^th frequency band energy to the 5^th frequency band energy Ratio of the 4^th frequency band energy to the 6^th frequency band energy Ratio of the 4^th frequency band energy to the 7^th frequency band energy Ratio of the 4^th frequency band energy to the 8^th frequency band energy Ratio of the 4^th frequency band energy to the 9^th frequency band energy Ratio of the 5^th frequency band energy to the 6^th frequency band energy Ratio of the 5^th frequency band energy to the 7^th frequency band energy Ratio of the 5^th frequency band energy to the 8^th frequency band energy Ratio of the 5^th frequency band energy to the 9^th frequency band energy Ratio of the 6^th frequency band energy to the 7^th frequency band energy Ratio of the 6^th frequency band energy to the 8^th frequency band energy Ratio of the 6^th frequency band energy to the 9^th frequency band energy Ratio of the 7^th frequency band energy to the 8^th frequency band energy Ratio of the 7^th frequency band energy to the 9^th frequency band energy Ratio of the 8^th frequency band energy to the 9^th frequency band energy Signal entropy Entropy of the 1^st frequency band Entropy of the 2^nd frequency band Entropy of the 3^rd frequency band Entropy of the 4^th frequency band Entropy of the 5^th frequency band Entropy of the 6^th frequency band Entropy of the 7^th frequency band Entropy of the 8^th frequency band Entropy of the 9^th frequency band Mutual information between the signals of the 1^st and the 2^nd frequency bands Mutual information between the signals of the 1^st and the 3^rd frequency bands Mutual information between the signals of the 1^st and the 4^th frequency bands Mmutual information between the signals of the 1^st and the 5^th frequency bands Mutual information between the signals of the 1^st and the 6^th frequency bands Mutual information between the signals of the 1^st and the 7^th frequency bands Mutual information between the signals of the 1^st and the 8^th frequency bands Mutual information between the signals of the 1^st and the 9^th frequency bands Mutual information between the signals of the 2^nd and the 3^rd frequency bands Mutual information between the signals of the 2^nd and the 4^th frequency bands Mutual information between the signals of the 2^nd and the 5^th frequency bands Mutual information between the signals of the 2^nd and the 6^th frequency bands Mutual information between the signals of the 2^nd and the 7^th frequency bands Mutual information between the signals of the 2^nd and the 8^th frequency bands Mutual information between the signals of the 2^nd and the 9^th frequency bands Mutual information between the signals of the 3^rd and the 4^th frequency bands Mutual information between the signals of the 3^rd and the 5^th frequency bands Mutual information between the signals of the 3^rd and the 6^th frequency bands Mutual information between the signals of the 3^rd and the 7^th frequency bands Mutual information between the signals of the 3^rd and the 8^th frequency bands Mutual information between the signals of the 3^rd and the 9^th frequency bands Mutual information between the signals of the 4^th and the 5^th frequency bands Mutual information between the signals of the 4^th and the 6^th frequency bands Mutual information between the signals of the 4^th and the 7^th frequency bands Mutual information between the signals of the 4^th and the 8^th frequency bands Mutual information between the signals of the 4^th and the 9^th frequency bands Mutual information between the signals of the 5^th and the 6^th frequency bands Mtual information between the signals of the 5^th and the 7^th frequency bands Mutual information between the signals of the 5^th and the 8^th frequency bands Mutual information between the signals of the 5^th and the 9^th frequency bands Mutual information between the signals of the 6^th and the 7^th frequency bands Mutual information between the signals of the 6^th and the 8^th frequency bands Mutual information between the signals of the 6^th and the 9^th frequency bands Mutual information between the signals of the 7^th and the 8^th frequency bands Mutual information between the signals of the 7^th and the 9^th frequency bands Mutual information between the signals of the 8^th and the 9^th frequency bands Minimum of signal in a given time interval Maximum of signal in a given time interval Mean value of the signal in a given time interval Median of the signal in a given time interval Standard deviation of the signal in a given time interval Variance of the signal in a given time interval Skewness in a given time interval; if the signal is constant, then skew = 0 Kurtosis in a given time interval; if the signal is constant, then Kurtosis = 0 The longest interval in which the signal value ranges from 0.25 to 0.75 of the maximum value of the given time interval.(That means that if 0 is the minimum value of the signal in the given time interval and 1 is the maximum value of the signal in the given time interval; therefore, it is the longest interval in which the signal value ranges between 0.25 and 0.75. The same applies to the following lines of the table). Relative time in which the signal value ranges between 0.25 and 0.75 of the maximum value of the given time interval. The number of intervals in which the signal value ranges between 0.25 and 0.75 of the maximum value of the given time interval. The longest interval in which the signal value ranges between 0.4 and 0.6 of the maximum value of the given time interval. Relative time in which the signal value ranges between 0.4 and 0.6 of the maximum value of the given time interval. The number of intervals in which the signal value ranges between 0.4 and 0.6 of the maximum value of the given time interval. The longest interval in which the signal value ranges between 0.45 and 0.55 of the maximum value of the given time interval. Relative time in which the signal value ranges between 0.45 and 0.55 of the maximum value of the given time interval. The number of intervals in which the signal value ranges between 0.45 and 0.55 of the maximum value of the given time interval. The longest interval in which the signal value ranges between 0.475 and 0.525 of the maximum value of the given time interval. Relative time in which the signal value ranges between 0.475 and 0.525 of the maximum value of the given time interval. The number of intervals in which the signal value ranges between 0.475 and 0.525 of the maximum value of the given time interval. The longest interval in which the signal value ranges between 0.49 and 0.51 of the maximum value of the given time interval. Relative time in which the signal value ranges between 0.49 and 0.51 of the maximum value of the given time interval. The number of intervals in which the signal value ranges between 0.49 and 0.51 of the maximum value of the given time interval. The longest interval in which the signal value ranges between 0.495 and 0.505 of the maximum value of the given time interval. Relative time in which the signal value ranges between 0.495 and 0.505 of the maximum value of the given time interval. The number of intervals in which the signal value ranges between 0.495 and 0.505 of the maximum value of the given time interval. Unique identifier

Note: A unique identifier for each sample (each row) of a predictive modeling table is used for data processing in the predictive modeling process.

Now, the predictive modeling itself follows. The next step includes a selection of relevant variables and creation of predictive models based on the data obtained. For example, the following can be used as predictive models: regression algorithms, decision trees or neural networks. The principle of the function of decision trees is a gradual division of the input data set into subsets, which differ from each other in the proportion of fatigued and alert operators. More detailed information on the algorithms used is published in, for example, the work by Breiman, L., et al., Classification and Regression Trees. 1984: Pacific Grove: Wadsworth. In Fig. 9, a procedure of predictive modeling in fatigue detection based on steering-wheel movement is shown. Description of Fig.9: The database table includes variables listed in individual columns. Each row of the table corresponds to a time interval of 1 minute of a drive. The data are divided into training, validation and test subsets in a 40-30-30% ratio. The first model is a decision tree. The model performs both a variable selection and a sample division depending on those variables (see below). The variables, which were selected by this model, are used as input variables for other models. Note: Variables are used only -no rules are used. Other models include neural networks, namely a single-layer network with 19 neurons, a network with 5 and 3 neurons in a hidden layer and a network with 7 and 5 neurons in a hidden layer. These models are denominated Nl 9, N5 3 and N7_5. It is suitable to name each of the models to distinguish which model is talked about. Then, individual models are compared to each other. For implementation in practice, the most successful model or combination of models is selected.

Model Performance Comparison:

The above mentioned table shows that the decision tree is not the best model. However, the model was used to select variables, which were used as input variables for the neural networks. A list of the variables, which was selected for the neural networks used is shown in the following table. Variable Description Vehicle speedsignal energy in frequency band from 1 to 1.25Hz Relative time in which the signal value of the steering wheel angular displacement ranges between 0.45 and 0.55 of the maximum value of the given time interval. Mean vehicle speed in a given time interval The longest interval in which the signal value of the steering-wheel angular displacement is within range from 0.475 to 0.525 of the maximum value in the given time interval Mutual information of the 2 and 9 frequency bands Signal energy of the frequency band from IHz to 1.25Hz related to the total signal energy Median of steering-wheel angular displacement The number of intervals in which the signal value of the steering wheel angular displacement ranges between 0.25 and 0.75 of the maximum value of the given time interval. Standard deviation of steering wheel angular displacement signal In Fig. 10, a diagram of the neural networks architecture used is shown. The model containing the best generalizing features and providing the best fatigue estimate is selected for a subsequent implementation. Atypical interconnections of neural network layers brought the best results on existing data. Neural network setting is as follows: 1) The input variables are, after subtraction of the arithmetic mean, normalized by standard deviation. The variables modified in this way forms the neural network inputs. 2) Hyperbolic tangent is used as a transfer function of the hidden layer. Weights and bias (threshold) are obtained, during the training of network, by a modification of back propagation training algorithm. 3) The output layer (identified as "Fatigue" - in the diagram) consists of a linear combination of the outputs from the hidden layer.

The value obtained is a real number, which, in the event of the network with 19 neurons, ranges from -13.01 to +5.73, with the "alert driver" prediction corresponds to the values les than 0 and the "fatigued driver" prediction corresponds to values higher than 0. The distance from zero is an approximate extent of the credibility of the operator condition modeled in the given time interval.

The model created can be used for the key component of the device to detect fatigue of an arbitrary motor vehicle driver -see the chapter dealing with the invention substance. The detection - see the previous example- can be carried out in real time or, to put it differently, on¬ line, directly during driving, without anyhow affecting the driver activities. A diagram of a possible implementation of the entire device is shown in Fig. 11. In this implementation, the device consists of the following parts: a signal pickup including, among other parts, a pickup to detect a steering-wheel angular displacement and vehicle speed as well as accelerometers to measure a longitudinal and lateral acceleration. The signals are subsequently preprocessed using, for example, a DSP processor. The fatigue assessment itself can be carried out using, for example, a suitable processor with a sufficient computing capacity, a microcomputer or palmtop or other programmable units. On those devices or processors, the signals are stored in a short-term memory and transformed, variables are generated, and assessment based on, for example, predictive models or decision rules is performed. After a summation processing of the results, the analogue or digital output can be transferred to an output device on which the driver is informed of his/her fatigue. The above stated description is only a single example of a fatigue detection implementation, however, there are many possibilities of implementing the fatigue detection based on this invention.

Advantages of the method based on this invention and, particularly, the device based on this invention are mainly the following: - detection credibility due to the use of entire useful information from more inputs of a given activity, which may include, for example, a steering wheel, speed and terrain. - real-time detection and assessment directly during vehicle driving, - detection credibility due to the use of data mining techniques, - continuous assessment of driver activities during driving, which cannot be cheated or bypassed. - very simple implementation which nowise affects driver activities or the inner space design of a motor vehicle. The fatigue detection device preferably contains a programmable unit, which is hidden, e.g. under the dashboard if driver fatigue is detected. To assess fatigue based on signals from, for example, a steering-wheel position pickup, accelerometer, detectors indicating a longitudinal or transverse acceleration, etc. Signals being measured or a method of measuring them are not included in the scope of this invention and nowise affect the invention scope. According to an advantageous embodiment the device is modified to improve the fatigue estimate accuracy. That can be accomplished by, for example, programming the programmable unit so that it also suitably cumulates the results of some directly preceding model outputs.

The operator can be acquainted with the measured extent of his/her fatigue in any suitable manner. For example, it is possible to indicate the fatigue extent e.g. by a signaling display color, digital digits of which the color will change to the red if a critical level is exceeded, acoustic signal or in a similar manner. Manners of operator fatigue signaling are not included in the scope of this invention and do not affect the scope of this invention anyhow. The device can be extended by a recording device to check the operator or assess and classify the cause of a possible accident or other additional devices of which the presence is suitable for purposes of a particular activity performed by the operator and consequences of his/her incapability of performing that activity in a reliable manner further.

The invention is utilizable wherever it is required to find operator incapability, due to fatigue in the common sense, of performing an activity or to warn of that incapability or to find operator quality in respect of performing a given activity. Herein, fatigue means any operator's physical or mental condition which causes the operator to insufficiently concentrate on the activity performed and, therefore, to threaten that activity. Examples of the activity performed by the operator include, motor vehicle drivers, pilots, helmsmen and various other operators, for example, in factories, nuclear power plants and similar plants. In addition, a fatigue test can be inserted to be performed, for example, before starting an operator activity itself, e.g. an operator at a nuclear power plant, who does not, in fact, perform any muscular activity, can be forced, before starting his/her activity, to undergo a test on a device employing the method based on this invention as well as, for example, after a period of performing his/her activity, the operator can be forced to undergo the test again to find his/her instant capability of performing the given activity.

Claims

C L A I M S

1. A method of operator fatigue or quality detection from operator's muscular activity, which method is characterized by detecting at least one parameter affected by operator's muscular activity, which parameter is evaluated based on fatigue evaluation rules obtained using a data mining method from a corresponding parameter of at least one operator for whom the extent of fatigue or quality is known.

2. A method according to Claim 1, characterized by generating variables from the parameter or parameters, said variables express the fatigue difference between a fatigued and alert operator and are compared using a data mining method, or the variables express the quality difference between a high-quality-operator and low-quality-operator and are compared using a data mining method.

3. A method according to Claim 2, which method is characterized by creating a fatigue detection model or a quality estimation model, preferably capable of generalization, from the variables using a data mining method, which model captures the relation between characteristics shown by an arbitrary operator during his/her activity and the extent of his/her fatigue or quality.

4. A method according to Claim 3, characterized in that, the fatigue detection model or the quality evaluation model is created based upon variables of more than 5 operators for whom the extent of fatigue or quality is known, the fatigue or the quality evaluation model is created as a set of decision rules designed to evaluate operator fatigue or to evaluate operator quality or to decide on whether the operator is capable of performing a given activity or not.

5. A method according to any of Claims 1 to 4, characterized in that the variables are generated from a motor vehicle steering-wheel movement being detected from any suitable part of the vehicle including the steering-wheel itself, the parts between the steering-wheel and the axle, or the axle itself.

6. A method according to Claim 5, characterized in that the variables are also generated from at least one of the following parameters: speed control related to an accelerator pedal movement, speed control related to a brake pedal movement, vehicle speed, vehicle lateral acceleration, vehicle longitudinal acceleration, windscreen wiper switched on as a rain indicator, vehicle loading, outside temperature values measured, and vehicle's position on the street.

7. A method according to any of Claims 1 to 4, characterized in that the variables are generated from the parameter of vehicle longitudinal and/or transverse acceleration.

8. A method of operator fatigue or quality evaluation from operator muscular activity, wherein: a) a) at least one parameter influenced by the operator muscular activity is detected or measured for at least one alert operator and at least one fatigued operator or at least one high-quality operator and at least one low-quality operator b) a set of data or preferably variables is obtained and stored in a suitable form, c) at least one parameter measured in step a) is measured for the evaluated operator, d) values measured in step c) are used to create data or preferably variables in the same or similar way as in step b), e) data or variables are compared with the set of the stored data or variables to find whether the operator is alert or fatigued or if the operator is of a high or of low quality.

9. A method of operator fatigue or quality evaluation from operator muscular activity, wherein: a) at least one parameter influenced by the operator muscular activity is detected for at least one alert operator and at least one fatigued operator or at least one high-quality operator and at least one low-quality operator b) a set of data or preferably variables is obtained and stored in a suitable form, c) based on the stored data or variables a fatigue detection model or a quality evaluation model is created using a data mining method, d) at least one parameter measured in step a) is measured for the evaluated operator, e) values measured in step d) are used to create data or preferably variables in the same or similar way as in step b) f) data or variables are used as input data for the model created in step c), whilst the model output provides the fatigue detection or quality estimation of the evaluated operator.

10. A device for fatigue or quality evaluation comprising a programmable unit, wherein the programmable unit includes fatigue or quality evaluation model for operator fatigue evaluation or for operator quality evaluation, the model being obtained by a data mining method using a signal of at least one of parameters generated by operator muscular activity for at least one fatigued operator and at least one alert operator or for at least one high-quality operator and at least one low-quality operator, the device further comprising at least one detector to measure or to measure-and-process the signal of the parameter or parameters used by the model, the detector or detectors are connected to the input of the programmable unit to evaluate operator fatigue or quality, the programmable unit output being connected to a fatigue or quality signaling device, while the device advantageously also comprises at least one unit for signal or data processing to generate suitable inputs for the fatigue or quality evaluation mode.

11. A device according to Claim 10, characterized by performing signal measuring, signal processing into the form of input data or variables of the detection model, and fatigue or quality estimation in real time, i.e. simultaneously with operator's routine activity.

12. A device according to Claims 10 or 11, characterized in that the programmable unit comprises a pre programmed fatigue detection model capable of generalization.