CN114495069A - Method and system for monitoring driving state of driver - Google Patents
Method and system for monitoring driving state of driver Download PDFInfo
- Publication number
- CN114495069A CN114495069A CN202011164123.0A CN202011164123A CN114495069A CN 114495069 A CN114495069 A CN 114495069A CN 202011164123 A CN202011164123 A CN 202011164123A CN 114495069 A CN114495069 A CN 114495069A
- Authority
- CN
- China
- Prior art keywords
- driver
- fatigue
- driving
- eye
- index
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000012544 monitoring process Methods 0.000 title claims abstract description 28
- 230000007246 mechanism Effects 0.000 claims abstract description 6
- 230000008569 process Effects 0.000 claims description 18
- 238000001514 detection method Methods 0.000 claims description 17
- 238000001914 filtration Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 5
- 239000003086 colorant Substances 0.000 claims description 4
- 210000001747 pupil Anatomy 0.000 claims description 4
- 230000007547 defect Effects 0.000 abstract description 3
- 210000001508 eye Anatomy 0.000 description 65
- 210000005252 bulbus oculi Anatomy 0.000 description 14
- 230000009471 action Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 241000282414 Homo sapiens Species 0.000 description 3
- 239000011521 glass Substances 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 230000003111 delayed effect Effects 0.000 description 2
- 230000008034 disappearance Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000001815 facial effect Effects 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000035790 physiological processes and functions Effects 0.000 description 2
- 230000008433 psychological processes and functions Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 206010010071 Coma Diseases 0.000 description 1
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000004399 eye closure Effects 0.000 description 1
- 230000008921 facial expression Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 230000004297 night vision Effects 0.000 description 1
- 229920006395 saturated elastomer Polymers 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Emergency Management (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides a method and a system for monitoring the driving state of a driver, wherein the method collects fatigue sign data of the driver in a non-contact mode, calculates the initial driving state index of the driver according to the collected fatigue sign data, judges that the driver is in a suspected fatigue state when the index reaches a set first fatigue driving index, and then generates auxiliary marking information according to a set mechanism and displays the auxiliary marking information facing the driver; and then calculating the standard driving state index of the driver by combining the display state of the auxiliary indication information and the corresponding fatigue sign data, and judging the driver to be in a fatigue driving state and outputting warning information when the standard driving state index of the driver reaches a set second fatigue driving index. By adopting the scheme, the defect of complicated operation in the prior art can be overcome, the limitation of application scenes is broken through, the driving states of various vehicle drivers are accurately and efficiently monitored, the fatigue states of the drivers are timely recognized, and the fatigue driving is accurately and efficiently stopped.
Description
Technical Field
The invention relates to the technical field of state monitoring and evaluation, in particular to a method and a system for monitoring a driving state of a driver.
Background
The fatigue state of a train driver directly influences the driving safety, the imbalance of physiological functions and psychological functions of the driver after the driver drives the train continuously for a long time can influence the aspects of attention, perception, thinking, judgment, intention, movement and the like of the driver, the trend of descending driving skills can objectively appear, unsafe factors such as delayed action or early action, improper operation pause or correction time and the like can appear, even absentmindedness or instant memory disappearance can appear, and if the driver is not found in time due to fatigue or other problems in the driving process, a larger safety accident can be caused. The fatigue driving seriously threatens the traffic safety, and the fatigue driving behavior discrimination, the fatigue early warning and control technology and the like are gradually becoming important research directions of the traffic safety.
The device of monitoring driver fatigue on present train is still more original, and train driver need press with the hand or step on the alert button with the foot in every second, if do not in time press, then report to the police and remind, and this kind of mode has increased driver's driving operation burden on the one hand, and on the other hand probably has action inertia trigger button when driver is slightly tired, leads to the tired condition of unable in time discovery. In addition, some systems for judging the dozing state of the driver mainly collect facial expression information of the driver to judge the fatigue state of the driver are available at present, but the judgment is easy to cause misjudgment and has low accuracy because the expressions of everyone in fatigue are different.
The document with the application number of CN2019114041917 provides a fatigue driving early warning device, which detects a pressure value between a safety belt and a body of a driver and a heart rate value of the driver respectively through a specially arranged pressure sensor and a heart rate tester in a driving process, and generates an early warning signal according to the combination of the pressure value and the heart rate value, a pressure threshold value and a heart rate threshold value range by a central processing unit. The scheme can monitor data only based on physical contact, is similar to a wearable device scheme, has limitation, is only suitable for vehicles with safety belts, and is not suitable for drivers who do not need to wear the safety belts, such as train drivers.
Disclosure of Invention
To solve the above problems, the present invention provides a method of monitoring a driving state of a driver, which in one embodiment comprises:
s1, collecting various fatigue sign data of a driver in a non-contact mode after the vehicle is started;
step S2, determining the initial driving state index of the current driver according to the collected fatigue sign data, and executing the following step S3 when the initial driving state index of the driver reaches the set first fatigue driving index and the driver is judged to be in a suspected fatigue state;
step S3, generating auxiliary mark information according to a set mechanism and making the auxiliary mark information visually displayed for a driver;
and step S4, further determining the standard driving state index of the driver by combining the display state of the auxiliary indication information and the collected corresponding fatigue sign data, and when the standard driving state index of the driver reaches a set second fatigue driving index, judging that the driver is in a fatigue driving state, and immediately generating and outputting corresponding warning information.
Preferably, in the step S1, in one embodiment, fatigue sign data of the driver is collected by using an image collecting sensor, where the fatigue sign data includes a face parameter, a mouth state parameter, and an eye state parameter of the driver at a corresponding time during the driving of the vehicle.
Further, in one embodiment, the step S1 includes:
acquiring a face image of a driver, and respectively intercepting an eye region image and a mouth region image based on set eye feature points and mouth feature points;
performing mean filtering based on the intercepted eye region image and mouth region image to reduce the influence of noise information in the acquisition process, and obtaining an original eye region image and an original mouth region image;
and performing histogram equalization processing on the obtained original eye image and the original mouth image to obtain a standardized eye image and a standardized mouth image.
In one embodiment, in step S2, a PERCLOS parameter representing the eye opening and closing probability of the driver is selected as a preliminary driving state index for evaluating the current driving state of the driver;
and acquiring a PERCLOS value for judging suspected fatigue of the driver based on the data of the simulated driving, and using the PERCLOS value as a first fatigue driving index.
In one embodiment, in step S3, the auxiliary indication information is generated at set indication time intervals, starting from the time when the driver is determined to be likely to be fatigued.
Further, in one embodiment, in the step S3, the auxiliary indication information is displayed at a set position of the vehicle windshield by the projector according to the set indication display parameter, so that the auxiliary indication information is displayed for the driver.
In one embodiment, the method further comprises: presetting a marking time interval and marking display parameters applied to different driving time periods, wherein the marking display parameters comprise: marking a display track, marking a display color, and marking display brightness data.
In one embodiment, the step S4 includes:
on the basis of the obtained standardized eye images of each frame, HOG characteristics representing the opening and closing degree of the eye and the relative position of the pupil are respectively extracted;
detecting the standardized eye image by using the classifier model obtained by training, giving a corresponding detection result, and obtaining an eye identification result of the current frame by using a hysteresis comparison method according to the detection result;
and combining eye recognition results of a plurality of adjacent frames to carry out smooth filtering, and determining the current standard driving state index of the driver by integrating the mouth type recognition information.
Based on the methods provided in the foregoing embodiments, the present invention also provides a storage medium having stored thereon program codes that can implement the methods described in any one or more of the above embodiments.
In accordance with another aspect of any one or more of the above embodiments, the present invention also provides a system for monitoring a driving state of a driver, the system performing the method of any one or more of the above embodiments.
Compared with the closest prior art, the invention also has the following beneficial effects:
according to the method and the system for monitoring the driving state of the driver, provided by the invention, the fatigue sign data of the driver is acquired in a non-contact manner, a data basis is provided for identifying the fatigue degree of the driver, the defects of high cost, easiness in damage and limited use scene of a wearable monitoring device are overcome, and no interference is generated on the operation and the body feeling of the driver; in addition, the method calculates the initial driving state index of the driver according to the collected fatigue sign data, calculates the standard driving state index of the driver by integrating the auxiliary indication information and the corresponding fatigue sign data after determining that the driver is in the suspected fatigue state, and performs double-layer calculation through different mechanisms, thereby avoiding redundant data operation when the user is not tired yet, improving the timeliness of data processing on the basis of ensuring the accuracy of the monitoring result, avoiding traffic accidents caused by delay of the data result, and having higher reliability.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a schematic flow chart of a method for monitoring a driving status of a driver according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating the flow of the driver in the method for monitoring the driving status of the driver according to the present invention;
FIG. 3 is a diagram illustrating an example of projection of auxiliary sign information for a method of monitoring a driver's driving status according to another embodiment of the present invention;
FIG. 4 is a schematic diagram illustrating a fatigue status determination principle of a method for monitoring a driving status of a driver according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a system for monitoring a driving state of a driver according to an embodiment of the present invention.
Detailed Description
The following detailed description will be provided for the embodiments of the present invention with reference to the accompanying drawings and examples, so that the practitioner of the present invention can fully understand how to apply the technical means to solve the technical problems, achieve the technical effects, and implement the present invention according to the implementation procedures. It should be noted that, unless otherwise conflicting, the embodiments and features of the embodiments of the present invention may be combined with each other, and the technical solutions formed are all within the scope of the present invention.
The fatigue state of a train driver directly influences the driving safety, the imbalance of physiological functions and psychological functions of the driver after the driver drives the train continuously for a long time can influence the aspects of attention, perception, thinking, judgment, intention, movement and the like of the driver, the trend of descending driving skills can objectively appear, unsafe factors such as delayed action or early action, improper operation pause or correction time and the like can appear, even absentmindedness or instant memory disappearance can appear, and if the driver is not found in time due to fatigue or other problems in the driving process, a larger safety accident can be caused. The fatigue driving seriously threatens the traffic safety, and the fatigue driving behavior discrimination, the fatigue early warning and control technology and the like are gradually becoming important research directions of the traffic safety.
The device of monitoring driver fatigue on present train is still more original, and train driver need press with the hand or step on the alert button with the foot in every second, if do not in time press, then report to the police and remind, and this kind of mode has increased driver's driving operation burden on the one hand, and on the other hand probably has action inertia trigger button when driver is slightly tired, leads to the tired condition of unable in time discovery. In addition, some systems for judging the dozing state of the driver mainly collect information of the face and eyes of the driver to judge the fatigue state of the driver at present, but because the expressions of each person who is tired are different, the judgment is easy to cause misjudgment, and the accuracy is not high.
The document with the application number of CN2019114041917 provides a fatigue driving early warning device, which detects a pressure value between a safety belt and a body of a driver and a heart rate value of the driver respectively through a specially arranged pressure sensor and a heart rate tester in a driving process, and generates an early warning signal according to the combination of the pressure value and the heart rate value, a pressure threshold value and a heart rate threshold value range by a central processing unit. The scheme can monitor data only based on physical contact, is similar to a wearable device scheme, has limitation, is only suitable for vehicles with safety belts, and is not suitable for drivers who do not need to wear the safety belts, such as train drivers.
In the existing research developed around the requirements of fatigue driving identification, early warning, control and the like, the application of a novel detection and identification system is also involved, but the system has certain defects in the aspects of accuracy, reliability, anti-interference performance, miniaturization, engineering and the like, and in addition, new dangers are inevitably introduced to safe driving by contact type physiological parameter detection, for example, the technical scheme of acquiring pressure data and heart rate data of a driver through a sensor arranged on a safety belt in the patent document is adopted; the early warning and control technology is still developed, a corresponding mature technology is not formed, and the engineering application requirements are difficult to meet.
The authoritative study shows that when the human beings are converted from the normal state to the fatigue state, the sequence of fatigue formation is as follows: eyes, neck, shoulders and waist, and the fatigue of eyes can cause the fatigue of brain, seriously threaten traffic safety. Therefore, in order to solve the above-mentioned deficiencies in the prior art, the present invention provides a method and a system for monitoring the driving state of a driver, the method of the present invention collects the eye state data and the mouth state data of the driver in the driving process of the vehicle in real time according to the set logic, respectively calculates and identifies the driving state of the driver from two levels, and when the driver is identified and determined to be in a suspected fatigue state, the method adopts the set scheme to generate and display the auxiliary indication information to assist in identifying whether the driver is in the fatigue driving state. Various embodiments of the present invention will be described below with reference to the accompanying drawings.
Example one
Fig. 1 is a schematic flow chart illustrating a method for monitoring a driving status of a driver according to an embodiment of the present invention, and as can be seen from fig. 1, the method includes the following steps.
Step S110, collecting various fatigue sign data of a driver in a non-contact mode after a vehicle is started;
in the embodiment of the invention, the contact type or wearable type test device is considered to be easy to influence the driving operation of a driver and interfere the driving comfort of the driver, so as to interfere the driving or generate a conflicting emotion, and in addition, the existing test equipment is often attached to one or more components of a vehicle seat and related components, such as the inner side of a seat back or a seat safety belt, has high daily wearing frequency, is easy to damage and has a suddenly increased cost, and cannot be applied to the type of vehicle without safety belt setting, such as a rail train.
Therefore, the present invention starts from the direction of collecting data without contact, and does not need to provide a wearable device through image recognition to collect the physiological status data of the driver in real time without affecting the operation of the driver, so in step S110, the present invention collects the fatigue sign data of the driver by using the image collecting sensor according to the set time interval, wherein the fatigue sign data includes the face parameter, the mouth status parameter and the eye status parameter of the driver at the corresponding time during the driving process of the vehicle.
Considering that the process of human entering the fatigue state is eyes, neck, shoulders and waist in turn, in order to capture the fatigue state of the driver at the first time, the embodiment of the invention takes the eye state data of the driver as the core data, and can keep the time of dangerous driving of the driver to the minimum.
Fig. 2 shows a detailed flowchart of the implementation of the method for monitoring the driving status of the driver in the embodiment of the present invention, as shown in fig. 2, in one embodiment, the step S110 at least includes the following operations:
acquiring a face image of a driver, and respectively intercepting an eye region image and a mouth region image based on set eye feature points and mouth feature points;
performing mean filtering based on the intercepted eye region image and mouth region image to reduce the influence of noise information in the acquisition process and obtain an original eye image and an original mouth image;
and performing histogram equalization processing on the obtained original eye image and the original mouth image to obtain a standardized eye image and a standardized mouth image.
In order to quantify the fatigue degree of the driver, firstly, a judgment index of the fatigue degree needs to be determined, in one embodiment, the method comprises a step S120 of determining a preliminary driving state index of the current driver according to collected fatigue sign data, and when the preliminary driving state index of the driver reaches a set first fatigue driving index, determining that the driver is in a suspected fatigue state, and executing a step S130; if not, the driver is in a normal driving state.
According to the method, the accuracy is improved by adopting an active detection mode after suspected fatigue is judged, and the initial judgment is not required to be very comprehensive, so that in the initial judgment process, in order to properly control the pressure in the data processing process and improve the calculation efficiency, when the driver is in the suspected fatigue state is initially judged, in the step S120, a PERCLOS parameter representing the eye opening and closing probability of the driver is selected as an initial driving state index for evaluating the current driving state of the driver; perclos (percentage of eye closure over the facial over time), defined as the time per unit time (typically 1 minute or 30 seconds) that a certain percentage (70% or 80%) of the eyes are closed.
Among them, the earliest PERCLOS-based fatigue measurement system was established by Wierville in a 1994 primary driving simulator study. The U.S. federal highway administration (FHWA) and the U.S. National Highway Traffic Safety Administration (NHTSA) simulated driving in a laboratory, and compared nine fatigue detection indicators. The results prove that the methods can predict the driving fatigue in different degrees, and the correlation between the PERCLOS and the driving fatigue is the best, so that the PERCLOS is selected as the judgment index of the fatigue degree of the driver, and the specific calculation formula is as follows:
it is important to extract from the video sequence the degree to which the eyes are open without error, as can be seen from the definition of PERCLOS. To do this, we use the feature points extracted previously to locate the eye and intercept the eye image. And selecting the characteristic points of the canthus for each eye, correspondingly expanding the characteristic points, and simultaneously carrying out mean value filtering to reduce the influence of noise in detection so as to obtain the original eye image. In order to remove the influence of light, histogram equalization is performed on the acquired eye image to obtain a normalized eye image for detecting the degree of opening and closing. The HOG (Histogram of Oriented gradients) features of the eye image are extracted, the trained classifier model is used for detecting the eye image, and the probability of opening and closing is given. Using hysteresis comparison to give the judgment result of the frame, and combining the identification results of several adjacent frames to perform smooth filtering, therefore, the embodiment of the invention has the following steps: on the basis of the obtained standardized eye images of the frames, HOG features representing the opening and closing degrees of the eyes are respectively extracted;
detecting the standardized eye image by using the classifier model obtained by training, giving a corresponding detection result, and obtaining an eye identification result of the current frame by using a hysteresis comparison method according to the detection result;
and performing smooth filtering processing by combining eye identification results of a plurality of adjacent frames to obtain the current initial driving state index of the driver.
In practical application, before calculating the real-time PERCLOS value of the driver, the method further comprises the step of acquiring the PERCLOS value for judging suspected fatigue of the driver based on the data of the simulated driving, and using the PERCLOS value as a first fatigue driving index. In order to ensure that the preliminary driving state index calculated in the above step can cover all suspected fatigue conditions of the driver, the first fatigue driving index is set to a small value by a worker according to the actual identification requirement, so that the driver can be effectively identified just after entering the suspected fatigue state, and the accuracy of the identification result is ensured on the basis of improving the identification efficiency.
After determining that the driver is in a suspected fatigue state, the method adopts an active detection mode to improve the accuracy rate according to a set scheme, and therefore the method comprises the following steps: step S130, generating auxiliary marking information according to a set mechanism and enabling the auxiliary marking information to be visually displayed for a driver;
specifically, in one embodiment, in the step S130, the auxiliary indication information is generated at a set indication time interval, with a time when the driver is determined to be fatigued as a starting point.
And then, the auxiliary indication information is displayed at the set position of the vehicle windshield by using the projector according to the set identification display parameters, so that the auxiliary indication information is displayed for the driver, and it needs to be explained that the identification of the road condition by the driver is not influenced in the display process of the auxiliary indication information. Fig. 3 is a diagram showing an example of projection of auxiliary indication information in the method for monitoring the driving state of a driver according to the embodiment of the present invention, as shown in fig. 3, firstly displaying an indication 1, after the indication 1 stays for a set time, disappearing, as shown in the indication 1, displaying an indication 2 after 1 to 2 minutes, and similarly, sequentially displaying an indication 3 and an indication 4 at the back, wherein the indication displayed on the glass needs to be eye-catching, and a certain distance needs to be left between the indications; comprehensively judging and researching the tracking condition, if the eyeball correctly tracks 3 marks or more, judging that no fatigue exists, and otherwise, judging that the fatigue exists.
In an optional embodiment, the image acquisition sensor mainly acquires face information, mouth shape information and eye state information of a driver, the acquired information is sent to the main controller, and the main controller judges whether the driver is tired or not. If the driver is suspected to be tired, the main controller controls the auxiliary marking information to be thrown onto the front windshield through the short-focus projector, the image acquisition sensor tracks the rotation condition of eyeballs, if the eyeballs (eyeball) can move along with the marks on the windshield, the driver is judged not to be tired, and if the eyeball cannot move along with the marks, the driver is judged to be tired, as shown in fig. 4. It should be noted that, other effective and reasonable technologies may also be utilized to realize active feedback display of the auxiliary indication information, for example, a HUD projection mode is used to project the information to be tracked by the eyes onto the road surface in front of the HUD, and if the fatigue of the driver is detected, the driver is required to track the mark by the eyeball.
In practical applications, the method further comprises: presetting a marking time interval and marking display parameters applied to different driving time periods, wherein the marking display parameters comprise: marking display tracks, marking display colors and marking display brightness data. Considering that the environmental conditions and the physiological habits of human bodies are different in different driving periods, the invention sets different indication time intervals and indication display parameters based on different driving periods, for example, during the daytime (08:00-21:30), the display time interval of the setting marks is longer, such as 5min, the display track range of the setting marks is smaller, the display colors of the marks are saturated, and under the condition of sunlight in the daytime, it is necessary to increase the display brightness so that the driver can clearly see the position of the representation, the sign display brightness is high, and other time quantum at night, then set up correspondingly and mark with shorter time interval demonstration, increase the orbit scope that marks information display simultaneously, improve the degree of discernment, in order to promote driver's night vision experience degree in addition, the color and the luminance that set up the information of marking are all on the verge, and are not dazzling, promote to watch the comfort level.
In addition, in an optional embodiment, the display setting in the present invention may be flexibly adjusted by a user according to actual requirements, so as to ensure that the influence on the driving process of the driver is minimal on the premise of effectively implementing the auxiliary detection function.
Specifically, in one embodiment, the overall identification of the degree of driver fatigue is achieved in step S140, which includes: and further determining a standard driving state index of the driver by combining the display state of the auxiliary indication information and the collected corresponding fatigue sign data, and when the standard driving state index of the driver reaches a set second fatigue driving index, judging that the driver is in a fatigue driving state, and immediately generating and outputting corresponding warning information.
In the step S140, the method includes:
on the basis of the obtained standardized eye images of each frame, HOG characteristics representing the opening and closing degree of the eye and the relative position of the pupil are respectively extracted;
detecting the standardized eye image by using the classifier model obtained by training, giving a corresponding detection result, and obtaining an eye identification result of the current frame by using a hysteresis comparison method according to the detection result;
and combining eye identification results of a plurality of adjacent frames to perform smooth filtering processing, and determining a standard driving state index corresponding to the frame of the driver by integrating the mouth type identification information.
Further, the invention considers that when the driver is determined to be in a fatigue state, the driver is possibly in a more serious state, for example, the disease relapse causes coma, therefore, in the process of generating and outputting corresponding warning information, on one hand, the invention outputs the warning information such as voice, buzzing or music, on the other hand, the information is sent to the ground command center, the ground command center makes a decision and continuously obtains the facial fatigue sign data of the driver, if the driver is still in the fatigue state within the set time, and if the situation is determined to be serious, the ground command center takes over the vehicle to prevent the vehicle from being in an accident.
Based on the above embodiments, the present invention further provides a storage medium having stored thereon a program code that can implement the method described in any one or more of the above embodiments.
The fatigue state of a driver is judged by adopting a passive monitoring and active feedback mode, the marking information needing to be tracked by the eyes of the driver is thrown on the front windshield, the marks displayed on the eyeball tracking glass are judged by the rotation condition of the eyeball, the fatigue is judged, the identification rate can be effectively improved on the basis of ensuring the driving safety of the driver, the fatigue monitoring button does not need to be triggered manually, only the eyeball tracks the target on the glass when the fatigue state is monitored, and the operation burden of the driver is reduced.
By adopting the technical means provided by the embodiment of the invention, the fatigue sign data of the driver can be accurately acquired under the condition of no contact, the primary identification and standard identification processes are designed to judge the fatigue degree of the driver, and on the basis of effectively identifying the fatigue driving, the main controller for controlling and realizing the operation does not have great data processing burden, the corresponding speed is improved, and the abnormal probability is reduced.
Example two
In accordance with another aspect of any one or more of the above embodiments, the present invention also provides a system for monitoring the driving state of a driver, the system performing the method of any one or more of the above embodiments.
Specifically, fig. 5 is a schematic structural diagram of a system for monitoring a driving state of a driver according to an embodiment of the present invention, and as shown in fig. 5, the system includes:
the physical sign data acquisition device 51 is configured to acquire various fatigue physical sign data of the driver in a non-contact mode after the vehicle is started;
the preliminary identification module 53 is configured to determine a preliminary driving state index of the current driver according to the collected fatigue sign data, determine that the driver is in a suspected fatigue state when the preliminary driving state index of the driver reaches a set first fatigue driving index, and start the auxiliary marking module;
an auxiliary indication module 55 configured to generate auxiliary indication information according to a set mechanism and to visually display the auxiliary indication information to the driver;
and a standard identification module 57 configured to further determine a standard driving state index of the driver in combination with the display state of the auxiliary indication information and the collected corresponding fatigue sign data, and when the standard driving state index of the driver reaches a second set fatigue driving index, determine that the driver is in a fatigue driving state, and immediately generate and output corresponding warning information.
In one embodiment, the vital sign data acquisition device 51 is specifically configured to: the method comprises the steps of utilizing an image acquisition sensor to acquire fatigue sign data of a driver, wherein the fatigue sign data comprise face parameters, mouth state parameters and eye state parameters of the driver at corresponding moments in the driving process of a vehicle.
In an optional embodiment, the image acquisition sensor mainly acquires face information, mouth shape information and eye state information of a driver, the acquired information is sent to the main controller, and the main controller judges whether the driver is tired or not.
Further, in practical application, the physical sign data acquisition device performs the following operations:
acquiring a face image of a driver, and respectively intercepting an eye region image and a mouth region image based on set eye feature points and mouth feature points;
performing mean filtering based on the intercepted eye region image and mouth region image to reduce the influence of noise information in the acquisition process, and obtaining an original eye region image and an original mouth region image;
and performing histogram equalization processing on the obtained original eye image and the original mouth image to obtain a standardized eye image and a standardized mouth image.
In one embodiment, the preliminary identification module 53 is configured to: selecting a PERCLOS parameter representing the eye opening and closing probability of a driver as a preliminary driving state index for evaluating the current driving state of the driver;
and acquiring a PERCLOS value for judging suspected fatigue of the driver based on the data of the simulated driving, and using the PERCLOS value as a first fatigue driving index.
The auxiliary labeling module 55 is configured to: and generating auxiliary marking information according to the set marking time interval by taking the time for judging the suspected fatigue of the driver as a starting point.
In one embodiment, if it is determined that the driver is suspected to be tired, the auxiliary indication module displays the auxiliary indication information at a set position of a windshield of the vehicle according to the set indication display parameters by using a projector, so that the auxiliary indication information is displayed for the driver.
In practical application, if the driver is judged to be suspected to be tired, the main controller casts auxiliary marking information onto the front windshield through the projector, the image acquisition sensor tracks the eyeball rotation condition, if the eyeballs (the eyeball) can move along with the mark on the windshield, the driver is judged not to be tired, and if the eyeball cannot move along with the mark, the driver is judged to be tired. It should be noted that, other effective and reasonable technologies may also be utilized to realize active feedback display of the auxiliary indication information, for example, a HUD projection mode is used to project the information to be tracked by the eyes onto the road surface in front of the HUD, and if the fatigue of the driver is detected, the driver is required to track the mark by the eyeball.
In an alternative embodiment, the auxiliary indication module 55 is further configured to: presetting a marking time interval and marking display parameters applied to different driving time periods, wherein the marking display parameters comprise: marking display tracks, marking display colors and marking display brightness data.
Further, the standard identification module 57 is specifically configured to: on the basis of the obtained standardized eye images of each frame, HOG characteristics representing the opening and closing degree of the eye and the relative position of the pupil are respectively extracted;
detecting the standardized eye image by using the classifier model obtained by training, giving a corresponding detection result, and obtaining an eye identification result of the current frame by using a hysteresis comparison method according to the detection result;
and combining eye recognition results of a plurality of adjacent frames to carry out smooth filtering, and determining the current standard driving state index of the driver by integrating the mouth type recognition information.
In the system for monitoring the driving state of the driver, provided by the embodiment of the invention, each module or unit structure can be independently operated or operated in a combined mode according to actual monitoring requirements so as to realize corresponding technical effects.
It is to be understood that the disclosed embodiments of the invention are not limited to the particular structures, process steps, or materials disclosed herein but are extended to equivalents thereof as would be understood by those ordinarily skilled in the relevant arts. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting.
Reference in the specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. Thus, appearances of the phrase "an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment.
Although the embodiments of the present invention have been described above, the above descriptions are only for the convenience of understanding the present invention, and are not intended to limit the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (10)
1. A method of monitoring a driver's driving condition, the method comprising:
s1, collecting various fatigue sign data of a driver in a non-contact mode after the vehicle is started;
step S2, determining the initial driving state index of the current driver according to the collected fatigue sign data, and executing the following step S3 when the initial driving state index of the driver reaches the set first fatigue driving index and the driver is judged to be in a suspected fatigue state;
step S3, generating auxiliary mark information according to a set mechanism and making the auxiliary mark information visually displayed for a driver;
and step S4, further determining the standard driving state index of the driver by combining the display state of the auxiliary indication information and the collected corresponding fatigue sign data, and when the standard driving state index of the driver reaches a set second fatigue driving index, judging that the driver is in a fatigue driving state, and immediately generating and outputting corresponding warning information.
2. The method according to claim 1, wherein in step S1, an image capturing sensor is used to capture fatigue sign data of the driver, wherein the fatigue sign data includes face parameters, mouth state parameters and eye state parameters of the driver at corresponding time points during the driving of the vehicle.
3. The method according to claim 1 or 2, wherein in the step S1, comprising:
acquiring a face image of a driver, and respectively intercepting an eye region image and a mouth region image based on set eye feature points and mouth feature points;
performing mean filtering based on the intercepted eye region image and mouth region image to reduce the influence of noise information in the acquisition process, and obtaining an original eye region image and an original mouth region image;
and performing histogram equalization processing on the obtained original eye image and the original mouth image to obtain a standardized eye image and a standardized mouth image.
4. The method as claimed in claim 1, wherein in step S2, the PERCLOS parameter representing the eye opening and closing probability of the driver is selected as a preliminary driving state index for assessing the current driving state of the driver;
and acquiring a PERCLOS value for judging suspected fatigue of the driver based on the data of the simulated driving, and using the PERCLOS value as a first fatigue driving index.
5. The method according to claim 1, wherein in step S3, the assist indicator information is generated at a set indicator time interval starting from a time at which the driver is determined to be suspected to be fatigued.
6. The method according to claim 1, wherein in step S3, the assistant indicating information is displayed at a set position of a vehicle windshield according to the set indication display parameter by a projector so that the assistant indicating information is displayed for a driver.
7. The method of any one of claims 1 to 6, further comprising: presetting a marking time interval and marking display parameters applied to different driving time periods, wherein the marking display parameters comprise: marking display tracks, marking display colors and marking display brightness data.
8. The method according to claim 1, wherein in the step S4, the method comprises:
on the basis of the obtained standardized eye images of each frame, HOG characteristics representing the opening and closing degree of the eye and the relative position of the pupil are respectively extracted;
detecting the standardized eye image by using the classifier model obtained by training, giving a corresponding detection result, and obtaining an eye identification result of the current frame by using a hysteresis comparison method according to the detection result;
and combining eye recognition results of a plurality of adjacent frames to carry out smooth filtering, and determining the current standard driving state index of the driver by integrating the mouth type recognition information.
9. A storage medium having program code stored thereon for implementing the method of any one of claims 1-8.
10. A system for monitoring the driving state of a driver, wherein the system performs the method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011164123.0A CN114495069A (en) | 2020-10-27 | 2020-10-27 | Method and system for monitoring driving state of driver |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011164123.0A CN114495069A (en) | 2020-10-27 | 2020-10-27 | Method and system for monitoring driving state of driver |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114495069A true CN114495069A (en) | 2022-05-13 |
Family
ID=81471287
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011164123.0A Pending CN114495069A (en) | 2020-10-27 | 2020-10-27 | Method and system for monitoring driving state of driver |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114495069A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107633657A (en) * | 2016-07-18 | 2018-01-26 | 中兴通讯股份有限公司 | The based reminding method and terminal of a kind of fatigue driving |
| CN109145719A (en) * | 2018-07-06 | 2019-01-04 | 北京理工大学 | A kind of driver fatigue state recognition method and system |
| CN109326085A (en) * | 2018-11-08 | 2019-02-12 | 上海掌门科技有限公司 | A kind of method and apparatus for the progress fatigue driving detection on vehicle arrangement |
| CN109523749A (en) * | 2018-11-07 | 2019-03-26 | 长沙创恒机械设备有限公司 | A kind of method of anti-fatigue-driving, equipment and storage medium |
| CN110232310A (en) * | 2019-04-26 | 2019-09-13 | 平安科技（深圳）有限公司 | A kind of method for detecting fatigue driving neural network based and relevant device |
| US20200037945A1 (en) * | 2019-07-11 | 2020-02-06 | Lg Electronics Inc. | Drowsy-driving prevention method and drowsy-driving prevention system |
| WO2020078465A1 (en) * | 2018-10-19 | 2020-04-23 | 上海商汤智能科技有限公司 | Method and device for driving state analysis, driver monitoring system and vehicle |
| WO2020078464A1 (en) * | 2018-10-19 | 2020-04-23 | 上海商汤智能科技有限公司 | Driving state detection method and apparatus, driver monitoring system, and vehicle |
-
2020
- 2020-10-27 CN CN202011164123.0A patent/CN114495069A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107633657A (en) * | 2016-07-18 | 2018-01-26 | 中兴通讯股份有限公司 | The based reminding method and terminal of a kind of fatigue driving |
| CN109145719A (en) * | 2018-07-06 | 2019-01-04 | 北京理工大学 | A kind of driver fatigue state recognition method and system |
| WO2020078465A1 (en) * | 2018-10-19 | 2020-04-23 | 上海商汤智能科技有限公司 | Method and device for driving state analysis, driver monitoring system and vehicle |
| WO2020078464A1 (en) * | 2018-10-19 | 2020-04-23 | 上海商汤智能科技有限公司 | Driving state detection method and apparatus, driver monitoring system, and vehicle |
| CN109523749A (en) * | 2018-11-07 | 2019-03-26 | 长沙创恒机械设备有限公司 | A kind of method of anti-fatigue-driving, equipment and storage medium |
| CN109326085A (en) * | 2018-11-08 | 2019-02-12 | 上海掌门科技有限公司 | A kind of method and apparatus for the progress fatigue driving detection on vehicle arrangement |
| CN110232310A (en) * | 2019-04-26 | 2019-09-13 | 平安科技（深圳）有限公司 | A kind of method for detecting fatigue driving neural network based and relevant device |
| US20200037945A1 (en) * | 2019-07-11 | 2020-02-06 | Lg Electronics Inc. | Drowsy-driving prevention method and drowsy-driving prevention system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108446600A (en) | A kind of vehicle driver's fatigue monitoring early warning system and method | |
| CN104183091B (en) | System for adjusting sensitivity of fatigue driving early warning system in self-adaptive mode | |
| CN106585629B (en) | A kind of control method for vehicle and device | |
| KR101078466B1 (en) | Awake state judging model making device, awake state judging device, warning device, vehicle, awake state judging model making method, awake state judging method, computer readable medium recording the awake state judging model making method, and computer readable medium recording the awake state judging method | |
| CN101639894B (en) | Method for detecting train driver behavior and fatigue state on line and detection system thereof | |
| CN106585624B (en) | Driver status monitoring method and device | |
| Bergasa et al. | Real-time system for monitoring driver vigilance | |
| US8823792B2 (en) | Wakefulness level estimation apparatus | |
| Baccour et al. | Camera-based eye blink detection algorithm for assessing driver drowsiness | |
| CN105719431A (en) | Fatigue driving detection system | |
| CN102047304B (en) | Driver awareness degree judgment device, method, and program | |
| CN105574487A (en) | Facial feature based driver attention state detection method | |
| CN102752458A (en) | Driver fatigue detection mobile phone and unit | |
| CN109367539A (en) | A kind of intelligence system detecting fatigue driving | |
| CN1936988A (en) | Method and apparatus for alarming and recording doze of driver | |
| CN108021875A (en) | A kind of vehicle driver's personalization fatigue monitoring and method for early warning | |
| Anjali et al. | Real-time nonintrusive monitoring and detection of eye blinking in view of accident prevention due to drowsiness | |
| CN109543577A (en) | A kind of fatigue driving detection method for early warning based on facial expression feature | |
| Alam et al. | Active vision-based attention monitoring system for non-distracted driving | |
| CN109866774A (en) | System and method is monitored based on Face datection and the driver fatigue state of authentication | |
| Li et al. | Fatigue driving detection system based on face feature | |
| Bergasa et al. | Visual monitoring of driver inattention | |
| CN113744499B (en) | Fatigue early warning method, glasses, system and computer readable storage medium | |
| Girish et al. | Driver fatigue detection | |
| Bergasa et al. | Real-time system for monitoring driver vigilance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |