CN111839508A - Vehicle safe driving system based on mental state detection and control - Google Patents
Vehicle safe driving system based on mental state detection and control Download PDFInfo
- Publication number
- CN111839508A CN111839508A CN202010748670.7A CN202010748670A CN111839508A CN 111839508 A CN111839508 A CN 111839508A CN 202010748670 A CN202010748670 A CN 202010748670A CN 111839508 A CN111839508 A CN 111839508A
- Authority
- CN
- China
- Prior art keywords
- driver
- module
- mental state
- electroencephalogram
- vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/18—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state for vehicle drivers or machine operators
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
- A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7253—Details of waveform analysis characterised by using transforms
- A61B5/7257—Details of waveform analysis characterised by using transforms using Fourier transforms
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7253—Details of waveform analysis characterised by using transforms
- A61B5/726—Details of waveform analysis characterised by using transforms using Wavelet transforms
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient ; user input means
- A61B5/7405—Details of notification to user or communication with user or patient ; user input means using sound
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient ; user input means
- A61B5/742—Details of notification to user or communication with user or patient ; user input means using visual displays
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient ; user input means
- A61B5/746—Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
- A61M21/00—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
- B60Q9/008—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling for anti-collision purposes
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
- A61M21/00—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
- A61M2021/0005—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus
- A61M2021/0027—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus by the hearing sense
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
- B60W2040/0818—Inactivity or incapacity of driver
- B60W2040/0827—Inactivity or incapacity of driver due to sleepiness
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
- B60W2040/0818—Inactivity or incapacity of driver
- B60W2040/0836—Inactivity or incapacity of driver due to alcohol
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
- B60W2040/0872—Driver physiology
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Veterinary Medicine (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Public Health (AREA)
- Pathology (AREA)
- Medical Informatics (AREA)
- Surgery (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Psychiatry (AREA)
- Artificial Intelligence (AREA)
- Physiology (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Mathematical Physics (AREA)
- Mechanical Engineering (AREA)
- Psychology (AREA)
- Child & Adolescent Psychology (AREA)
- Evolutionary Computation (AREA)
- Hospice & Palliative Care (AREA)
- Educational Technology (AREA)
- Developmental Disabilities (AREA)
- Human Computer Interaction (AREA)
- Fuzzy Systems (AREA)
- Social Psychology (AREA)
- Acoustics & Sound (AREA)
- Anesthesiology (AREA)
- Transportation (AREA)
- Automation & Control Theory (AREA)
- Hematology (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention discloses a vehicle safe driving system based on mental state detection and control, which is characterized in that electroencephalogram signals of a driver are collected in real time through an electroencephalogram collection device, the mental state of the driver at present is analyzed in real time through an STFT-CNN mental state discrimination model trained in an MCU, when the detection device judges that the mental state of the driver is abnormal, a music player is immediately controlled to play electroencephalogram signals and music to change the electroencephalogram signals of the driver so as to change the mental state of the driver, when the mental state of the driver is still not available after a period of time, an acousto-optic alarm device inside and outside a vehicle is started to remind the driver of safe driving, and therefore passengers and vehicles coming and going are reminded of being in dangerous driving at present. The invention can help the driver to drive safely, avoid traffic accidents in the driving process of the vehicle, bring life danger and property loss to the driver and other people, and has strong practicability and popularization value.
Description
Technical Field
The invention belongs to the field of safe driving of vehicles, and particularly relates to a safe driving system of a vehicle based on mental state detection and control.
Background
The occurrence of traffic accidents not only causes property loss but also causes difficultly-compensated sadness to families and relatives, so safe driving should be the primary target of reducing accident rate and death rate. The main causes of traffic accidents include drunk driving, fatigue driving, road rage and other factors.
The mental condition of a driver directly influences the driving safety of a vehicle, but with the progress of society and the acceleration of life rhythm, people have to accelerate the pace and follow the development of society, so that the life pressure is increased, the incidence rate of insomnia of Chinese adults is up to 38.2 percent according to the statistics of Chinese sleep research society data, the number of sleep disorder people reaches 3 hundred million, and the sleep disorder people is increased year by year. The lack of sleep can cause the people to be distracted, fatigued and have large emotional fluctuation, and the danger coefficient of driving the vehicle by dragging the tired body of the driver is greatly increased and traffic accidents are easily caused.
At present, a plurality of music playing libraries classify music according to different scenes, different types are selected under different situations, the music library can improve emotion, but the effect is not good and the pertinence is not strong, the music is only played according to the libraries and is not interactive with personal emotion, the played music is not consistent with the emotion requirement, and further the emotion improving effect is not satisfactory.
Disclosure of Invention
The invention aims to provide a vehicle safe driving system based on mental state detection and control aiming at dangerous driving conditions in the driving process, and the system realizes online control or out-of-control warning for distinguishing and controlling the mental state of a driver to safely drive a vehicle.
In order to achieve the purpose, the invention adopts the following technical scheme:
a safe driving system for a vehicle based on mental state detection and control, comprising: the system comprises an electroencephalogram signal acquisition device, a preprocessing module, a drinking state identification module, an STFT-CNN mental state identification module, a drinking state control module, a mental state control module, an electroencephalogram signal music playing module, a vehicle out-of-control acousto-optic alarm module and a wearing detection module; the EEG signal acquisition device can acquire EEG signals (EEG) of a driver in real time and then carry out denoising treatment through pretreatment, the denoised EEG signals are sent into a Convolutional Neural Network (CNN) model through short-time Fourier transform (STFT) to distinguish the mental states of the driver such as emotion, fatigue degree and drinking, and when the abnormal mental states of the driver are detected, a music playing module or a vehicle out-of-control acousto-optic alarm module is used for regulating and reminding the driver; the safe driving system can monitor and control the state of the driver, so that the safe driving of the vehicle is guaranteed.
The electroencephalogram signal acquisition device and the preprocessing module are characterized by comprising an active electrode, a conditioning circuit, an AD conversion module, an MCU, a wireless module and a power module, wherein the preprocessing module adopts a wavelet threshold denoising algorithm, and uses a threshold method to denoise the electroencephalogram signals within the frequency band range of 0.5-30 Hz and remove external interference signals.
The self-made electroencephalogram signal acquisition device is characterized by being used for acquiring electroencephalogram signal information of a driver in real time; the collecting device is in a soft silica gel cap shape, 16 electrode leads are arranged in the collecting device, conductive materials such as conductive paste do not need to be smeared, the collecting device can be comfortably worn for a long time, and other limb behaviors are not influenced.
The electroencephalogram music playing module is characterized by being used for playing electroencephalogram music and controlling and improving the mental state of a driver; the module is composed of an electroencephalogram signal music storage module and a music player module, the music storage module mainly comprises a Micro SD card and is used for storing music made by electroencephalogram signals, and the music player module mainly comprises two sound devices which are respectively placed on seats on two sides of ears of a driver, so that the sound emitted can exert the best effect in the hearing range of the driver.
The drinking state identification module is characterized in that whether a driver has drinking behavior can be monitored in real time; the detection principle is that drinking can cause the EEG signal to be abnormal, and the abnormal EEG signal is mainly expressed as slowing of alpha wave (lower than 8.3 Hz) or losing of rhythm wave, the proportion of the alpha wave in the EEG signal after drinking is far lower than the normal ratio, the EEG signal of a driver is subjected to relative power calculation of characteristic waves by wavelet transformation, and drinking behaviors of the driver can be judged to be possibly existed greatly when the power value of the alpha wave is obviously reduced.
The mental state identification module is characterized in that whether the mental states of fatigue, sadness and anger of the driver occur or not can be monitored in real time; the mental state detection method comprises the steps that a driver watches videos which are processed to cause people to be sad and angry, electroencephalogram signals of the driver when watching different videos are collected, electroencephalogram signals of a fatigue state are collected to deprive the driver of partial sleep at night and drive a vehicle simulator for a long time, every 30s of the obtained electroencephalogram signals are subjected to denoising processing and marked with a mental state, sample data and a label are formed, the sample data are divided into a training set and a testing set and are sent into a convolutional neural network training model, so that three mental states can be accurately identified, and the model and parameter information are stored after training is finished.
The drinking state control module is characterized in that when electroencephalogram information is collected through the electroencephalogram collecting device and the electroencephalogram information is analyzed and detected to be abnormal due to the fact that drinking of a driver exists, the acousto-optic alarm device is started immediately to remind the driver, passengers and surrounding vehicles to inform that the vehicle is currently in dangerous driving, and the acousto-optic alarm device is installed inside the vehicle and outside the vehicle respectively.
The mental state control module is characterized in that an EEG acquisition device can monitor the mental state of a driver in real time, a music playing device is started immediately when the fact that the driver is tired, sad and angry is detected by analyzing EEG signals of the driver, brain waves of the brain are induced by vibration frequency of sound waves to generate entrainment of assimilation so as to change brain electric signals of the driver based on brain waves and acoustic principles, and acousto-optic alarm devices placed in and out of a vehicle are started immediately when the fact that the mental state of the driver is still not improved is detected so as to inform the vehicle of the current dangerous driving state, and the driver, passengers and surrounding vehicles are warned.
The wearing detection module is characterized in that when a driver does not wear the electroencephalogram acquisition device for a long time, the electroencephalogram signals which are not acquired by the device can be continuously in an unactivated state, and at the moment, the audible and visual alarm devices inside and outside the vehicle can be triggered to promote the driver to drive safely.
The electroencephalogram music is characterized in that the electroencephalogram music is made of electroencephalogram signals with good mental states (clear-headed, happy, pleasant and the like) for a driver; taking alpha wave as a main part and the interval between adjacent wave crests as the sound length, sampling the wave crests at equal intervals, and taking the average peak value of the section: u shapeAre all made ofWill U ist<UAre all made ofDiscarding by a threshold value method, wherein the amplitude of a sampling point is a pitch, and solving the power P of each sectionTAnd total signal power PGeneral assemblyThe ratio is: pt=PT/PGeneral assemblyAs the sound intensity.
The invention provides a vehicle safe driving system based on mental state detection and control, which is characterized in that electroencephalogram signals of a driver are collected in real time through an electroencephalogram collecting device, the current mental state of the driver is analyzed in real time through an MCU (microprogrammed control Unit), when the detection device judges that the mental state of the driver is abnormal, a music player is controlled to play electroencephalogram signals and music to change the electroencephalogram signals of the driver so as to change the mental state of the driver, when the mental state of the driver is continuously unchanged, an acousto-optic alarm device inside and outside a vehicle is started to remind the driver of safe driving, and meanwhile, passengers and vehicles coming and going are reminded of being in dangerous driving currently. The invention can help the driver to drive safely, avoid traffic accidents in the driving process of the vehicle, bring life danger and property loss to the driver and other people, and has stronger practicability.
Drawings
FIG. 1 is a block diagram of the present invention;
FIG. 2 is a hardware diagram of the electroencephalogram acquisition device;
FIG. 3 is a diagram of a brain electrical signal music generation system;
FIG. 4 is a diagram of a mental state discrimination system;
FIG. 5 is a mental state control process diagram;
FIG. 6 is a diagram of a vehicle runaway alarm module.
Detailed Description
The technical scheme of the invention is further explained by combining the attached drawings and the detailed implementation mode.
The invention mainly aims at the problem that the vehicle is out of control due to the fact that the vehicle enters an unconscious state due to fatigue and drowsiness in the process of driving the vehicle; the drinking causes the mental anesthesia of an individual, the consciousness is fuzzy, and at the moment, the safety accident is easy to happen when a vehicle is driven and the accident is often serious; the problem that drivers with road irritability (anger, impatience) and low mood (fatigue, sadness, pain) are in danger of driving due to loss of mood during driving is solved.
Referring to fig. 1, the vehicle safe driving system based on mental state detection and control provided by the invention comprises an electroencephalogram signal acquisition device, a preprocessing module, a drinking state identification module, an STFT-CNN mental state identification module, a drinking state control module, a mental state control module, an electroencephalogram signal music playing module, a vehicle out-of-control acousto-optic alarm module and a wearing detection module.
A vehicle safe driving system based on mental state detection and control can acquire electroencephalogram (EEG) of a driver in real time through an EEG signal acquisition device, then carry out denoising treatment through preprocessing, send the denoised EEG signal into a Convolutional Neural Network (CNN) model through short-time Fourier transform (STFT) to distinguish mental states such as emotion, fatigue degree and drinking of the driver, and use a music playing module or a vehicle out-of-control acousto-optic alarm module to regulate and remind the driver when the mental state of the driver is detected to be abnormal; the safe driving system can monitor and control the state of the driver, so that the safe driving of the vehicle is guaranteed.
Referring to fig. 2, the electroencephalogram signal acquisition device and the preprocessing module are composed of an active electrode, a conditioning circuit, an AD conversion, an MCU, a wireless module and a power module, and the preprocessing module performs denoising processing on the electroencephalogram signal in a frequency band range of 0.5-30 Hz by using a wavelet threshold denoising algorithm and a threshold method to remove external interference signals. The EEG signal acquisition module can be used for acquiring EEG signals of a driver, and the amplitude of the EEG signals is very weak compared with that of common digital AD input, so that the driving capability of the EEG signal acquisition module is improved by using an active probe to increase current output. In the later stage acquisition circuit, in order to avoid the situation that effective signals are submerged in noise due to excessive amplification factor, a multi-stage amplification filtering processing method is adopted, wherein a high-performance instrument differential input amplifier is adopted in the first stage processing, and the input impedance and CMRR of the signals are effectively improved. And a DRL feedback cancellation circuit commonly used in bioelectricity collection is introduced into the part, so that internal noise interference can be effectively eliminated. Meanwhile, a primary power frequency wave trap is added in the circuit to eliminate power frequency interference, a band-pass filter is added, and an EEG effective signal between 0.5 Hz and 40Hz is extracted. Because the whole circuit is finally sent into the AD of the MCU for analog-to-digital conversion, the whole module operational amplifier power supply adopts double power supplies to supply power, thereby avoiding level lifting and then carrying out subsequent processing in the singlechip.
The self-made electroencephalogram signal acquisition device is characterized by being used for acquiring electroencephalogram signal information of a driver in real time; the collecting device is in a soft silica gel cap shape, 16 electrode leads are arranged in the collecting device, conductive materials such as conductive paste do not need to be smeared, the collecting device can be comfortably worn for a long time, and other limb behaviors are not influenced.
The electroencephalogram music playing module is characterized by being used for playing electroencephalogram music and controlling and improving the mental state of a driver; the module is composed of an electroencephalogram signal music storage module and a music player module, the music storage module mainly comprises a Micro SD card and is used for storing music made by electroencephalogram signals, and the music player module mainly comprises two sound devices which are respectively placed on seats on two sides of ears of a driver, so that the sound emitted can exert the best effect in the hearing range of the driver.
The drinking state identification module is characterized in that whether a driver has drinking behavior can be monitored in real time; the detection principle is that drinking can cause the EEG signal to be abnormal, and the abnormal EEG signal is mainly expressed as slowing of alpha wave (lower than 8.3 Hz) or losing of rhythm wave, the proportion of the alpha wave in the EEG signal after drinking is far lower than the normal ratio, the EEG signal of a driver is subjected to relative power calculation of characteristic waves by wavelet transformation, and drinking behaviors of the driver can be judged to be possibly existed greatly when the power value of the alpha wave is obviously reduced.
The mental state identification module is characterized in that whether the mental states of fatigue, sadness and anger of the driver occur or not can be monitored in real time; the mental state detection method comprises the steps that a driver watches videos which are processed to cause people to be sad and angry, electroencephalogram signals of the driver when watching different videos are collected, electroencephalogram signals of a fatigue state are collected to deprive the driver of partial sleep at night and drive a vehicle simulator for a long time, every 30s of the obtained electroencephalogram signals are subjected to denoising processing and marked with a mental state, sample data and a label are formed, the sample data are divided into a training set and a testing set and are sent into a convolutional neural network training model, so that three mental states can be accurately identified, and the model and parameter information are stored after training is finished.
The drinking state control module is characterized in that when electroencephalogram information is collected through the electroencephalogram collecting device and the electroencephalogram information is analyzed and detected to be abnormal due to the fact that drinking of a driver exists, the acousto-optic alarm device is started immediately to remind the driver, passengers and surrounding vehicles to inform that the vehicle is currently in dangerous driving, and the acousto-optic alarm device is installed inside the vehicle and outside the vehicle respectively.
The mental state control module is characterized in that an EEG acquisition device can monitor the mental state of a driver in real time, a music playing device is started immediately when the fact that the driver is tired, sad and angry is detected by analyzing EEG signals of the driver, brain waves of the brain are induced by vibration frequency of sound waves to generate entrainment of assimilation so as to change brain electric signals of the driver based on brain waves and acoustic principles, and acousto-optic alarm devices placed in and out of a vehicle are started immediately when the fact that the mental state of the driver is still not improved is detected so as to inform the vehicle of the current dangerous driving state, and the driver, passengers and surrounding vehicles are warned.
The wearing detection module is characterized in that when a driver does not wear the electroencephalogram acquisition device for a long time, the electroencephalogram signals which are not acquired by the device can be continuously in an unactivated state, and at the moment, the audible and visual alarm devices inside and outside the vehicle can be triggered to promote the driver to drive safely.
The electroencephalogram music is characterized in that the electroencephalogram music is made of electroencephalogram signals with good mental states (clear-headed, happy, pleasant and the like) for a driver; taking alpha wave as a main part and the interval between adjacent wave crests as the sound length, sampling the wave crests at equal intervals, and taking the average peak value of the section: u shapeAre all made ofWill U ist<UAre all made ofDiscarding by a threshold value method, wherein the amplitude of a sampling point is a pitch, and solving the power P of each sectionTAnd total signal power PGeneral assemblyThe ratio is: pt=PT/PGeneral assemblyAs the sound intensity.
Referring to fig. 3, the electroencephalogram signal music generating system can be used for regulating and controlling the mental state of a driver, firstly three segments of music with cheerful rhythm and three segments of comedy movie clips are selected, the electroencephalogram signal collecting device worn and made by the driver under the condition that the sleep quality of the driver is good firstly listens to the music with cheerful rhythm, then the driver takes a rest and returns to be calm, and then the driver watches one segment of comedy movie clip, so that the electroencephalogram signal with good mental state of the driver under the passive stimulation is collected under the cross condition. Dividing the acquired preprocessed electroencephalogram signals into a section of signals according to every 3s, wherein 3s is a sound length, taking the average amplitude of each section as a pitch, taking the average power of each section as a sound intensity to synthesize electroencephalogram music, and storing the music in a Micro CD to form the electroencephalogram music.
Referring to fig. 4 (a), the mental state determining module can be used for determining the mental state of the driver in real time, and inviting three experimenters to mainly determine four mental states of sadness, anger, fatigue and drinking in order to ensure the data volume of the sample, and the implementation process is as follows:
(1) the method for acquiring the electroencephalogram signals in the sad and angry mental states is approximately the same as the method for acquiring the mental states of drivers, firstly, video clips of sadness and angry in two different types are selected and edited, three experimenters wear the electroencephalogram signal acquisition device to watch videos, the electroencephalogram signals of the experimenters watching different videos are acquired and recorded, the two electroencephalogram signals are separated, and one mental state is marked every 3 s;
(2) the electroencephalogram signals under the fatigue state are mainly sourced from two parts, the first part deprives the experimenter of partial sleep at night to enable the experimenter to be in the sleepy and fatigue state, at the moment, the experimenter drives the automobile simulator to collect electroencephalogram signals, the other part enables the experimenter to drive the automobile simulator for a long time on a single road to collect electroencephalogram signals of the experimenter, and the electroencephalogram signals are marked into a mental state according to 3 s;
(3) the electroencephalogram signal acquisition process in the drinking state is to enable experimenters to simulate drinking in a real environment, enable each experimenter to appropriately drink the wine to enable the experimenters to be in a moderate drunk state due to the difference of personal drinking capacity, acquire the electroencephalogram signals of the experimenters at the moment, analyze the electroencephalogram signals to reserve electroencephalogram signal segments which are generated by alcohol and act on the alcohol to eliminate the effect, and mark a mental state every 3 s;
(4) forming a two-dimensional time-frequency graph of a time domain and a frequency domain by short-time Fourier transform of the marked one-dimensional electroencephalogram signals in four states of every 3s, and using the two-dimensional time-frequency graph as sample and label data in four mental states of sadness, anger, fatigue and drinking;
(5) training and classifying data through the established convolutional neural network model to realize identification of mental states, dividing sample data and labels into a training set, a verification set and a test set according to a ratio of 8:1:1, wherein the training set is used as data input in a network training process to train the model, the verification set is used for continuously verifying a model parameter optimization network in the training process, and the test set is used for detecting the accuracy of the network model. The network structure is shown in fig. 4 (b) as three convolutional layers, three pooling layers and two fully-connected layers.
Referring to fig. 5, the mental state control module can be used for regulating and improving the mental state of the driver, electroencephalogram signals of the driver are collected in real time through an electroencephalogram collecting device worn by the driver in the driving process of the vehicle, the current mental state of the driver is judged through a mental state distinguishing model stored in the MCU, when the model judges that the driver is in a sadness state, an anger state or a fatigue state at present, the music player module is started immediately to play electroencephalogram music, and the mental state of the driver is regulated and improved through music formed by electroencephalogram signals of the driver.
Referring to fig. 6, the vehicle runaway alarm module can be used for immediately starting acousto-optic alarm devices installed inside and outside a vehicle to remind a driver, passengers and surrounding vehicles of informing that the vehicle is currently in dangerous driving when the mental state of the driver is continuously in a sad state, an angry state and a tired state or the driver is detected to have drinking behavior.
The method comprises the following steps:
step 1, firstly, collecting electroencephalogram signals when the consciousness state of a driver is clear and the mood is pleasant;
step 2, performing wavelet threshold denoising pretreatment on the acquired electroencephalogram signals, extracting parameter information such as amplitude, frequency and power of the signals, generating signals in a music form through corresponding relations such as parameter mapping, and storing the signals in a Micro CD card to form a brain wave music library;
step 3, a driver wears the head-wearing electroencephalogram acquisition device to acquire electroencephalogram signals of the driver in real time during driving;
step 4, sending the electroencephalogram signals collected in real time into an MCU module, and judging the current mental state of the driver;
step 5, when the abnormal mental state (sadness, anger and fatigue) of the driver is detected, the electroencephalogram music stored in the Micro CD card is immediately played through the music player, so that the mental state of the driver is promoted to be improved;
and 6, when the continuous abnormality of the mental state of the driver is detected or the driver is detected to be unconscious when drinking, the sound and light alarm device is immediately started to emit harsh sound and flash light, so that the prompting and warning effects on the driver, passengers and vehicles coming and going are generated.
The above-mentioned contents are only for illustrating the technical idea of the present invention, and the protection scope of the present invention is not limited thereby, and any modification made on the basis of the technical idea of the present invention falls within the protection scope of the claims of the present invention.
Claims (8)
1. A vehicle safe driving system based on mental state monitoring and control is characterized in that: the system comprises a wearing detection module, an electroencephalogram signal music playing module and a vehicle out-of-control acousto-optic alarm module, wherein the wearing detection module comprises an electroencephalogram signal acquisition device, a preprocessing module, a drinking state identification module, an STFT-CNN mental state identification module, a drinking state control module and a mental state control module; firstly, electroencephalogram signals of a driver are collected in real time through an electroencephalogram signal collecting device, then denoising processing is carried out through a preprocessing module, denoised EEG signals are sent to a convolutional neural network model in an STFT-CNN mental state identification module through short-time Fourier transform, the emotion, the fatigue degree and the drinking mental state of the driver are distinguished through a drinking state identification module, when the mental state of the driver is detected to be abnormal, the drinking state control module or the mental state control module is called, and the electroencephalogram signal music playing module or the vehicle out-of-control acousto-optic alarm module is used for regulating and reminding the driver.
2. The system for safely driving a vehicle based on mental state monitoring and control as claimed in claim 1, wherein: the electroencephalogram signal acquisition device is composed of an active electrode, a conditioning circuit, an AD conversion circuit, an MCU, a wireless module and a power module.
3. The system for safely driving a vehicle based on mental state monitoring and control as claimed in claim 2, wherein: the module can be used for collecting electroencephalogram information of a driver in real time; the collecting device is in a soft silica gel cap shape, and 16 electrode leads are arranged in the collecting device.
4. The system for safely driving a vehicle based on mental state monitoring and control as claimed in claim 3, wherein: the module can be used for playing electroencephalogram music and controlling and improving the mental state of a driver; the module consists of an electroencephalogram signal music storage module and a music player module, wherein the music storage module mainly comprises a Micro SD card for storing music made by electroencephalogram signals, and the music player module mainly comprises two sound devices which are respectively placed on seats on two sides of ears of a driver.
5. The system for safely driving a vehicle based on mental state monitoring and control as claimed in claim 4, wherein: when the electroencephalogram information is acquired through the electroencephalogram acquisition device and the driver is analyzed and detected to have electroencephalogram signal abnormality caused by drinking, the acousto-optic alarm device is started immediately to remind the driver, passengers and surrounding vehicles to inform that the vehicle is currently in dangerous driving, and the acousto-optic alarm device is mounted inside and outside the vehicle respectively.
6. The mental state recognition module of claim 5, wherein: the EEG acquisition device can monitor the mental state of a driver in real time, immediately start the music playing device when detecting that the driver is tired, sad and angry by analyzing the EEG signal of the driver, change the EEG signal of the driver by the entrainment principle that the brain wave of the brain is induced by the vibration frequency of the sound wave to generate assimilation phenomenon based on the brain wave and the acoustics principle, thereby changing the mental state of the driver, immediately start the acousto-optic alarm device arranged in and out of the vehicle when detecting that the mental state of the driver is still not improved, inform the vehicle of the current dangerous driving state, and warn the driver, passengers and surrounding vehicles.
7. The drinking status control module according to claim 6, wherein: when a driver does not wear the electroencephalogram acquisition device for a long time, the electroencephalogram signals which are not acquired by the device can be continuously in an unactivated state, and the acousto-optic alarm devices inside and outside the vehicle can be triggered at the moment.
8. The mental state control module of claim 7, wherein: the electroencephalogram music is made of electroencephalogram signals with good mental states of drivers; taking alpha wave as a main part and the interval between adjacent wave crests as the sound length, sampling the wave crests at equal intervals, and taking the average peak value of the section: u shapeAre all made ofWill U ist<UAre all made ofDiscarding by a threshold value method, wherein the amplitude of a sampling point is a pitch, and solving the power P of each sectionTAnd total signal power PGeneral assemblyThe ratio is: pt=PT/PGeneral assemblyAs the sound intensity.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010748670.7A CN111839508A (en) | 2020-07-30 | 2020-07-30 | Vehicle safe driving system based on mental state detection and control |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010748670.7A CN111839508A (en) | 2020-07-30 | 2020-07-30 | Vehicle safe driving system based on mental state detection and control |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111839508A true CN111839508A (en) | 2020-10-30 |
Family
ID=72946227
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010748670.7A Pending CN111839508A (en) | 2020-07-30 | 2020-07-30 | Vehicle safe driving system based on mental state detection and control |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111839508A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112451833A (en) * | 2020-11-10 | 2021-03-09 | 东北电力大学 | System and method for detecting and relieving driving fatigue based on personalized music recommender |
| CN113208624A (en) * | 2021-04-07 | 2021-08-06 | 北京脑陆科技有限公司 | Fatigue detection method and system based on convolutional neural network |
| CN114504330A (en) * | 2022-01-30 | 2022-05-17 | 天津大学 | Fatigue state monitoring system based on portable electroencephalogram acquisition head ring |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104841057A (en) * | 2015-05-08 | 2015-08-19 | 山东大学 | Electroencephalogram brainwave alpha based infrasonic wave driving fatigue relieving method and device |
| CN106073770A (en) * | 2016-07-28 | 2016-11-09 | 宇龙计算机通信科技(深圳)有限公司 | Control method and control system |
| CN106529421A (en) * | 2016-10-21 | 2017-03-22 | 燕山大学 | Emotion and fatigue detecting auxiliary driving system based on hybrid brain computer interface technology |
| KR20180124327A (en) * | 2017-05-11 | 2018-11-21 | 광운대학교 산학협력단 | wearable device and method for measuring the degree of drowsiness or concentration of the user |
| CN110422174A (en) * | 2018-04-26 | 2019-11-08 | 李尔公司 | Biometric sensor is merged to classify to Vehicular occupant state |
| CN110947075A (en) * | 2019-11-27 | 2020-04-03 | 华南理工大学 | Personalized mental state adjusting system and method based on brainwave music |
-
2020
- 2020-07-30 CN CN202010748670.7A patent/CN111839508A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104841057A (en) * | 2015-05-08 | 2015-08-19 | 山东大学 | Electroencephalogram brainwave alpha based infrasonic wave driving fatigue relieving method and device |
| CN106073770A (en) * | 2016-07-28 | 2016-11-09 | 宇龙计算机通信科技(深圳)有限公司 | Control method and control system |
| CN106529421A (en) * | 2016-10-21 | 2017-03-22 | 燕山大学 | Emotion and fatigue detecting auxiliary driving system based on hybrid brain computer interface technology |
| KR20180124327A (en) * | 2017-05-11 | 2018-11-21 | 광운대학교 산학협력단 | wearable device and method for measuring the degree of drowsiness or concentration of the user |
| CN110422174A (en) * | 2018-04-26 | 2019-11-08 | 李尔公司 | Biometric sensor is merged to classify to Vehicular occupant state |
| CN110947075A (en) * | 2019-11-27 | 2020-04-03 | 华南理工大学 | Personalized mental state adjusting system and method based on brainwave music |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112451833A (en) * | 2020-11-10 | 2021-03-09 | 东北电力大学 | System and method for detecting and relieving driving fatigue based on personalized music recommender |
| CN113208624A (en) * | 2021-04-07 | 2021-08-06 | 北京脑陆科技有限公司 | Fatigue detection method and system based on convolutional neural network |
| CN114504330A (en) * | 2022-01-30 | 2022-05-17 | 天津大学 | Fatigue state monitoring system based on portable electroencephalogram acquisition head ring |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111839508A (en) | Vehicle safe driving system based on mental state detection and control | |
| CN2770008Y (en) | Dozing detection alarm | |
| Murugan et al. | Detection and analysis: Driver state with electrocardiogram (ECG) | |
| Karuppusamy et al. | Multimodal system to detect driver fatigue using EEG, gyroscope, and image processing | |
| CN109498041B (en) | Driver road rage state identification method based on electroencephalogram and pulse information | |
| US20090171232A1 (en) | Drowsiness detection system | |
| CN110604565A (en) | Brain health screening method based on portable EEG equipment | |
| Thiessen | Disturbance of sleep by noise | |
| CN101243973A (en) | Method and apparatus for monitoring and awakening fatigue doze | |
| CN106114516A (en) | The angry driver behavior modeling of a kind of drive automatically people's characteristic and tampering devic | |
| Dong et al. | A WPCA-based method for detecting fatigue driving from EEG-based internet of vehicles system | |
| CN110751381A (en) | Road rage vehicle risk assessment and prevention and control method | |
| Aboalayon et al. | Multi-class SVM based on sleep stage identification using EEG signal | |
| CN113397548A (en) | Technique for separating driving emotion from media-induced emotion in driver monitoring system | |
| Purnamasari et al. | Mobile EEG based drowsiness detection using K-nearest neighbor | |
| CN110584657A (en) | Attention detection method and system | |
| Suman et al. | EOG based vigilance monitoring system | |
| Abdel-Rahman et al. | An affordable approach for detecting drivers' drowsiness using EEG signal analysis | |
| Zobaed et al. | Real time sleep onset detection from single channel EEG signal using block sample entropy | |
| CN111685779A (en) | Schizophrenia disorder screening method based on portable EEG equipment | |
| Zhu et al. | Heavy truck driver's drowsiness detection method using wearable eeg based on convolution neural network | |
| Fu et al. | Real-time alarm monitoring system for detecting driver fatigue in wireless areas | |
| Albalawi et al. | Single-channel real-time drowsiness detection based on electroencephalography | |
| Dey et al. | Automatic detection of drowsiness in EEG records based on time analysis | |
| RU2111134C1 (en) | Method of and system for providing vigilance of vehicle driver |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |