CN110123266A - A kind of maneuvering decision modeling method based on multi-modal physiologic information - Google Patents
A kind of maneuvering decision modeling method based on multi-modal physiologic information Download PDFInfo
- Publication number
- CN110123266A CN110123266A CN201910365772.8A CN201910365772A CN110123266A CN 110123266 A CN110123266 A CN 110123266A CN 201910365772 A CN201910365772 A CN 201910365772A CN 110123266 A CN110123266 A CN 110123266A
- Authority
- CN
- China
- Prior art keywords
- feature
- maneuvering
- modal
- eye movement
- eeg signals
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/11—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for measuring interpupillary distance or diameter of pupils
- A61B3/112—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for measuring interpupillary distance or diameter of pupils for measuring diameter of pupils
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/318—Heart-related electrical modalities, e.g. electrocardiography [ECG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/02—Preprocessing
- G06F2218/04—Denoising
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Physics & Mathematics (AREA)
- Animal Behavior & Ethology (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- Ophthalmology & Optometry (AREA)
- Pathology (AREA)
- Human Computer Interaction (AREA)
- Psychiatry (AREA)
- Psychology (AREA)
- Cardiology (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention discloses a kind of maneuvering decision modeling methods based on multi-modal physiologic information, multi-modal physiologic information is extracted during directly executing maneuver from people, carry out the building of model, both workload was not only reduced also not dependent on the Knowledge Discovery of computer independent of the summary of experience of domain expert, cost of labor is saved, and modeling pattern is changed into from rationality has the characteristics that perception, so that the model established has higher fidelity, closer to the behaviour decision making process of people；Also, maneuvering decision modeling is carried out using multi-modal physiologic information, can also solve the problems, such as that carrying out maneuvering decision modeling using single physiological signal has one-sidedness；Furthermore, it is the objectivity of its feature using a considerable advantage of multi-modal physiologic information, compared with traditional modeling pattern dependent on the total eliminant of domain expertise, the data collected are more true and reliable, also more can objectively reflect the true motor-driven decision process of people.
Description
Technical field
The present invention relates to life interaction, mathematical modeling, Human Engineering integration technology fields, more particularly to one kind to be based on multimode
The maneuvering decision modeling method of state physiologic information.
Background technique
The research contents of the behavior modeling of people includes perception, decision, planning, memory and the study etc. to environment.By
It is various informative in ambient enviroment, and complexity with higher, the modeling object being related to also have very much, for example, behavior it is motor-driven,
Maneuvering decision modeling etc..
In recent years, there are mainly two types of the methods of the behavior modeling of people: one is in traditional sense, people are by manually summarizing
Or the knowledge in analysis data discovery data；Another kind is knowledge based discovery technique, by computer self learning mechanization
Obtain the knowledge in data.
The method that traditional dependence expertise manually obtains behavioral data knowledge, has in terms of the acquisition of data knowledge
Certain difficulty needs to make full use of the knowledge summary finishing of related fields at rule, so to improve " fidelity " of model
And with the explosive growth of data volume, this manual type is hard to carry on.It relies on expertise and manually obtains behavior number
It is reported that the method known is difficult to overcome the difficulty of the acquisition of knowledge and expression, lack the benefit to time series attribute in behavioral data
With.
By the method that computer self learning generates behavioral data knowledge, although solving the work of people to a certain extent
Amount problem, but need to fully rely on computer and obtain behavioral data, calculating cost is too high, excessively rationality, lacks the spy of true people
Property, also, the behavioral data obtained is also required to artificial treatment, lacks entire thinking of the people in combat manoeuvre decision and changed
Journey.
Summary of the invention
In view of this, the present invention provides a kind of maneuvering decision modeling methods based on multi-modal physiologic information, to solve
Heavy workload existing for certainly existing maneuvering decision modeling method, the excessively low problem of rationality, fidelity.
Therefore, the present invention provides a kind of maneuvering decision modeling methods based on multi-modal physiologic information, including walk as follows
It is rapid:
S1: true man's immersion operation simulating scenes are built；
S2: experimental design is carried out to the acquisition of the multi-modal physiologic information；Wherein, the multi-modal physiologic information includes
EEG signals, eye movement signal and electrocardiosignal；
S3: the multi-modal physiologic information is acquired；
S4: the EEG signals of acquisition are pre-processed；
S5: to the feature of the eye movement signal of acquisition, the electrocardiosignal and the pretreated EEG signals
It extracts and screens；
S6: behavior maneuvering decision model is constructed by the way of support vector machines.
In one possible implementation, in above-mentioned maneuvering decision modeling method provided by the invention, step S2 is right
The acquisition of the multi-modal physiologic information carries out experimental design, specifically includes:
S21: recruiting subject, is screened according to the physiological condition of subject and task experience；
S22: task training is carried out to the subject filtered out, judges whether subject learns flight simulator within experimental period
Basic operation and the preset experimental duties of complete independently；If so, thening follow the steps S23；If it is not, then return step S21, continues
The new subject of equal amount is recruited until number is up to standard；
S23: carrying out preliminary experiment to subject, test to the training result of subject and to the feasibility of experimental design into
Performing check；
S24: formally testing subject, be sequentially completed each experimental duties according to the experimental sequence pre-established,
Acquire multi-modal Physiological Experiment data when people executes maneuvering decision.
In one possible implementation, in above-mentioned maneuvering decision modeling method provided by the invention, step S4 is right
The EEG signals of acquisition are pre-processed, and are specifically included:
S41: the EEG signals of acquisition are pre-processed using the Open-Source Tools case of MATLAB, are obtained muting
EEG signals；
S42: the muting EEG signals are stored.
In one possible implementation, in above-mentioned maneuvering decision modeling method provided by the invention, step S41,
The EEG signals of acquisition are pre-processed using the Open-Source Tools case of MATLAB, obtain muting EEG signals, are had
Body includes:
Electrode positioning, bandpass filtering, superposition are carried out to the EEG signals of acquisition using the Open-Source Tools case of MATLAB
Average, baseline correction refers to and independent component analysis again, obtains muting EEG signals.
In one possible implementation, in above-mentioned maneuvering decision modeling method provided by the invention, step S5 is right
The feature of the eye movement signal, the electrocardiosignal and the pretreated EEG signals that acquire is extracted and is sieved
Choosing, specifically includes:
S51: for the EEG signals of different maneuvering decisions, time-frequency characteristics extracting method, adaptive recurrence side is respectively adopted
Method, the method for common space mode and power spectrum analysis method are extracted and are screened to the feature of EEG signals；
S52: for the eye movement signal of different maneuvering decisions, to the blink rate feature of eye movement signal, watch rate feature attentively, average
Watch duration feature attentively and average pupil sizes feature is extracted and screened；
S53: for the electrocardiosignal of different maneuvering decisions, temporal analysis, frequency domain analysis and non-thread is respectively adopted
Property analytic approach is extracted and is screened to the feature of electrocardiosignal；
S54: the feature of the feature of the EEG signals after screening, the feature of eye movement signal and electrocardiosignal is converged
Always, group becomes multi-modal mixing physiological characteristic.
In one possible implementation, in above-mentioned maneuvering decision modeling method provided by the invention, step S52,
For the eye movement signal of different maneuvering decisions, to the blink rate feature of eye movement signal, rate feature, average fixation duration are watched attentively
Feature and average pupil sizes feature extract, and specifically include:
The blink rate f of eye movement is calculated using following formulabFeature:
Wherein, n represents blink total degree, and T represents task total time；
Rate f is watched attentively using following formula calculating eye movementgFeature:
Wherein, total degree is watched in m representative attentively；
The average fixation duration of eye movement is calculated using following formulaFeature:
Wherein, dfiRepresent the duration that i-th watches behavior attentively；
The average pupil sizes of eye movement are calculated using following formulaFeature:
Wherein, ldiIt represents i-th and watches the pupil diameter size measured during behavior attentively.
In one possible implementation, in above-mentioned maneuvering decision modeling method provided by the invention, step is being executed
Rapid S6 further includes following steps after constructing behavior maneuvering decision model by the way of support vector machines:
S7: model training is carried out to the behavior maneuvering decision model by the way of cross validation；
S8: the parameter of the behavior maneuvering decision model is optimized using the optimization algorithm of grid search.
Above-mentioned maneuvering decision modeling method provided by the invention extracts multimode during directly executing maneuver from people
State physiologic information carries out the building of model, both independent of the summary of experience of domain expert, also not dependent on the knowledge of computer
It was found that the workload of people can be reduced compared with traditional modeling pattern dependent on the total eliminant of domain expertise, people is saved
Work cost, compared with dependent on the modeling pattern of the Knowledge Discovery of computer, modeling pattern is changed into from rationality has perception
The characteristics of, so that the model established has higher fidelity, closer to the behaviour decision making process of people；Also, it utilizes multi-modal
Physiologic information carries out maneuvering decision modeling, can also solve to carry out maneuvering decision modeling using single physiological signal with one-sidedness
The problem of；In addition, being the objectivity of its feature using a considerable advantage of multi-modal physiologic information, depended on traditional
The modeling pattern of the total eliminant of domain expertise is compared, and the data collected are more true and reliable, also more can objectively be reflected
The true motor-driven decision process of people.
Detailed description of the invention
Fig. 1 is that the process of the maneuvering decision modeling method provided in an embodiment of the present invention based on multi-modal physiologic information is illustrated
Figure；
Fig. 2 be the maneuvering decision modeling method provided in an embodiment of the present invention based on multi-modal physiologic information flow chart it
One；
Fig. 3 is in the maneuvering decision modeling method provided in an embodiment of the present invention based on multi-modal physiologic information to multi-modal
The acquisition of physiologic information carries out the flow diagram of experimental design；
Fig. 4 be the maneuvering decision modeling method provided in an embodiment of the present invention based on multi-modal physiologic information flow chart it
Two；
Fig. 5 is in the maneuvering decision modeling method provided in an embodiment of the present invention based on multi-modal physiologic information based on brain electricity
The pretreatment process schematic diagram of signal；
Fig. 6 is in the maneuvering decision modeling method provided in an embodiment of the present invention based on multi-modal physiologic information based on brain electricity
The Technology Roadmap of the feature extraction of signal；
Fig. 7 is that eye movement is based in the maneuvering decision modeling method provided in an embodiment of the present invention based on multi-modal physiologic information
The Technology Roadmap of the feature extraction of signal；
Fig. 8 is that electrocardio is based in the maneuvering decision modeling method provided in an embodiment of the present invention based on multi-modal physiologic information
The Technology Roadmap of the feature extraction of signal；
Fig. 9 is that multimode is based in the maneuvering decision modeling method provided in an embodiment of the present invention based on multi-modal physiologic information
State mixes physiological characteristic composition figure；
Figure 10 is the maneuvering decision modeling method building using provided in an embodiment of the present invention based on multi-modal physiologic information
Maneuvering decision model input and output figure.
Specific embodiment
Below in conjunction with the attached drawing in the application embodiment, the technical solution in the application embodiment is carried out clear
Chu, complete description, it is clear that described embodiment is merely possible to illustrate, and is not intended to limit the application.
A kind of maneuvering decision modeling method based on multi-modal physiologic information provided in an embodiment of the present invention, process signal
Figure and flow chart difference are as depicted in figs. 1 and 2, include the following steps:
S1: true man's immersion operation simulating scenes are built；
S2: experimental design is carried out to the acquisition of multi-modal physiologic information；Wherein, multi-modal physiologic information includes brain telecommunications
Number, eye movement signal and electrocardiosignal；
S3: multi-modal physiologic information is acquired；
S4: the EEG signals of acquisition are pre-processed；
S5: the feature of the eye movement signal of acquisition, electrocardiosignal and pretreated EEG signals is extracted and is sieved
Choosing；
S6: behavior maneuvering decision model is constructed by the way of support vector machines.
Above-mentioned maneuvering decision modeling method provided in an embodiment of the present invention mentions during directly executing maneuver from people
Multi-modal physiologic information is taken, the building of model is carried out, both independent of the summary of experience of domain expert, also not dependent on computer
Knowledge Discovery can reduce the workload of people compared with traditional modeling pattern dependent on the total eliminant of domain expertise,
Cost of labor is saved, compared with dependent on the modeling pattern of the Knowledge Discovery of computer, modeling pattern is changed into tool from rationality
There is perception, so that the model established has higher fidelity, closer to the behaviour decision making process of people；Also, it utilizes
Multi-modal physiologic information carries out maneuvering decision modeling, can also solve to have using the progress maneuvering decision modeling of single physiological signal
The problem of one-sidedness；It is and traditional in addition, be the objectivity of its feature using a considerable advantage of multi-modal physiologic information
Modeling pattern dependent on the total eliminant of domain expertise is compared, and the data collected are more true and reliable, also more can be objective
The true motor-driven decision process of ground reflection people.Above-mentioned maneuvering decision modeling method provided in an embodiment of the present invention has benefited from neural work
The improvement of physiological signal collections equipment such as the fast development of Cheng Xue, especially EEG signals, eye movement signal and electrocardiosignal and
The great development of signal processing method promotes the detection of behavior physiologic information to study.
In the specific implementation, the step S1 in above-mentioned maneuvering decision modeling method provided in an embodiment of the present invention, to experiment
Simulating scenes are built, and provide the simulated environment of true man's immersion for subject, carry out effectively maneuvering decision with auxiliary subject.
Specifically, experiment simulation scene can be built using the experiment purpose that the maneuvering decision to people is modeled as starting point.With aircraft
For simulator, Aircraft Simulator includes visual system, air-combat simulator cockpit and computer network system.Visual system uses
Computer generated image system generates the view appearance outside fighter plane cockpit, mainly includes airfield runway, building, field and road
Equal topography and geomorphologies, can operation scene to complex condition, such as rainy day, snowy day, thunderstorm day and daytime, Night
Scene carries out analogue simulation.The appearance of air-combat simulator cockpit uses all cover cockpit, has various instrument face plates, manipulation in cabin
Device and unmanned plane seat；Wherein, instrument face plate can be divided into instrumentation module, center control according to functional structure and functional module
Platform module and control panel；Control panel is multifunction display, and main flight display panel is according to the positional symmetry cloth of driving
It sets；Manipulation device includes throttle lever, handle and foot-operated etc., is arranged according to the positional symmetry of driving.Computer network system is whole
The core of a Aircraft Simulator, hardware include host, interface and bus, and software includes software management, application software and supports soft
Part, including three what comes into a driver's computer, server computer and middle control machine computers, are cooperated by Ethernet, are carried out real-time
Data exchange, common cooperation completes flight simulation task.
In the specific implementation, the step S2 in above-mentioned maneuvering decision modeling method provided in an embodiment of the present invention is being executed,
When carrying out experimental design to the acquisition of multi-modal physiologic information, specific experiment flow may include subject recruit, subject training,
Preliminary experiment and formal experiment four-stage (as shown in Figure 3), step S2, as shown in figure 4, can specifically come in the following manner real
It is existing:
S21: recruiting subject, is screened according to the physiological condition of subject and task experience；
Specifically, since this experiment to the more demanding of subject and is only limitted to specific crowd, subject needs when recruiting
Strictly screened according to the physiological condition of subject and task experience；
S22: task training is carried out to the subject filtered out, judges whether subject learns flight simulator within experimental period
Basic operation and the preset experimental duties of complete independently；If so, thening follow the steps S23；If it is not, then return step S21, continues
The new subject of equal amount is recruited until number is up to standard；
Specifically, subject needs the basic operation of association's flight simulator within experimental period and energy complete independently is preset
Experimental duties, subject fails the case where smoothly completing training if it exists, needs to continue to recruit the new subject of equal amount until people
Number is up to standard；
S23: carrying out preliminary experiment to subject, test to the training result of subject and to the feasibility of experimental design into
Performing check；
Specifically, main to ping preliminary experiment control experimental period and experiment flow, and according to being carried out the case where preliminary experiment after
The fine tuning of afterflow journey improves；
S24: formally testing subject, be sequentially completed each experimental duties according to the experimental sequence pre-established,
Acquire multi-modal Physiological Experiment data when people executes maneuvering decision.
In the specific implementation, the step S4 in above-mentioned maneuvering decision modeling method provided in an embodiment of the present invention is being executed,
When being pre-processed to the EEG signals of acquisition, as shown in figure 4, can specifically be accomplished by the following way:
S41: pre-processing the EEG signals of acquisition using the Open-Source Tools case of MATLAB, obtains muting brain electricity
Signal；
Specifically, the Open-Source Tools case of MATLAB can be Letswave；
S42: muting EEG signals are stored；
Specifically, it can store as txt format.
In the specific implementation, the step S41 in above-mentioned maneuvering decision modeling method provided in an embodiment of the present invention is being executed,
The EEG signals of acquisition are pre-processed using the Open-Source Tools case of MATLAB, when obtaining muting EEG signals, specifically
The Open-Source Tools case that can use MATLAB carries out electrode positioning, bandpass filtering, superposed average, baseline to the EEG signals of acquisition
Correction refers to and independent component analysis again, pure muting EEG signals is obtained as far as possible, based on the pre- of EEG signals
Processing flow schematic diagram is as shown in Figure 5.
In the specific implementation, the step S5 in above-mentioned maneuvering decision modeling method provided in an embodiment of the present invention is being executed,
When the feature of the eye movement signal of acquisition, electrocardiosignal and pretreated EEG signals is extracted and screened, such as Fig. 4 institute
Show, can specifically be accomplished by the following way:
S51: for the EEG signals of different maneuvering decisions, time-frequency characteristics extracting method, adaptive recurrence side is respectively adopted
Method, the method for common space mode and power spectrum analysis method are extracted and are screened to the feature of EEG signals；
Specifically, when employed frequency analysis method when, the classical Fast Fourier Transform (FFT) method (FFT) of comparison, in short-term can be used
Fourier transform method (STFT), small wave converting method extract the feature of five frequency ranges of EEG signals, and the frequency domain character of extraction includes
Alpha wave, beta wave, delta wave, five wave bands of theta wave and gamma wave frequency subband.By comparing different time-frequency sides
The power of frequecy characteristic between method selects frequecy characteristic strongest alternative as feature, then again with adaptive regression model, altogether
Isospace mode, power spectrumanalysis, average energy value, the feature of variance analysis carry out the comparison of significant difference, select different machines
The spare feature as the feature after screening as model foundation between the feature of dynamic decision with significant difference, these sieves
The maneuvering decision that feature after choosing can effectively decode people is intended to.The technology path of feature extraction based on EEG signals is as schemed
Shown in 6；
S52: for the eye movement signal of different maneuvering decisions, to the blink rate feature of eye movement signal, watch rate feature attentively, average
Watch duration feature attentively and average pupil sizes feature is extracted and screened；
Specifically, the eye movement signal of different maneuvering decisions is another good index for decoding man-machine dynamic behavior.Eye movement letter
Number feature mainly include blink rate, pupil size, average fixation time of blinkpunkt etc..Work as in the process for executing maneuver
In, as people executes the acceleration of task intense strain, blink rate shows the trend of reduction.When completing same task, by
It can first be expanded in the pupil of the influence of time pressure, people；And with the progress of combat duty, people progressivelyes reach fatigue state, pupil
Hole can reduce instead.Therefore, blink rate finally chosen to the processing of eye movement signal, watch rate attentively, average fixation duration, average
Pupil diameter is as Main Analysis feature.The feature of the above eye movement signal is screened, selects to have between different maneuvering decisions
Have a feature of significant difference, the change of eye movement signal help to decode comprehensively the holding of attention when people executes maneuver,
Conversion and distribution condition.The technology path of feature extraction based on eye movement signal is as shown in Figure 7；
S53: for the electrocardiosignal of different maneuvering decisions, temporal analysis, frequency domain analysis and non-thread is respectively adopted
Property analytic approach is extracted and is screened to the feature of electrocardiosignal；
Specifically, man-machine dynamic another index is considered using the electrocardiosignal of different maneuvering decisions as decoding.Analyze heart rate
Variability metrics use three kinds of temporal analysis, frequency domain analysis and nonlinear analysis method solutions, to identify that behavior is determined
The maneuvering decision information contained in plan.The technology path of feature extraction based on electrocardiosignal is as shown in Figure 8；
The variation of the R -- R interval of electrocardiosignal, i.e. primary two waves of heartbeat are calculated by the discrete trend analysis method of statistics
Gap between peak is R -- R interval.Be respectively adopted temporal analysis and frequency domain analysis by electrocardiosignal be decomposed into it is a series of not
Co-energy, different frequency range ingredient simultaneously analyzes it, can effectively make up the heart rate variability behavioral characteristics of timing method missing, can determine
Amount judges sympathetic nerve and parasympathetic proportionality action, and effect is preferable in terms of index sensibility and specificity；
Each heart rate variability time-domain and frequency-domain feature for specifically introducing proposed adoption in table form below, such as table 1
With shown in table 2, the screening of the feature of electrocardiosignal has been carried out according to result.These heart rate variability features are physiologic information foundation
Maneuvering decision model is provided fundamental basis；
The common heart rate variability temporal signatures table of table 1
The common heart rate variability frequency domain character table of table 2
S54: the feature of the feature of the EEG signals after screening, the feature of eye movement signal and electrocardiosignal is converged
Always, group becomes multi-modal mixing physiological characteristic；As shown in figure 9, this provides basis for the foundation of the maneuvering decision model of step S5.
It should be noted that the key of modeling is the similarity degree with truth, the modeling of truth is not met
Also existing value is just lost.It is provided in an embodiment of the present invention under the premise of the fast development of nowadays physiological measurements technology
Above-mentioned maneuvering decision modeling method, can measure comprehensively people carry out maneuvering decision every physical signs, in addition to EEG signals,
Except eye movement signal and electrocardiosignal, skin electric signal, respiratory wave and functional near infrared spectrum etc. and maincenter mind can also be measured
Through the direct or indirect relevant physiological signal of system activity.Multi-modal physiology is handled using a variety of physiologic information processing methods to believe
Breath decodes decision process when human brain carries out information processing by the feature extraction to EEG signals, by eye movement signal
Feature extraction parses attention of people during execution task, decodes people's maneuvering decision by the feature extraction to electrocardiosignal
Psychologic status, the feature of multi-modal physiologic information combines the fusion feature being used as, and that reflects multi-modal behavioural characteristics
Diversity plays an important role in maneuvering decision modeling.Pass through EEG signals, eye movement signal and the heart to human body behavior
The multi-modal feature of electric signal is merged, and constructs behavior model, the maneuvering decision model that the present invention constructs is directly from the body of people
Upper acquisition maneuvering decision data had not both depended on the summary of domain expert or had not depended on the Knowledge Discovery of computer, and can overcome people
Behavior model fidelity it is low, excessively rationality the shortcomings that, can also solve utilize single physiological signal carry out maneuvering decision modeling
There is the problem of one-sidedness.
In the specific implementation, in the step S52 of above-mentioned maneuvering decision modeling method provided in an embodiment of the present invention, for not
With the eye movement signal of maneuvering decision, to the blink rate feature of eye movement signal, watch attentively rate feature, average fixation duration feature with
And during average pupil sizes feature extracts, blink rate refers to the number of winks in the unit time, and usually, people is lasting
The eye closing behavior of 70-500ms can be regarded as once blinking, and specifically can use the blink rate f that following formula calculates eye movementbIt is special
Sign:
Wherein, n represents blink total degree, and T represents task total time；
The rate of watching attentively of eye movement refers to fixation times in the unit time, the duration can will be considered as one not less than 100ms
Secondary to watch attentively, specifically can use following formula calculating eye movement watches rate f attentivelygFeature:
Wherein, total degree is watched in m representative attentively；
The average fixation duration of eye movement refers to averagely watches behavior duration attentively every time, specifically can use as follows
The average fixation duration of formula calculating eye movementFeature:
Wherein, dfiRepresent the duration that i-th watches behavior attentively；
The average pupil sizes of eye movement refer to that the mean value of whole pupil diameter size measurement results, eye movement acquire equipment a
Body carries out one-shot measurement to it during watching behavior attentively every time, specifically can be in favor of the average pupil sizes of following formula calculating eye movementFeature:
Wherein, ldiIt represents i-th and watches the pupil diameter size measured during behavior attentively.
In the specific implementation, in the step S6 of above-mentioned maneuvering decision modeling method provided in an embodiment of the present invention, using branch
The mode for holding vector machine constructs in behavior maneuvering decision model, and the mode of support vector machines is with good classification performance and remarkably
Generalization ability, the pseudocode of building process is as shown in table 3.
Table 3: support vector machine classifier pseudocode
In the specific implementation, the step S6 in above-mentioned maneuvering decision modeling method provided in an embodiment of the present invention is being executed,
After constructing behavior maneuvering decision model by the way of support vector machines, as shown in figure 4, can also include the following steps:
S7: model training is carried out to behavior maneuvering decision model by the way of cross validation；
Specifically, the over-fitting of maneuver modeling, model training are carried out by the way of cross validation in the process in order to prevent
Model training；
S8: the parameter of behavior maneuvering decision model is optimized using the optimization algorithm of grid search；In this way, can be into
One step improves recognition correct rate, more accurately establishes behavior maneuver modeling to multi-modal physiologic information.
From the point of view of model construction, the present invention is based on the maneuvering decision models of multi-modal physiologic information with maneuvering decision
The input for the maneuvering decision model that multi-modal physiology composite character is established as mode input, than traditional single physiological signal is more
Horn of plenty.Based on different maneuvering decisions, as the output of model, the results are shown in Figure 10.
Above-mentioned maneuvering decision modeling method provided in an embodiment of the present invention mentions during directly executing maneuver from people
Multi-modal physiologic information is taken, the building of model is carried out, both independent of the summary of experience of domain expert, also not dependent on computer
Knowledge Discovery can reduce the workload of people compared with traditional modeling pattern dependent on the total eliminant of domain expertise,
Cost of labor is saved, compared with dependent on the modeling pattern of the Knowledge Discovery of computer, modeling pattern is changed into tool from rationality
There is perception, so that the model established has higher fidelity, closer to the behaviour decision making process of people；Also, it utilizes
Multi-modal physiologic information carries out maneuvering decision modeling, can also solve to have using the progress maneuvering decision modeling of single physiological signal
The problem of one-sidedness；It is and traditional in addition, be the objectivity of its feature using a considerable advantage of multi-modal physiologic information
Modeling pattern dependent on the total eliminant of domain expertise is compared, and the data collected are more true and reliable, also more can be objective
The true motor-driven decision process of ground reflection people.Above-mentioned maneuvering decision modeling method provided in an embodiment of the present invention has benefited from neural work
The improvement of physiological signal collections equipment such as the fast development of Cheng Xue, especially EEG signals, eye movement signal and electrocardiosignal and
The great development of signal processing method promotes the detection of behavior physiologic information to study.
Obviously, various changes and modifications can be made to the invention without departing from essence of the invention by those skilled in the art
Mind and range.In this way, if these modifications and changes of the present invention belongs to the range of the claims in the present invention and its equivalent technologies
Within, then the present invention is also intended to include these modifications and variations.
Claims (7)
1. a kind of maneuvering decision modeling method based on multi-modal physiologic information, which comprises the steps of:
S1: true man's immersion operation simulating scenes are built；
S2: experimental design is carried out to the acquisition of the multi-modal physiologic information；Wherein, the multi-modal physiologic information includes brain electricity
Signal, eye movement signal and electrocardiosignal；
S3: the multi-modal physiologic information is acquired；
S4: the EEG signals of acquisition are pre-processed；
S5: the feature of the eye movement signal of acquisition, the electrocardiosignal and the pretreated EEG signals is carried out
It extracts and screens；
S6: behavior maneuvering decision model is constructed by the way of support vector machines.
2. maneuvering decision modeling method as described in claim 1, which is characterized in that step S2 believes the multi-modal physiology
The acquisition of breath carries out experimental design, specifically includes:
S21: recruiting subject, is screened according to the physiological condition of subject and task experience；
S22: task training is carried out to the subject filtered out, the base for whether learning flight simulator within experimental period judged to be tested
This operation and the preset experimental duties of complete independently；If so, thening follow the steps S23；If it is not, then return step S21, continues to recruit
The new subject of equal amount is until number is up to standard；
S23: preliminary experiment is carried out to subject, tests to the training result of subject and the feasibility of experimental design is examined
It tests；
S24: formally testing subject, be sequentially completed each experimental duties according to the experimental sequence pre-established, acquires
People executes multi-modal Physiological Experiment data when maneuvering decision.
3. maneuvering decision modeling method as described in claim 1, which is characterized in that step S4, to the brain telecommunications of acquisition
It number is pre-processed, is specifically included:
S41: pre-processing the EEG signals of acquisition using the Open-Source Tools case of MATLAB, obtains muting brain electricity
Signal；
S42: the muting EEG signals are stored.
4. maneuvering decision modeling method as claimed in claim 3, which is characterized in that step S41 utilizes the open source work of MATLAB
Tool case pre-processes the EEG signals of acquisition, obtains muting EEG signals, specifically includes:
Using MATLAB Open-Source Tools case to the EEG signals of acquisition carry out electrode positioning, bandpass filtering, superposed average,
Baseline correction refers to and independent component analysis again, obtains muting EEG signals.
5. maneuvering decision modeling method as described in claim 1, which is characterized in that step S5 believes the eye movement of acquisition
Number, the features of the electrocardiosignal and the pretreated EEG signals extract and screen, specifically include:
S51: for the EEG signals of different maneuvering decisions, time-frequency characteristics extracting method is respectively adopted, adaptive homing method, is total to
The method and power spectrum analysis method of isospace mode are extracted and are screened to the feature of EEG signals；
S52: for the eye movement signal of different maneuvering decisions, to the blink rate feature of eye movement signal, watch rate feature, average fixation attentively
Duration feature and average pupil sizes feature are extracted and are screened；
S53: for the electrocardiosignal of different maneuvering decisions, temporal analysis, frequency domain analysis and non-linear point is respectively adopted
Analysis method is extracted and is screened to the feature of electrocardiosignal；
S54: the feature of the feature of the EEG signals after screening, the feature of eye movement signal and electrocardiosignal is summarized, group
As multi-modal mixing physiological characteristic.
6. maneuvering decision modeling method as claimed in claim 5, which is characterized in that step S52, for different maneuvering decisions
Eye movement signal, to the blink rate feature of eye movement signal, to watch rate feature, average fixation duration feature and average pupil attentively straight
Diameter feature extracts, and specifically includes:
The blink rate f of eye movement is calculated using following formulabFeature:
Wherein, n represents blink total degree, and T represents task total time；
Rate f is watched attentively using following formula calculating eye movementgFeature:
Wherein, total degree is watched in m representative attentively；
The average fixation duration of eye movement is calculated using following formulaFeature:
Wherein, dfiRepresent the duration that i-th watches behavior attentively；
The average pupil sizes of eye movement are calculated using following formulaFeature:
Wherein, ldiIt represents i-th and watches the pupil diameter size measured during behavior attentively.
7. maneuvering decision modeling method as described in claim 1, which is characterized in that step S6 is being executed, using supporting vector
Further include following steps after the mode of machine constructs behavior maneuvering decision model:
S7: model training is carried out to the behavior maneuvering decision model by the way of cross validation；
S8: the parameter of the behavior maneuvering decision model is optimized using the optimization algorithm of grid search.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910365772.8A CN110123266B (en) | 2019-05-05 | 2019-05-05 | Maneuvering decision modeling method based on multi-modal physiological information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910365772.8A CN110123266B (en) | 2019-05-05 | 2019-05-05 | Maneuvering decision modeling method based on multi-modal physiological information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110123266A true CN110123266A (en) | 2019-08-16 |
| CN110123266B CN110123266B (en) | 2021-06-15 |
Family
ID=67576128
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910365772.8A Active CN110123266B (en) | 2019-05-05 | 2019-05-05 | Maneuvering decision modeling method based on multi-modal physiological information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110123266B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111067553A (en) * | 2019-12-30 | 2020-04-28 | 中国船舶工业综合技术经济研究院 | Human body efficiency experiment system for operating personnel under action of multiple environmental elements |
| CN111067552A (en) * | 2019-12-30 | 2020-04-28 | 中国船舶工业综合技术经济研究院 | System for measuring influence of illumination factors on operation performance of special shift workers |
| CN112043252A (en) * | 2020-10-10 | 2020-12-08 | 山东大学 | Emotion recognition system and method based on respiratory component in pulse signal |
| CN115120240A (en) * | 2022-08-30 | 2022-09-30 | 山东心法科技有限公司 | Sensitivity evaluation method, equipment and medium for special industry target perception skills |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103218929A (en) * | 2013-03-19 | 2013-07-24 | 哈尔滨工业大学 | In-spaceport-bin navigation analogy method and system based on head-down bed resting |
| CN104575155A (en) * | 2015-02-03 | 2015-04-29 | 扬州大学 | Driving ergonomics designing platform for all-windshield head-up display |
| CN104665849A (en) * | 2014-12-11 | 2015-06-03 | 西南交通大学 | Multi-physiological signal multi-model interaction-based high-speed railway dispatcher stress detecting method |
| CN107007291A (en) * | 2017-04-05 | 2017-08-04 | 天津大学 | Intense strain intensity identifying system and information processing method based on multi-physiological-parameter |
| CN107799165A (en) * | 2017-09-18 | 2018-03-13 | 华南理工大学 | A kind of psychological assessment method based on virtual reality technology |
| CN108509040A (en) * | 2018-03-28 | 2018-09-07 | 哈尔滨工业大学深圳研究生院 | Mixing brain machine interface system based on multidimensional processiug and adaptive learning |
| CN108904163A (en) * | 2018-06-22 | 2018-11-30 | 北京信息科技大学 | wheelchair control method and system |
-
2019
- 2019-05-05 CN CN201910365772.8A patent/CN110123266B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103218929A (en) * | 2013-03-19 | 2013-07-24 | 哈尔滨工业大学 | In-spaceport-bin navigation analogy method and system based on head-down bed resting |
| CN104665849A (en) * | 2014-12-11 | 2015-06-03 | 西南交通大学 | Multi-physiological signal multi-model interaction-based high-speed railway dispatcher stress detecting method |
| CN104575155A (en) * | 2015-02-03 | 2015-04-29 | 扬州大学 | Driving ergonomics designing platform for all-windshield head-up display |
| CN107007291A (en) * | 2017-04-05 | 2017-08-04 | 天津大学 | Intense strain intensity identifying system and information processing method based on multi-physiological-parameter |
| CN107799165A (en) * | 2017-09-18 | 2018-03-13 | 华南理工大学 | A kind of psychological assessment method based on virtual reality technology |
| CN108509040A (en) * | 2018-03-28 | 2018-09-07 | 哈尔滨工业大学深圳研究生院 | Mixing brain machine interface system based on multidimensional processiug and adaptive learning |
| CN108904163A (en) * | 2018-06-22 | 2018-11-30 | 北京信息科技大学 | wheelchair control method and system |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111067553A (en) * | 2019-12-30 | 2020-04-28 | 中国船舶工业综合技术经济研究院 | Human body efficiency experiment system for operating personnel under action of multiple environmental elements |
| CN111067552A (en) * | 2019-12-30 | 2020-04-28 | 中国船舶工业综合技术经济研究院 | System for measuring influence of illumination factors on operation performance of special shift workers |
| CN111067552B (en) * | 2019-12-30 | 2022-07-01 | 中国船舶工业综合技术经济研究院 | System for measuring influence of illumination factors on operation performance of special shift workers |
| CN112043252A (en) * | 2020-10-10 | 2020-12-08 | 山东大学 | Emotion recognition system and method based on respiratory component in pulse signal |
| CN115120240A (en) * | 2022-08-30 | 2022-09-30 | 山东心法科技有限公司 | Sensitivity evaluation method, equipment and medium for special industry target perception skills |
| CN115120240B (en) * | 2022-08-30 | 2022-12-02 | 山东心法科技有限公司 | Sensitivity evaluation method, equipment and medium for special industry target perception skills |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110123266B (en) | 2021-06-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110123266A (en) | A kind of maneuvering decision modeling method based on multi-modal physiologic information | |
| Shukla et al. | Feature extraction and selection for emotion recognition from electrodermal activity | |
| CN102715889B (en) | Mental load detection method | |
| CN109165556A (en) | One kind being based on GRNN personal identification method | |
| CN109299751B (en) | EMD data enhancement-based SSVEP electroencephalogram classification method of convolutional neural model | |
| CN106569604A (en) | Audiovisual dual-mode semantic matching and semantic mismatch co-stimulus brain-computer interface paradigm | |
| CN111783942B (en) | Brain cognitive process simulation method based on convolutional recurrent neural network | |
| CN112274162B (en) | Cross-tested EEG fatigue state classification method based on generation of anti-domain self-adaption | |
| CN111553618B (en) | Operation and control work efficiency analysis method, device and system | |
| CN114533086B (en) | Motor imagery brain electrolysis code method based on airspace characteristic time-frequency transformation | |
| CN104503916A (en) | Quantitative evaluation method for availability of system interface | |
| CN104182621A (en) | DBN based ADHD discriminatory analysis method | |
| CN111714118A (en) | Brain cognition model fusion method based on ensemble learning | |
| CN104035563A (en) | W-PCA (wavelet transform-principal component analysis) and non-supervision GHSOM (growing hierarchical self-organizing map) based electrocardiographic signal identification method | |
| CN104636890A (en) | Measurement method for workload of air traffic controller | |
| CN109480834A (en) | A kind of Method of EEG signals classification based on quick multiple dimension empirical mode decomposition | |
| CN109271887A (en) | A kind of composite space filtering and template matching method for the identification of brain power mode | |
| CN111598453A (en) | Control work efficiency analysis method, device and system based on execution force in virtual scene | |
| CN104490391A (en) | Combatant state monitoring system based on electroencephalogram signals | |
| CN113158964A (en) | Sleep staging method based on residual learning and multi-granularity feature fusion | |
| Chen et al. | Nonradial directional performance measurement with undesirable outputs: An application to OECD and non-OECD countries | |
| CN103488297B (en) | A kind of online semi-supervised character input system based on brain-computer interface and method | |
| CN117547270A (en) | Pilot cognitive load feedback system with multi-source data fusion | |
| Wang et al. | Statistical analysis of air traffic controllers’ eye movements | |
| CN110192860A (en) | A kind of the Brian Imaging intelligent test analyzing method and system of network-oriented information cognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |