CN115153545A - Method and system for detecting emotion of person based on multi-modal physiological signals - Google Patents

Abstract

The invention provides a method and a system for detecting the emotion of a person based on a multi-modal physiological signal, which relate to the field of artificial intelligence, and comprise the following steps: the electrocardio-sensor and the skin resistance sensor are used for collecting electrocardiosignals and skin resistance signals of the epidermis of a human body. The method is used as input, and 3 physiological characteristic indexes of SDNN, SCR peak value and SCR frequency are calculated in real time. And on the basis of a dynamic change rule, comprehensively analyzing to obtain 2 indexes capable of evaluating the sudden change of the individual emotion and corresponding judgment threshold values, if the real-time emotion characterization index is larger than a threshold value, considering that the emotion of the person to be detected is fluctuated violently in a short time, and giving an alarm to the person to be detected by a system, otherwise, giving no alarm. The technical problem that micro emotion changes hidden by a person to be detected are difficult to detect accurately is solved. The technical effect that the physiological change of the person to be detected through the intention or behavior hiding is dynamically sensed, and the micro emotion change of the person to be detected is accurately detected is achieved.

Description

Personnel emotion detection method and system based on multi-modal physiological signals

Technical Field

The invention relates to the field of artificial intelligence, in particular to a method and a system for detecting emotion of a person based on a multi-mode physiological signal.

Background

For some special groups in the society, the emotional state of the person to be detected during the meeting activities can be sensed by effectively detecting the emotion, for example, state analysis, psychological diagnosis and treatment and the like of the person to be detected, physiological signals are collected in the process of physiological change caused by psychological change of the person to be detected under the influence of negative events, and the emotional runaway can be effectively monitored through the change of the physiological signals.

However, in the prior art, when the physiological change of the person to be detected is collected, it is difficult to dynamically perceive the physiological change hidden by the intention or behavior of the person to be detected, which results in a technical problem that it is difficult to accurately detect the micro-emotional change hidden by the person to be detected.

Disclosure of Invention

The application aims to provide a personnel emotion detection method and system based on multi-mode physiological signals, and the technical problem that when physiological changes of personnel to be detected are collected, the personnel to be detected are difficult to dynamically perceive through intention or behavior hidden physiological changes, and therefore micro emotion changes hidden by the personnel to be detected are difficult to accurately detect is solved. When the physiological changes of the personnel to be detected are collected, the dynamic perception of the physiological changes hidden by the emotion or behavior of the personnel to be detected through the SDNN index and the SCR index for evaluating the sudden emotion changes of the individual and the corresponding judgment threshold is realized, and further the technical effect of accurately detecting the micro emotion changes hidden by the personnel to be detected is realized.

In view of the above problems, the present application provides a method and a system for detecting a person's emotion based on a multi-modal physiological signal.

In a first aspect of the application, a method for detecting emotion of a person based on multi-modal physiological signals is provided, the method is applied to an emotion detection system, and the system is in communication connection with an electrocardio sensor and a skin resistance sensor, and the method comprises the following steps: acquiring electrocardiosignals of a target user by using the electrocardio sensor to obtain dynamic electrocardiosignal data; acquiring skin resistance signals of a target user by using the skin resistance sensor to obtain dynamic skin resistance signal data; preprocessing the dynamic electrocardiosignal data and the dynamic skin resistance signal data for a preset window long time period to respectively obtain an SDNN index and an SCR fusion index; judging whether the SDNN index meets a reduction situation; if the SDNN index meets the reduction situation, judging whether the SCR fusion index meets the increase situation synchronously; and if the SDNN index meets the reduction situation, the SCR fusion index synchronously meets the increase situation, and a first alarm instruction is triggered to alarm the emotional state of the target user.

In a second aspect of the application, there is provided a system for emotion detection of a person based on multi-modal physiological signals, the system comprising: the electrocardiosignal acquisition module is used for acquiring electrocardiosignals of a target user by utilizing an electrocardio sensor so as to obtain dynamic electrocardiosignal data; the skin resistance signal acquisition module is used for acquiring skin resistance signals of a target user by using the skin resistance sensor so as to obtain dynamic skin resistance signal data; the data preprocessing module is used for preprocessing the dynamic electrocardiosignal data and the dynamic skin resistance signal data for a preset window long time period so as to respectively obtain an SDNN index and an SCR fusion index; the SDNN index judging module is used for judging whether the SDNN indexes meet the reduction situation or not; the SCR fusion index judgment module is used for judging whether the SCR fusion index synchronously meets an increasing situation or not if the SDNN index meets a decreasing situation; and the state alarm module is used for triggering a first alarm instruction to alarm the emotional state of the target user if the SDNN index meets the reduction situation and the SCR fusion index meets the increase situation synchronously.

One or more technical solutions provided in the present application have at least the following technical effects or advantages:

according to the method for detecting the emotion of the person based on the multi-mode physiological signals, an Electrocardiogram (ECG) sensor and a skin resistance (GSR) sensor are arranged on the surface of the skin of the person to be detected, and the ECG signals and the skin resistance signals of the epidermis of a human body are collected. Electrocardio signals and skin resistance signals are used as input, and 3 physiological characteristic indexes of SDNN, SCR peak value and SCR frequency are calculated in real time. And based on the dynamic change rules of the 3 indexes, comprehensively analyzing to obtain 2 indexes capable of evaluating the sudden change of the individual emotion and corresponding judgment threshold values, if the real-time emotion characterization index is larger than the threshold value, considering that the emotion of the person to be detected is severely fluctuated in a short time, and giving an alarm by the system, otherwise, giving no alarm. When the physiological changes of the personnel to be detected are collected, the dynamic perception of the physiological changes hidden by the emotion or behavior of the personnel to be detected through the SDNN index and the SCR index for evaluating the sudden emotion changes of the individual and the corresponding judgment threshold is realized, and further the technical effect of accurately detecting the micro emotion changes hidden by the personnel to be detected is realized.

The foregoing description is only an overview of the technical solutions of the present application, and the present application can be implemented according to the content of the description in order to make the technical means of the present application more clearly understood, and the following detailed description of the present application is given in order to make the above and other objects, features, and advantages of the present application more clearly understandable.

Drawings

Fig. 1 is a schematic flow chart of a method for detecting emotion of a person based on a multi-modal physiological signal according to the present application;

fig. 2 is a schematic flow chart of preprocessing performed for a predetermined window period in a method for detecting emotion of a person based on multi-modal physiological signals according to the present application;

fig. 3 is a schematic flow chart illustrating a process of determining whether the SCR fusion indicator satisfies an increase situation in synchronization in the method for detecting a person emotion based on a multi-modal physiological signal according to the present application;

FIG. 4 is a schematic diagram of a system for detecting emotion of a person based on multi-modal physiological signals according to the present application;

Detailed Description

The technical problem that when physiological changes of a person to be detected are collected, the person to be detected is difficult to dynamically perceive the physiological changes hidden by intention or behavior in the prior art, so that micro-emotion changes hidden by the person to be detected are difficult to accurately detect is solved.

In view of the above technical problems, the technical solution provided by the present application has the following general idea:

the application provides a person emotion detection method based on multi-modal physiological signals. Because the personnel to be detected is easily influenced by negative events, an Electrocardiogram (ECG) sensor and a skin resistance (GSR) sensor are arranged on the skin surface of the personnel to be detected during the period of meeting activities of emotional runaway, and the electrocardio signals and the skin resistance signals of the epidermis of the human body are collected. Electrocardio signals and skin resistance signals are used as input, and 3 physiological characteristic indexes of SDNN, SCR peak value and SCR frequency are calculated in real time. And based on the dynamic change rules of the 3 indexes, comprehensively analyzing to obtain 2 indexes capable of evaluating the individual emotion abrupt change and corresponding judgment threshold values, if the real-time emotion characterization index is larger than the threshold value, considering that the emotion of the person to be detected is fluctuated violently in a short time, and giving an alarm to the person to be detected by the system, otherwise, giving no alarm.

Having described the basic principles of the present application, the technical solutions in the present application will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments of the present application, and the present application is not limited to the exemplary embodiments described herein. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without making any creative effort belong to the protection scope of the present application. It should be further noted that, for the convenience of description, only some but not all of the elements relevant to the present application are shown in the drawings.

Example one

As shown in fig. 1, the present application provides a method for detecting a person's emotion based on multi-modal physiological signals, the method is applied to an emotion detection system, and the system is in communication connection with an electrocardiogram sensor and a skin resistance sensor, and the method includes:

step S100: acquiring electrocardiosignals of a target user by using the electrocardio sensor to obtain dynamic electrocardio signal data;

step S200: acquiring skin resistance signals of a target user by using the skin resistance sensor so as to obtain dynamic skin resistance signal data;

specifically, the person to be detected exists as a special group in the society, the emotion detection is effectively carried out on the person to be detected, the emotion state of the person to be detected in the meeting activity period can be effectively sensed, physiological signals are collected in the process of physiological change caused by psychological change under the influence of negative events on the person to be detected, and the emotion runaway can be effectively monitored through the change of the physiological signals.

However, in the prior art, when the physiological change of the person to be detected is collected, it is difficult to dynamically perceive the physiological change hidden by the intention or behavior of the person to be detected, which results in a technical problem that it is difficult to accurately detect the micro-emotional change hidden by the person to be detected.

In order to solve the problems in the prior art, the application provides a method for detecting the emotion of a person based on a multi-modal physiological signal. Because the person to be detected is easily influenced by negative events, an Electrocardiogram (ECG) sensor and a skin resistance (GSR) sensor are arranged on the skin surface of the person to be detected during the period of meeting activities of the relatives with out-of-control emotion, and electrocardiosignals and skin resistance signals of the epidermis of a human body are acquired. Electrocardio signals and skin resistance signals are used as input, and 3 physiological characteristic indexes of SDNN, SCR peak value and SCR frequency are calculated in real time. And based on the dynamic change rules of the 3 indexes, comprehensively analyzing to obtain 2 indexes capable of evaluating the sudden change of the individual emotion and corresponding judgment threshold values, if the real-time emotion characterization index is larger than the threshold value, considering that the emotion of the person to be detected is severely fluctuated in a short time, and giving an alarm by the system, otherwise, giving no alarm. When the physiological change of the person to be detected is collected, the SDNN index and the SCR index of the sudden emotion change of the individual are evaluated, and the corresponding judgment threshold is reached, so that the person to be detected can dynamically sense the physiological change hidden by the intention or behavior, and the micro emotion change hidden by the person to be detected can be accurately detected.

Specifically, the electrocardio-sensor can be arranged on the skin surface of a person to be detected, and the electrocardio-sensor can dynamically sense the electrocardio-signal of the person to be detected, so that the change graph of the electrical activity generated in each cardiac cycle of the heart is recorded from the body surface, the dynamic electrocardio-signal data is the result obtained by dynamically and uninterruptedly collecting the electrocardio-signal of the person to be detected when meeting, and the skin resistance sensor and the GSR replace the epidermal skin galvanic reaction, so that the method is a method for measuring the skin conductance. The strong mood stimulates your sympathetic nervous system, causing the sweat glands to secrete more sweat. For example, painful stimuli such as needle sticks can cause sympathetic responses to the sweat glands, increasing sweat secretion. Although this increase is usually small, sweat contains water and electrolytes, increasing conductivity and thus decreasing the resistance of the skin. These changes in turn affect GSR. The dynamic skin resistance signal data reflects the result obtained by dynamically and uninterruptedly collecting the skin electric signal when the person to be detected meets the surface.

The adopted electrocardio-sensor obtains stable electrocardiosignals which can draw the heart beat change rule of a person to be detected under the scene of meeting in the familiarity, reflect the autonomic nerve activation condition of the body of the person to be detected and simultaneously reduce the influence of artifacts caused by limb movement to the maximum extent. The adopted skin resistance sensor obtains a skin resistance signal which can effectively represent the physiological and psychological awakening state of an individual under the influence of emotional fluctuation.

Step S300: preprocessing the dynamic electrocardiosignal data and the dynamic skin resistance signal data for a preset window long time period to respectively obtain an SDNN index and an SCR fusion index;

further, as shown in fig. 2, step S300 includes:

step S310: based on a peak extraction algorithm, performing adjacent peak collection on the dynamic electrocardiosignal data to obtain a peak interval of two adjacent R waves;

step S320: based on a sliding window calculation algorithm, dynamically calculating the peak interval of the two adjacent R waves for the preset window length time period so as to determine the SDNN index;

step S330: based on a feature extraction algorithm, carrying out SCR component extraction on the dynamic skin resistance signal data to obtain SCR components;

step S340: performing dynamic calculation on the SCR component for the preset window length time period by using the sliding window calculation algorithm to determine SCR peak value characteristics and SCR frequency characteristics;

step S350: and performing characteristic fusion on the SCR peak characteristic and the SCR frequency characteristic to determine the SCR fusion index.

Specifically, step S350 includes:

step S351: carrying out weighted average calculation on the indexes of the SCR peak value characteristic and the SCR frequency characteristic so as to obtain an SCR fusion index;

step S352: and performing first derivative calculation on the SCR fusion index to obtain a fusion index change slope.

In particular, after obtaining the dynamic electrocardiographic signal data and the dynamic skin resistance signal data, signal analysis may be performed based thereon. Firstly, data preprocessing can be carried out on the target signal, namely, target signal features required to be used for analysis are extracted and obtained through target feature extraction on collected signals, and then the target signal features are processed. Specifically, adjacent peak collection is performed on the dynamic electrocardiosignal data based on a peak lifting algorithm to obtain a peak interval of two adjacent R waves, and the collected electrocardiosignal is a fluctuation graph which dynamically reflects the electrocardio data of a person to be detected, so that the peak data in the fluctuation graph can be collected to obtain a point with the largest fluctuation of the electrocardio data of the person to be detected.

Further, the dynamic calculation of the predetermined window length period may be performed on the peak interval of the two adjacent R-waves based on a sliding window calculation algorithm to determine the SDNN index. The electrocardiosignal obtains two adjacent R wave crest intervals through a peak extraction algorithm, and the Standard Deviation (SDNN) characteristic of the beat intervals in a certain window long time is dynamically calculated in a sliding window calculation mode. The preset window long time period is the preset certain window long time. The sliding Window algorithm, similar to the hopping Window algorithm, is also used to control the traffic by limiting the maximum number of cells that can be received in each time Window. The difference is that in the sliding window algorithm, the time window is not a forward jump, but a forward slide every one cell time, and the length of the slide is one cell time. Wherein, the fixed window comprises a set of several cell times. SDNN is an indicator of heart rate variability, and refers to the standard deviation of all sinus heart beat RR intervals (NN intervals for short), in microseconds. The larger the SDNN, the greater the variability in heart rate. Generally, the smaller the age, the larger the SDNN. When SDNN is below normal, it suggests that there may be a manifestation of decreased parasympathetic activity, i.e., decreased vagal tone activity. The SDNN index can be used for reflecting the heart rate variability of the person to be detected.

Meanwhile, SCR component extraction can be carried out on the dynamic skin resistance signal data based on a feature extraction algorithm to obtain SCR components, and the SCR components are subjected to dynamic calculation of the preset window long time period by utilizing the sliding window calculation algorithm to determine SCR peak value features and SCR frequency features. The skin resistance signal is calculated in a sliding window mode, a Skin Conductance Response (SCR) component is obtained through a feature extraction algorithm, and the peak value of SCR in a certain window for a long time and the frequency of SCR in the window for a long time are dynamically calculated. Among them, the GSR signal is mainly composed of slowly varying basal activity-skin conductance levels and rapidly varying phase activity-skin conductance responses. The phase response of the skin electrical signal is above the base level, with higher amplitude and faster speed, and is shown in the form of a "GSR burst" or "GSR peak". The phase response is a transient, rapid fluctuation in the level of skin conductance, a physiologically psychologically activated state caused by a stimulus. When a person presents with stimulation, the conductance is increased, and a waveform skin conductance response is formed. The Skin Conductance Response (SCR) refers to the periodic sympathetic nerve discharge, which is the physiological and psychological activation state caused by stimulation, reflects the index of short-term brain processes, and can be used as a positioning marker of emotional arousal points generated by external new and abnormal event stimulation. SCR is sensitive to specific emotional stimuli events, and event-related skin conductance responses (ER-SCRs) can burst between 1 and 5 seconds after emotional stimuli; non-specific skin conductance responses (NS-SCRs) occur spontaneously in humans at a rate of 1-3 minutes, independent of any irritation. The SCR component reflects the psychophysiological ability of the person to be detected to respond to sensitive events at the meeting. The SCR peak value characteristic reflects a psychological reaction extreme point of the person to be detected for an external sensitive event, and the SCR frequency characteristic reflects the frequency of the reaction extreme point.

After determining the SCR peak characteristics and the SCR frequency characteristics, the SCR peak characteristics and the SCR frequency characteristics may be feature-fused for comprehensive analysis thereof. Specifically, the weighted average calculation of the indexes is performed on the SCR peak characteristics and the SCR frequency characteristics to obtain the SCR fusion index. The SCR fusion index reflects the psychological reaction extreme point and the occurrence frequency of the psychological reaction extreme point in the person to be detected, and further, the SCR fusion index can be subjected to first derivative calculation to obtain the change slope of the fusion index. By first derivative, calculus term, the first derivative represents the rate of change of a function, and the most intuitive expression lies in the monotonicity theorem of the function. The fusion index change slope reflects the change slope of the fused index, and generally, when the slope is higher, the change is faster. In conclusion, the SCR peak index and the SCR frequency index can be subjected to weighted average calculation to obtain the SCR fusion index, and the change slope of the fusion index is obtained by calculating the first-order reciprocal of the fusion index in real time and is used as a threshold value for judging instantaneous and large change of emotion in the follow-up process.

Step S400: judging whether the SDNN index meets a reduction situation;

step S500: if the SDNN index meets the reduction situation, judging whether the SCR fusion index meets the increase situation synchronously;

step S600: and if the SDNN index meets the reduction situation, the SCR fusion index synchronously meets the increase situation, and a first alarm instruction is triggered to alarm the emotional state of the target user.

Further, as shown in fig. 3, step S500 includes:

step S510: judging whether the SDNN index is lower than a preset index standard deviation threshold value or not;

step S520: if the SDNN index is lower than the preset index standard deviation threshold, judging the change characteristic of the change slope of the fusion index;

step S530: judging whether the change slope of the fusion index is synchronously higher than a preset slope standard deviation threshold value;

step S540: and if the change slope of the fusion index is synchronously higher than the preset slope standard deviation threshold, triggering the first alarm instruction to alarm the emotional state of the target user.

Specifically, after obtaining the SDNN index and the SCR fusion index, it is necessary to determine the change situation. Specifically, it is first determined whether the SDNN index is lower than a preset index standard deviation threshold, where the preset index standard deviation threshold may be understood as a preset standard deviation threshold of the SDNN index, and when the SCNN index of the person to be detected decreases (the standard deviation of the SDNN index is lower than the threshold in a certain time interval), it indicates that the activity of sympathetic nerves increases due to reasons such as stress, pressure, and arousal and increase of emotion of the person to be detected, and the body is in an excited state. Meanwhile, whether the change slope of the fusion index is synchronously higher than a preset slope standard deviation threshold can be judged, wherein the preset slope standard deviation threshold can be understood as a slope standard deviation threshold in a preset certain time interval, and at the moment, when the change slope of the SCR fusion index is synchronously increased (the slope standard deviation higher than the certain time interval is taken as the threshold), the emotion of a person to be detected reaches a relatively high awakening level at a high speed, the person is judged to be in a severe emotion fluctuation state, the first alarm instruction can be triggered, and the emotion state of the person to be detected is alarmed.

To sum up, the embodiment of the present application has at least the following technical effects:

1. because the person to be detected is easily influenced by negative events, an Electrocardiogram (ECG) sensor and a skin resistance (GSR) sensor are arranged on the skin surface of the person to be detected during the period of meeting activities of the relatives with out-of-control emotion, and electrocardiosignals and skin resistance signals of the epidermis of a human body are acquired. Electrocardio signals and skin resistance signals are used as input, and 3 physiological characteristic indexes of SDNN, SCR peak value and SCR frequency are calculated in real time. And based on the dynamic change rules of the 3 indexes, comprehensively analyzing to obtain 2 indexes capable of evaluating the individual emotion abrupt change and corresponding judgment threshold values, if the real-time emotion characterization index is larger than the threshold value, considering that the emotion of the person to be detected is fluctuated violently in a short time, and giving an alarm to the person to be detected by the system, otherwise, giving no alarm. When the physiological change of the person to be detected is collected, the SDNN index and the SCR index of the sudden emotion change of the individual are evaluated, and the corresponding judgment threshold is reached, so that the person to be detected can dynamically sense the physiological change hidden by the intention or behavior, and the micro emotion change hidden by the person to be detected can be accurately detected.

2. The adopted electrocardio-sensor obtains stable electrocardiosignals which can draw the heart beat change rule of a person to be detected under the scene of meeting in the familiarity, reflect the autonomic nerve activation condition of the body of the person to be detected and simultaneously reduce the influence of artifacts caused by limb movement to the maximum extent. The adopted skin resistance sensor obtains a skin resistance signal which can effectively represent the physiological and psychological awakening state of an individual under the influence of emotional fluctuation.

Example two

Based on the same inventive concept as the method for detecting the emotion of the person based on the multi-modal physiological signal in the foregoing embodiment, as shown in fig. 4, the present application provides a system for detecting the emotion of the person based on the multi-modal physiological signal, wherein the system includes:

the electrocardiosignal acquisition module is used for acquiring electrocardiosignals of a target user by utilizing an electrocardio sensor so as to obtain dynamic electrocardiosignal data;

the skin resistance signal acquisition module is used for acquiring skin resistance signals of a target user by using the skin resistance sensor so as to obtain dynamic skin resistance signal data;

the data preprocessing module is used for preprocessing the dynamic electrocardiosignal data and the dynamic skin resistance signal data for a preset window long time period so as to respectively obtain an SDNN index and an SCR fusion index;

the SDNN index judging module is used for judging whether the SDNN index meets the reduction situation or not;

the SCR fusion index judgment module is used for judging whether the SCR fusion index synchronously meets an increase situation or not if the SDNN index meets a decrease situation;

and the state alarm module is used for triggering a first alarm instruction to alarm the emotional state of the target user if the SDNN index meets the reduction situation and the SCR fusion index meets the increase situation synchronously.

Further, the system further comprises:

the peak acquisition unit is used for carrying out adjacent peak acquisition on the dynamic electrocardiosignal data based on a peak extraction algorithm so as to obtain a peak interval of two adjacent R waves;

and the peak interval calculating unit is used for dynamically calculating the peak interval of the two adjacent R waves for the preset window length time period based on a sliding window calculating algorithm so as to determine the SDNN index.

Further, the system further comprises:

the component extraction unit is used for carrying out SCR component extraction on the dynamic skin resistance signal data based on a feature extraction algorithm so as to obtain SCR components;

the characteristic determining unit is used for carrying out dynamic calculation on the SCR component for the preset window length period by utilizing the sliding window calculation algorithm so as to determine SCR peak characteristics and SCR frequency characteristics;

and the characteristic fusion unit is used for performing characteristic fusion on the SCR peak characteristic and the SCR frequency characteristic so as to determine the SCR fusion index.

Further, the system further comprises:

the weighted calculation unit is used for carrying out weighted average calculation on indexes of the SCR peak value characteristic and the SCR frequency characteristic so as to obtain the SCR fusion index;

and the reciprocal calculation unit is used for performing first derivative calculation on the SCR fusion index so as to obtain the change slope of the fusion index.

Further, the system further comprises:

the SDNN index judging unit is used for judging whether the SDNN index is lower than a preset index standard deviation threshold value or not;

and the fusion index judging unit is used for judging the change characteristic of the change slope of the fusion index if the SDNN index is lower than the preset index standard deviation threshold.

Further, the system further comprises:

the change slope judgment unit is used for judging whether the change slope of the fusion index is synchronously higher than a preset slope standard deviation threshold;

and the instruction triggering unit is used for triggering the first alarm instruction to alarm the emotional state of the target user if the change slope of the fusion index is synchronously higher than the preset slope standard deviation threshold.

The application provides a person emotion detection method based on a multi-modal physiological signal, which comprises the following steps: an Electrocardiogram (ECG) sensor and a skin resistance (GSR) sensor are arranged on the skin surface of a person to be detected, and electrocardiosignals and skin resistance signals of the human epidermis are collected. Electrocardio signals and skin resistance signals are used as input, and 3 physiological characteristic indexes of SDNN, SCR peak value and SCR frequency are calculated in real time. And based on the dynamic change rules of the 3 indexes, comprehensively analyzing to obtain 2 indexes capable of evaluating the individual emotion abrupt change and corresponding judgment threshold values, if the real-time emotion characterization index is larger than the threshold value, considering that the emotion of the person to be detected is fluctuated violently in a short time, and giving an alarm to the person to be detected by the system, otherwise, giving no alarm. The method and the device solve the technical problem that when the physiological changes of the person to be detected are collected, the person to be detected is difficult to dynamically perceive the physiological changes hidden by intention or behavior, so that the micro emotion changes hidden by the person to be detected are difficult to accurately detect. When the physiological change of the person to be detected is collected, the SDNN index and the SCR index of the sudden emotion change of the individual are evaluated, and the corresponding judgment threshold is reached, so that the person to be detected can dynamically sense the physiological change hidden by the intention or behavior, and the micro emotion change hidden by the person to be detected can be accurately detected.

The specification and drawings are merely illustrative of the present application, and it is intended that the present invention cover modifications and variations of this invention provided they come within the scope of the invention and their equivalents.

Claims (7)

1. A person emotion detection method based on multi-modal physiological signals is applied to an emotion detection system, and the system is in communication connection with an electrocardio sensor and a skin resistance sensor, and the method comprises the following steps:

acquiring electrocardiosignals of a target user by using the electrocardio sensor to obtain dynamic electrocardiosignal data;

acquiring skin resistance signals of a target user by using the skin resistance sensor so as to obtain dynamic skin resistance signal data;

preprocessing the dynamic electrocardiosignal data and the dynamic skin resistance signal data for a preset window long time period to respectively obtain an SDNN index and an SCR fusion index;

judging whether the SDNN index meets a reduction situation;

if the SDNN index meets the reduction situation, judging whether the SCR fusion index meets the increase situation synchronously;

and if the SDNN index meets the reduction situation, the SCR fusion index meets the increase situation synchronously, and a first alarm instruction is triggered to alarm the emotional state of the target user.

2. The method of claim 1, wherein the pre-processing for a predetermined window length period comprises:

based on a peak extraction algorithm, performing adjacent peak collection on the dynamic electrocardiosignal data to obtain a peak interval of two adjacent R waves;

and dynamically calculating the peak interval of the two adjacent R waves for the preset window length time period based on a sliding window calculation algorithm so as to determine the SDNN index.

3. The method of claim 2, wherein the method comprises:

based on a feature extraction algorithm, carrying out SCR component extraction on the dynamic skin resistance signal data to obtain SCR components;

performing dynamic calculation on the SCR component for the preset window long time period by using the sliding window calculation algorithm to determine SCR peak characteristics and SCR frequency characteristics;

and performing feature fusion on the SCR peak features and the SCR frequency features to determine the SCR fusion index.

4. The method of claim 3, wherein the method comprises:

carrying out weighted average calculation on the indexes of the SCR peak value characteristic and the SCR frequency characteristic so as to obtain the SCR fusion index;

and performing first-order derivative calculation on the SCR fusion index to obtain a fusion index change slope.

5. The method of claim 4, wherein the method comprises:

judging whether the SDNN index is lower than a preset index standard deviation threshold value or not;

and if the SDNN index is lower than the preset index standard deviation threshold, judging the change characteristic of the change slope of the fusion index.

6. The method of claim 5, wherein the method comprises:

judging whether the change slope of the fusion index is synchronously higher than a preset slope standard deviation threshold value;

and if the change slope of the fusion index is synchronously higher than the preset slope standard deviation threshold, triggering the first alarm instruction to alarm the emotional state of the target user.

7. A system for emotion detection of a person based on multi-modal physiological signals, the system comprising:

the electrocardiosignal acquisition module is used for acquiring electrocardiosignals of a target user by utilizing an electrocardio sensor so as to obtain dynamic electrocardiosignal data;

the skin resistance signal acquisition module is used for acquiring skin resistance signals of a target user by using the skin resistance sensor so as to acquire dynamic skin resistance signal data;

the data preprocessing module is used for preprocessing the dynamic electrocardiosignal data and the dynamic skin resistance signal data for a preset window long time period so as to respectively obtain an SDNN index and an SCR fusion index;

the SDNN index judging module is used for judging whether the SDNN indexes meet the reduction situation or not;

the SCR fusion index judgment module is used for judging whether the SCR fusion index synchronously meets an increase situation or not if the SDNN index meets a decrease situation;

and the state alarm module is used for triggering a first alarm instruction to alarm the emotional state of the target user if the SDNN index meets the reduction situation and the SCR fusion index meets the increase situation synchronously.

Images