CN114818814A

CN114818814A - Processing method and device for emotion recognition, electronic equipment and storage medium

Info

Publication number: CN114818814A
Application number: CN202210478131.5A
Authority: CN
Inventors: 邬霞; 徐雪远; 贾甜远
Original assignee: Beijing Normal University
Current assignee: Beijing Normal University
Priority date: 2022-04-29
Filing date: 2022-04-29
Publication date: 2022-07-29

Abstract

The embodiment of the disclosure discloses a processing method, a processing device, electronic equipment and a storage medium for emotion recognition, wherein the method comprises the following steps: acquiring an emotion electroencephalogram signal to be identified; extracting at least two types of emotional electroencephalogram characteristics based on the emotional electroencephalogram signals to be identified to form a first characteristic matrix; determining a characteristic weight vector based on a multi-dimensional emotional characteristic selection model obtained through pre-training, wherein an objective function of the multi-dimensional emotional characteristic selection model comprises a global characteristic redundancy matrix, a global label incidence matrix and an orthogonal regression matrix; based on the feature weight vector, performing feature selection on the first feature matrix to obtain an electroencephalogram feature subset; and obtaining an emotion recognition result based on the electroencephalogram feature subset and a multi-dimensional emotion classification model obtained by pre-training. According to the method and the device, the characteristic and non-redundant electroencephalogram feature subset can be selected, the feature dimension is effectively reduced, and the problems that highly-relevant features are easily reserved in the feature subset in the prior art and the like are solved.

Description

Processing method and device for emotion recognition, electronic equipment and storage medium

Technical Field

The present disclosure relates to emotion recognition technologies, and in particular, to a processing method and apparatus for emotion recognition, an electronic device, and a storage medium.

Background

In recent years, Emotion Recognition (Emotion Recognition) has become one of the important research subjects in the fields of human-computer interaction, signal processing, machine learning, and the like. Since the generation of emotion is accompanied by the progress of physiological and psychological activities, it is one of key technologies to utilize human neurophysiological signals to mine emotion information. Among them, electroencephalograms (EEG) have been widely used in emotion recognition research due to their advantages of high time resolution, non-invasiveness, low cost, and the like. Compared with a small sample size, the electroencephalogram features have high dimensionality and bring difficulty to electroencephalogram-based emotion classification. Therefore, when the number of extracted electroencephalogram features is large, the Feature Selection (Feature Selection) step is very critical. Feature selection refers to selecting M (M < D) features from existing D features to optimize specific indexes of a system, and is a process of selecting some most effective features from original features to reduce dimensionality of a data set. In the prior art, electroencephalogram feature selection methods are classified into the following three categories: filter (Filter), Wrapper (Wrapper) and Embedded (Embedded). The general idea of the filtering method is that the feature selection step is independent of the learner construction process, and the classification model is trained after the original features are screened. This approach is not linked to the construction of classification models, so filtering approaches tend to have difficulty obtaining discriminative feature subsets. The wrapped approach is computationally complex. Although the importance of features in a classification task is considered in the embedded method, due to the fact that the electroencephalogram signals have a volume conduction phenomenon, the signals collected on each electrode are often linear superposition results of a plurality of potential source signals, and therefore high correlation exists among the channels. The correlation makes a great deal of redundant and similar information possibly existing in the features extracted from the electroencephalogram signals, especially the features extracted from adjacent or symmetrical electrodes, and the redundant features cause the problems of higher calculation complexity, overfitting of a classifier and the like, so that the subsequent emotion recognition becomes difficult. The existing electroencephalogram feature selection method ignores the correlation among the selected features, so that the highly correlated features are reserved in the feature subset, and the problem of high redundancy caused by the volume conduction effect cannot be solved.

Disclosure of Invention

The present disclosure is proposed to solve the above technical problems. The embodiment of the disclosure provides a processing method and device for emotion recognition, electronic equipment and a storage medium.

According to an aspect of the embodiments of the present disclosure, there is provided a processing method for emotion recognition, including: acquiring an emotion electroencephalogram signal to be identified; extracting at least two types of emotional electroencephalogram characteristics based on the emotional electroencephalogram signals to be recognized to form a first characteristic matrix; determining a characteristic weight vector based on a multi-dimensional emotional characteristic selection model obtained by pre-training, wherein an objective function of the multi-dimensional emotional characteristic selection model comprises a global characteristic redundancy matrix, a global label incidence matrix and an orthogonal regression matrix; based on the feature weight vector, performing feature selection on the first feature matrix to obtain an electroencephalogram feature subset; and obtaining an emotion recognition result based on the electroencephalogram feature subset and a multi-dimensional emotion classification model obtained by pre-training.

According to another aspect of the embodiments of the present disclosure, there is provided a processing apparatus for emotion recognition, including: the first acquisition module is used for acquiring emotion electroencephalogram signals to be identified; the first feature extraction module is used for extracting at least two types of emotional electroencephalogram features based on the emotional electroencephalogram signals to be identified to form a first feature matrix; the first processing module is used for selecting a model based on multi-dimensional emotional characteristics obtained by pre-training and determining a characteristic weight vector; the second processing module is used for performing feature selection on the first feature matrix based on the feature weight vector to obtain an electroencephalogram feature subset; and the third processing module is used for obtaining an emotion recognition result based on the electroencephalogram feature subset and a multi-dimensional emotion classification model obtained through pre-training.

According to a further aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium storing a computer program for executing the method according to any one of the above-mentioned embodiments of the present disclosure.

According to still another aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including: a processor; a memory for storing the processor-executable instructions; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method according to any of the above embodiments of the present disclosure.

Based on the processing method, device, electronic device and storage medium for emotion recognition provided by the embodiments of the present disclosure, a feature weight vector is determined based on a multi-dimensional emotion feature selection model obtained by pre-training, then a characteristic and non-redundant electroencephalogram feature subset is selected based on the feature weight vector, and then emotion recognition is performed on the selected electroencephalogram feature subset based on a multi-dimensional emotion classification model obtained by training to obtain a corresponding emotion recognition result. Because the multidimensional emotion feature selection model comprehensively considers the local correlation among multidimensional emotion labels, the global redundancy among electroencephalogram features and the global correlation among multidimensional emotion labels based on the orthogonal regression matrix, the global label correlation matrix and the global feature redundancy matrix, the characteristic selection based on the obtained feature weight vector can select a characteristic and non-redundant electroencephalogram feature subset, effectively reduces the feature dimension, improves the accuracy of feature selection, and solves the problem that the prior art is easy to retain highly-related features in the feature subset and cannot overcome the high redundancy caused by the volume conduction effect.

The technical solution of the present disclosure is further described in detail by the accompanying drawings and examples.

Drawings

The above and other objects, features and advantages of the present disclosure will become more apparent by describing in more detail embodiments of the present disclosure with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the principles of the disclosure and not to limit the disclosure. In the drawings, like reference numbers generally represent like parts or steps.

FIG. 1 is an exemplary application scenario of the emotion recognition processing method provided by the present disclosure;

FIG. 2 is a flowchart illustrating a processing method for emotion recognition provided by an exemplary embodiment of the present disclosure;

FIG. 3 is a flowchart illustrating a processing method for emotion recognition provided by another exemplary embodiment of the present disclosure;

FIG. 4 is a graph illustrating a comparison of Reduncyny metrics provided by an exemplary embodiment of the present disclosure;

FIG. 5 is a graph illustrating comparative results of the Coverage indicator provided by an exemplary embodiment of the present disclosure;

FIG. 6 is a graph illustrating comparative results of a Hamming loss index provided by an exemplary embodiment of the present disclosure;

FIG. 7 is a graph illustrating a comparison of the Ranking loss metric provided by an exemplary embodiment of the present disclosure;

FIG. 8 is a graph illustrating a comparison of Average precision indicators provided by an exemplary embodiment of the present disclosure;

FIG. 9 is a graphical illustration of comparative results of the Macro-F1 metric provided by an exemplary embodiment of the present disclosure;

FIG. 10 is a graphical illustration of a comparison of the Micro-F1 metric provided by an exemplary embodiment of the present disclosure;

FIG. 11 is a graph illustrating a convergence curve of objective function values provided by an exemplary embodiment of the present disclosure;

FIG. 12 is a schematic structural diagram of a processing apparatus for emotion recognition provided in an exemplary embodiment of the present disclosure;

FIG. 13 is a schematic structural diagram of a processing apparatus for emotion recognition provided in another exemplary embodiment of the present disclosure;

fig. 14 is a schematic structural diagram of an application embodiment of the electronic device of the present disclosure.

Detailed Description

Hereinafter, example embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. It is to be understood that the described embodiments are merely a subset of the embodiments of the present disclosure and not all embodiments of the present disclosure, with the understanding that the present disclosure is not limited to the example embodiments described herein.

It should be noted that: the relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

It will be understood by those of skill in the art that the terms "first," "second," and the like in the embodiments of the present disclosure are used merely to distinguish one element from another, and are not intended to imply any particular technical meaning, nor is the necessary logical order between them.

It is also understood that in embodiments of the present disclosure, "a plurality" may refer to two or more and "at least one" may refer to one, two or more.

It is also to be understood that any reference to any component, data, or structure in the embodiments of the disclosure, may be generally understood as one or more, unless explicitly defined otherwise or stated otherwise.

In addition, the term "and/or" in the present disclosure is only one kind of association relationship describing an associated object, and means that three kinds of relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in the present disclosure generally indicates that the former and latter associated objects are in an "or" relationship.

It should also be understood that the description of the embodiments in the present disclosure emphasizes the differences between the embodiments, and the same or similar parts may be referred to each other, and are not repeated for brevity.

Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

The disclosed embodiments may be applied to electronic devices such as terminal devices, computer systems, servers, etc., which are operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known terminal devices, computing systems, environments, and/or configurations that may be suitable for use with electronic devices, such as terminal devices, computer systems, servers, and the like, include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, networked personal computers, minicomputer systems, mainframe computer systems, distributed cloud computing environments that include any of the above, and the like.

Electronic devices such as terminal devices, computer systems, servers, etc. may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer system/server may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

Summary of the disclosure

In the process of implementing the present disclosure, the inventors find that, in the existing emotion recognition method, when electroencephalogram feature selection is performed, correlation among the selected features is ignored, so that highly correlated features are retained in a feature subset, and the problem of high redundancy caused by a volume conduction effect cannot be overcome.

Brief description of the drawings

FIG. 1 is an exemplary application scenario of the emotion recognition processing method provided by the present disclosure.

During emotion recognition, emotion electroencephalogram signals to be recognized can be collected in any implementable mode, at least two types of emotion electroencephalogram features can be extracted based on the emotion electroencephalogram signals to be recognized to form a feature matrix (called as a first feature matrix), a feature weight vector is determined based on a multi-dimensional emotion feature selection model of orthogonal regression obtained through pre-training, the feature weight vector comprises weighted values of the emotion electroencephalogram features of all types, the size of the weighted values represents the importance of the emotion electroencephalogram features of the types in emotion recognition, the importance comprehensively considers the local relevance among multi-dimensional emotion labels, the global redundancy among electroencephalogram features and the global relevance among the multi-dimensional emotion labels, therefore, the emotion electroencephalogram signals to be recognized are subjected to feature selection based on the weighted values, and a characteristic and non-redundant electroencephalogram feature subset can be selected, the accuracy of feature selection is improved, the problem that in the prior art, highly-related features are easily reserved in feature subsets, and high redundancy caused by volume conduction effect cannot be solved is solved, and then a multi-dimensional emotion classification model is adopted to carry out emotion recognition on the selected electroencephalogram feature subsets, so that the emotion recognition efficiency can be effectively improved.

Exemplary method

FIG. 2 is a flowchart illustrating a processing method for emotion recognition according to an exemplary embodiment of the present disclosure. The embodiment can be applied to electronic devices, such as a server, a terminal, and the like, specifically, as shown in fig. 2, the method includes the following steps:

step 201, obtaining an emotion electroencephalogram signal to be identified.

The emotion electroencephalogram signals to be recognized can be obtained in any implementable mode, for example, the emotion electroencephalogram signals are collected based on a brain-computer interface. The brain-computer interface is a direct communication channel established between the brain and external devices. The detailed functional principle of the brain-computer interface is not described in detail. The emotion electroencephalogram signals to be recognized can be emotion electroencephalogram signals induced by any user in any mode. For example, the specific inducing mode of the emotional electroencephalogram signal generated when the user watches the video is not limited.

Step 202, extracting at least two types of emotional electroencephalogram characteristics based on the emotional electroencephalogram signals to be identified to form a first characteristic matrix.

The types of the emotional electroencephalogram features can include non-stationary index values, high-order intersection, spectral entropy, shannon entropy, C0 complexity, differential entropy, absolute power ratio of beta wave bands and theta wave bands, amplitude of intrinsic mode functions, instantaneous phase of the intrinsic mode functions, differential entropy difference values of spatially symmetric electrodes, differential entropy ratio values and functional connections of the spatially symmetric electrodes, and the like. The specifically extracted emotion electroencephalogram features can be set according to actual requirements, for example, all types of emotion electroencephalogram features can be extracted to form a first feature matrix. The specific extracted type needs to be consistent with a training sample of a training process of the multi-dimensional emotion feature selection model.

Step 203, determining a feature weight vector based on a multi-dimensional emotional feature selection model obtained through pre-training, wherein an objective function of the multi-dimensional emotional feature selection model comprises a global feature redundancy matrix, a global label incidence matrix and an orthogonal regression matrix.

The multi-dimensional emotion feature selection model is an algorithm model, the input of the training process is a standardized feature matrix obtained based on the training emotion electroencephalogram signals and a corresponding emotion label matrix, the output is an objective function value, and the objective function represents the training target of the multi-dimensional emotion feature selection model. In the training process, local correlation information between the electroencephalogram characteristics and the multi-dimensional emotion labels can be mined through the orthogonal regression matrix, correlation between the characteristics can be evaluated from a global angle through the global characteristic redundancy matrix, so that redundant characteristics can be removed, characteristic and non-redundant electroencephalogram emotion characteristic subsets can be selected, the characteristic dimension is reduced, correlation of the global labels can be considered through the global label correlation matrix, construction of label information in a low-dimensional space is facilitated, and local correlation among the multi-dimensional emotion labels, global redundancy among the electroencephalogram characteristics and global correlation among the multi-dimensional emotion labels are comprehensively considered. The multi-dimensional emotional feature selection model further comprises a mapping matrix, a potential semantic matrix and a feature weight matrix which are obtained through iterative training, and the feature weight vector is obtained based on the feature weight matrix of the multi-dimensional emotional feature selection model. For example, the obtained feature weight matrix is diagonalized, and the weight values on the diagonal form a feature weight vector.

The step 203 and the step 201-202 are not in sequence, that is, the step 203 can be executed at any time before, during and after the step 201-201 is executed, and may be specifically configured according to actual requirements.

And 204, performing feature selection on the first feature matrix based on the feature weight vector to obtain an electroencephalogram feature subset.

The feature weight vector comprises weighted values of various types of emotion electroencephalogram features, the weighted values represent the importance of the emotion electroencephalogram features of the types in emotion recognition, and the local relevance among multi-dimensional emotion labels, the global redundancy among electroencephalogram features and the global relevance among the multi-dimensional emotion labels are comprehensively considered in the multi-dimensional emotion feature selection model training process, so that the feature selection is performed on the emotion electroencephalogram signals to be recognized based on the weighted values in the feature weight vector, a characteristic and non-redundant electroencephalogram feature subset can be selected, the accuracy of feature selection is improved, the problems that in the prior art, highly-related features are easily reserved in the feature subset, and the high redundancy caused by volume conduction effect cannot be overcome are solved.

And step 205, obtaining an emotion recognition result based on the electroencephalogram feature subset and a multi-dimensional emotion classification model obtained through pre-training.

The multi-dimensional emotion classification model is obtained by training a training electroencephalogram feature subset selected by a multi-dimensional emotion feature selection model obtained by training, classification labels of the multi-dimensional emotion classification model in the training process are multi-dimensional emotion labels, for example, for each type of electroencephalogram feature, the corresponding classification label comprises emotion labels of at least two dimensions, such as at least two of activation (aroma), Valence (Valence), control (Dominance) and other related dimensions, and the multi-dimensional emotion classification model can be specifically set according to actual requirements. Therefore, the emotion recognition result corresponding to the emotion electroencephalogram signal to be recognized, which is obtained in the application process, comprises a multi-dimensional emotion recognition result. For example, the activation degree, the titer and the control degree are respectively 1 to 9 in each dimension, the value greater than 5 is set as 1, the value less than or equal to 5 is set as 0, and the emotion recognition result (activation degree, titer and control degree) is (0,1,1) representing the emotion recognition result of three dimensions, wherein the activation degree is less than or equal to 5, the titer is greater than 5, and the control degree is greater than 5, so that the multi-dimensional emotion recognition is realized.

In the processing method for emotion recognition provided by this embodiment, a feature weight vector is determined based on a multi-dimensional emotion feature selection model obtained through pre-training, a characteristic and non-redundant electroencephalogram feature subset is selected based on the feature weight vector, and emotion recognition is performed on the selected electroencephalogram feature subset based on a multi-dimensional emotion classification model obtained through training to obtain a corresponding emotion recognition result. Because the multidimensional emotion feature selection model comprehensively considers the local correlation among multidimensional emotion labels, the global redundancy among electroencephalogram features and the global correlation among multidimensional emotion labels based on the orthogonal regression matrix, the global label correlation matrix and the global feature redundancy matrix, the characteristic selection based on the obtained feature weight vector can select a characteristic and non-redundant electroencephalogram feature subset, effectively reduces the feature dimension, improves the accuracy of feature selection, and solves the problem that the prior art is easy to retain highly-related features in the feature subset and cannot overcome the high redundancy caused by the volume conduction effect.

FIG. 3 is a flowchart illustrating a processing method for emotion recognition according to another exemplary embodiment of the present disclosure.

In an alternative example, before step 203, the following steps are further included:

step 301, obtaining training sample data, wherein the training sample data comprises training emotion electroencephalogram signals and corresponding emotion label data.

The emotion electroencephalogram signals can be obtained in any implementable mode, for example, the emotion electroencephalogram signals can be generated by watching a second number of video bands by a first number of users, and when the users watch videos, the emotion electroencephalogram signals of the users are collected based on a brain-computer interface and serve as the emotion electroencephalogram signals for training. The first number and the second number may be set according to actual requirements, and the disclosure is not limited. The label data corresponding to the training emotion electroencephalogram signals can be obtained by evaluating after the user watches the video band, for example, after the user watches the video band, the user is prompted to evaluate the emotion of the dimensionality such as the activation degree, the valence and the control degree, and the emotion label data are determined based on the evaluation result of the user.

Illustratively, taking three dimensions of activation degree, valence and control degree as examples, setting the evaluation range of each dimension to be 1-9 points, representing from weak to strong, when a user watches a section of video segment, performing evaluation scoring of the three dimensions on the video segment to obtain an evaluation result of the user, based on a preset scoring threshold, mapping the score of the user in each dimension to an 0/1 label, for example, the preset scoring threshold is 5, when the scoring value is higher than 5, setting the corresponding label to be 1, when the scoring value is lower than or equal to 5, determining the corresponding label to be 0, and obtaining the label of each user in the three dimensions on each video segment to form emotion label data. It should be noted that the corresponding relationship between the emotion electroencephalogram signal and the emotion tag data needs to be established. For example, the emotion tag data generated by 3 users watching 2 video segments can be expressed as:

Y＝[y ₁ ,y ₂ ,y ₃ ]

wherein Y ∈ R ^6×3 That is, 6 emotion electroencephalogram signal samples can be generated when 3 users watch 2 video bands, and each emotion electroencephalogram signal sample corresponds to an emotion label with 3 dimensions. That is, y ₁ 、y ₂ 、y ₃ Respectively, 6-dimensional label vectors.

Step 302, determining a training emotion electroencephalogram feature matrix and a corresponding emotion label matrix based on training sample data.

The training emotion electroencephalogram feature matrix is obtained by extracting features of the training emotion electroencephalogram signals, the specific number of extracted features and the extraction mode are similar to those in step 202, and the details are not repeated here. And the emotion tag matrix is a tag matrix which is formed by emotion tag data and corresponds to the training emotion electroencephalogram characteristic matrix.

Step 303, carrying out normal distribution standardization on each type of emotion electroencephalogram characteristics in the training emotion electroencephalogram characteristic matrix to obtain a corresponding standardized characteristic matrix.

The emotion electroencephalogram characteristics of different types can include one or more characteristics, and because the characteristic values of the electroencephalogram characteristics of different types are likely to be different greatly, for example, the characteristic value of one characteristic is different from the characteristic value of another characteristic by one or more orders of magnitude, if the characteristic values are directly adopted, the importance of the characteristics of different types to emotion recognition cannot be accurately determined, so that each type of emotion electroencephalogram characteristics in the emotion electroencephalogram characteristic matrix for training are subjected to normal distribution standardization, the emotion electroencephalogram characteristics of different types are unified into a standard range, and a standardized characteristic matrix is obtained and used for subsequent model training. The details of the normalization principle of normal distribution are not repeated.

And 304, performing iterative training on the mapping matrix, the potential semantic matrix and the feature weight matrix of the multi-dimensional emotional feature selection model based on the standardized feature matrix, the emotional tag matrix and the preset objective function until the value of the preset objective function is converged, and obtaining the trained multi-dimensional emotional feature selection model.

The preset objective function is an objective function for multi-dimensional emotional characteristic selection model training, and comprises a global characteristic redundancy matrix, a global label incidence matrix, an orthogonal regression matrix, a mapping matrix needing iterative updating, a potential semantic matrix and a characteristic weight matrix. And taking the standardized feature matrix and the emotion label matrix as the input of the multi-dimensional emotion feature selection model, carrying out iterative operation on the target function according to an optimization algorithm, solving the mapping matrix, the latent semantic matrix and the feature weight matrix, realizing iterative updating of the mapping matrix, the latent semantic matrix and the feature weight matrix until the target function is converged, and obtaining the trained multi-dimensional emotion feature selection model.

In an alternative example, the step 203 of determining the feature weight vector based on the multi-dimensional emotion feature selection model obtained by pre-training includes:

step 2031, determining a feature weight vector based on the feature weight matrix of the multi-dimensional emotion feature selection model.

In the training process, the feature weight matrix is obtained by diagonalizing a feature weight vector, when training starts, random initialization needs to be carried out on a mapping matrix and a potential semantic matrix, each element in the feature weight vector is assigned as an initial weight value 1/D, D represents the number of features in a standardized feature matrix, each element in the feature weight vector represents the weight of a corresponding feature, each weight value is greater than or equal to 0, and the addition result of the D weight values is 1. The feature weight vectors are diagonalized to construct a feature weight matrix. The off-diagonal elements of the feature weight matrix are all 0.

Illustratively, the feature weight vector is represented as θ ═ θ ₁ ,θ ₂ ,…,θ _D ] ^T ，θ∈R ^D×1 And the constructed characteristic weight matrix is expressed as theta:

after the training is finished, obtaining a trained characteristic weight matrix

And converting the vector into a corresponding feature weight vector for subsequent feature selection.

In an optional example, after obtaining the trained multidimensional emotion feature selection model, the method further includes:

and 305, determining the selected training electroencephalogram feature subset and corresponding emotion label data based on the feature weight vector.

The training electroencephalogram feature subset can be obtained by selecting from the standardized feature matrix based on the feature weight vector. For example, a preset weight threshold value can be set, and the features with the weight values larger than the preset weight threshold value in the standardized feature matrix are extracted to form a training electroencephalogram feature subset. The features in the standardized feature matrix can be sorted according to the weight values, and a certain number of features with the weight values ranked in the front are extracted to be used as training electroencephalogram feature subsets. And determining emotion label data corresponding to the training electroencephalogram feature subset according to the corresponding relation between the features in the standardized feature matrix and the emotion labels.

And step 306, training to obtain a multi-dimensional emotion classification model based on the training electroencephalogram feature subset and the corresponding emotion label data, wherein the multi-dimensional emotion classification model adopts a multi-label k neighbor classifier.

The emotion classification model adopts a multi-label k neighbor classifier, and can realize the identification of multi-dimensional emotion. The nearest neighbor number and the smoothing number of the k-nearest neighbor classifier can be set according to actual requirements, for example, can be set to 10 and 1 respectively, and are not limited in particular. The specific principle of the k-nearest neighbor classifier is not described herein.

In one alternative example, the preset objective function is:

s.t.W ^T W＝I _K ,θ ^T 1 _D ＝1,θ≥0

wherein X represents a normalized feature matrix, and X is equal to R ^D×N D represents the number of features in the normalized feature matrix, N represents the number of samples, s.t. represents the condition that follows is satisfied, theta represents the feature weight vector, theta is equal to R ^D×1 Theta ≧ 0 denotes that the weight value in the feature weight vector is greater than or equal to 0, theta denotes the feature weight matrix, theta ∈ R ^D×D Theta is a matrix obtained by diagonalizing the feature weight vector theta, W represents a mapping matrix, W is an orthogonal regression matrix, and W is an element of W ∈ R ^D×K K represents the dimension number of the emotion labels, and W satisfies the orthogonal regression constraint W ^T W＝I _K ，I _K Is an identity matrix of K multiplied by K, V represents a potential semantic matrix, and V belongs to R ^N×K Y represents an emotion tag matrix, and Y is equal to R ^N×K Alpha, eta, lambda, beta are equilibrium parameters, 1 _N A column vector of all 1, i.e. 1 _N ＝[1,1,…,1] ^T ∈R ^N×1 ，b∈R ^K×1 For the deviation vector, L is the graph Laplace matrix of the normalized feature matrix X, L ∈ R ^N×N A represents a global feature redundancy matrix, and A is equal to R ^D×D A is determined based on a standardized feature matrix X, R represents a global label incidence matrix, and R belongs to R ^K×K R is based on emotionThe determination of the tag matrix is performed,

the Frobenius norm of the matrix is expressed, i.e. the sum of the squares of the absolute values of the elements of the matrix is found.

Wherein, the Frobenius norm of the matrix is the sum of the squares of the absolute values of each element of the matrix, for example,

tr (B) represents summing the eigenvalues of the matrix B, and the detailed calculation principle is not repeated.

The global feature redundancy matrix a is defined as follows:

wherein, f _i ∈R ^N×1 And f _j ∈R ^N×1 Denotes x _i And x _j The corresponding ith and jth lumped features (i 1, 2.. multidot.D; j 1, 2.. multidot.D) respectively. N denotes the number of samples and D denotes the number of features, see in particular the foregoing.

f _i And f _j The calculation method is as follows:

wherein the content of the first and second substances,

equation 1 can be transformed into the following form:

O＝CF ^T FC＝(FC) ^T FC equation 3

Wherein F ═ F ₁ ，f ₂ ，...，f _D ]. C is a diagonal matrix whose diagonal elements are

1, 2, D, matrix O is a semi-positive definite matrix. According to

(

Representing the Hadamard product of matrices O and O), and matrix a is also non-negative and semi-positive.

The global label incidence matrix R is defined as follows:

R _ij ＝1-Z _ij

wherein Z is _ij Indicating label y _i And y _j I 1, 2, K, j 1, 2, K, Z _ij And the second-order correlation between a plurality of labels in the emotion label matrix is mined by cosine similarity calculation.

Based on the 1 st element in the above-mentioned objective function

And mapping the input normalized feature matrix from the original feature space to a low-dimensional space (a subspace same as V) through an orthogonal regression matrix W, thereby mining the local correlation information between the electroencephalogram features and the multi-dimensional emotion labels. Item 2 element in objective function

And item 5 element tr (RV) ^T V) the global label correlation in the multi-dimensional emotion label space is embodied through the global label incidence matrix R, and label information in the low-dimensional space is constructed. Element θ of item 4 in the objective function ^T And A theta is used for realizing the evaluation of the correlation among the features from the global perspective through the global feature redundancy matrix A, so that redundant features are removed, characteristic and non-redundant electroencephalogram emotion feature subsets are selected, the feature dimension is reduced, and the performance and the effect of multi-dimensional emotion recognition are improved.

In practical applications, the multidimensional emotion classification model can be implemented by any other practicable classification model, and is not limited to the k-nearest neighbor classifier.

In an optional example, the iteratively training the mapping matrix, the latent semantic matrix, and the feature weight matrix of the multidimensional emotion feature selection model based on the normalized feature matrix, the emotion tag matrix, and the preset objective function in step 304 until the value of the preset objective function converges to obtain the trained multidimensional emotion feature selection model, which includes: in each iteration process, for the updating of the mapping matrix, the latent semantic matrix and the feature weight matrix, the other matrix is updated by fixing two of the matrixes.

The two matrices are fixed, so that the two matrices are assumed to be known quantities, for example, a mapping matrix and a latent semantic matrix are fixed, a feature weight matrix is updated, and the mapping matrix and the latent semantic matrix obtained in the previous iteration process can be used as the known quantities of the mapping matrix and the latent semantic matrix in the current iteration process to update the feature weight matrix according to a preset rule.

In an optional example, in the process of training to obtain the multidimensional emotion feature selection model, iterative operation needs to be performed on an objective function according to an optimization algorithm, and the iterative operation process specifically includes:

1. calculating the partial derivative of the deviation vector b for the preset target function l, and making the partial derivative equal to 0 to obtain the deviation vector

2. Substituting the deviation vector into the objective function l, and calculating to obtain an updated objective function

Namely:

s.t.W ^T W＝I _K ,θ ^T 1 _D ＝1,θ≥0

the meaning of each symbol is referred to the objective function, and is not described herein again.

3. Based on the updated objective function

Alternately fixing the mapping matrix W, the potential semantic matrix V and the characteristic weight matrix theta, and carrying out comparison on the updated target function

And (4) performing iterative operation, and respectively solving the mapping matrix W, the potential semantic matrix V and the characteristic weight matrix theta to realize updating.

In an alternative example, fixing the latent semantic matrix V and the feature weight matrix Θ, and solving the mapping matrix W includes:

target function after updating

Is converted into

Using a generalized iterative power method pair

A mapping matrix W is calculated.

Wherein the content of the first and second substances,

a is a symmetric matrix, and the condition that the symmetric matrix meets is that A belongs to R ^D×D ，B＝ΘXHY ^T Is a second alternative parameter.

The general power iteration method (GPI) is an algorithm proposed to effectively and meaningfully solve the quadratic problem of grassmann manifold (QPSM), i.e., the problem of orthogonal least squares regression and unbalanced orthogonal process. Compared with other algorithms, the GPI algorithm monotonically reduces the target value of the QPSM problem to local minimum until final convergence, and the optimal solution is converged by the random initial guess value, so that the convergence speed is higher, the processing time is shorter, and the calculation of the high-dimensional data matrix is more efficient. As shown in table 1, is the main flow of the GPI algorithm:

TABLE 1 Main flow of GPI Algorithm

In an alternative example, the latent semantic matrix V and the mapping matrix W are fixed, and the weight matrix Θ is solved, including:

target function after updating

Is converted into

Namely that

The condition satisfied is

And using augmented Lagrange multiplier method to update target function

The feature weight matrix Θ is calculated.

The Augmented Lagrangian Multiplier (ALM) is an algorithm for optimizing a target function and solving a regression matrix with orthogonal constraint and a diagonal matrix related to characteristic weight by combining GPI. The algorithm mainly solves the optimization problem with constraint conditions by decomposing the problem into a plurality of subproblems, and combines the constraint condition function with the primitive function, thereby solving the solution of each variable which enables the primitive function to obtain an extreme value, and the algorithm is a conventional algorithm for solving the extreme value with the constraint terms. As shown in table 2, is the main flow of the ALM algorithm.

TABLE 2 ALM AlM Alm Alc

In an alternative example, the feature weight matrix Θ and the mapping matrix W are fixed, and the latent semantic matrix V is solved, including:

for updated objective function

And solving the partial derivatives of the potential semantic matrix V to obtain the following result:

2[H ^T (V-X ^T ΘW)+α(V-Y)+ηLV+βVR]＝0

converting the above formula to (H) ^T +ηL)V+V(αI _K +βR)＝H ^T X ^T Θ W + α Y, i.e. the schervite equation MV + VE ═ P, where

The latent semantic matrix V is calculated for the above-mentioned seivits equation. The principle of solving the seersist equation is conventional and will not be described in detail herein.

In an optional example, the feature selection of the first feature matrix based on the feature weight vector in step 204 to obtain the electroencephalogram feature subset includes:

step 2041, carrying out normal distribution standardization on each type of emotion electroencephalogram characteristics in the first characteristic matrix to obtain a corresponding standardized second characteristic matrix.

The detailed operation of this step is referred to the aforementioned step 303, and is not described herein again.

Step 2042, based on the feature weight vector, determining the target emotion electroencephalogram features of which the corresponding feature weight values in the second feature matrix are greater than a preset threshold.

The preset threshold value may be set according to actual requirements, and the disclosure is not limited.

And 2043, taking the target emotion electroencephalogram characteristics as an electroencephalogram characteristic subset.

The characteristic weight vector is obtained by comprehensively considering the local relevance among the multi-dimensional emotion labels, the global redundancy among the electroencephalogram characteristics and the global relevance among the multi-dimensional emotion labels, so that the characteristic and non-redundant electroencephalogram characteristic subset can be selected based on the characteristic weight vector, the characteristic dimension is effectively reduced, and the problem that the prior art is easy to retain highly-related characteristics in the characteristic subset and cannot overcome the high redundancy caused by the volume conduction effect is solved.

In an optional example, the extracting, in step 202, at least two types of emotion electroencephalogram features based on an emotion electroencephalogram signal to be identified to form a first feature matrix, includes:

step 2021, preprocessing the emotion electroencephalogram signal to be recognized based on the independent component analysis, removing the physiological artifact in the emotion electroencephalogram signal to be recognized, and obtaining a preprocessed first emotion electroencephalogram signal.

The physiological artifacts include noises caused by electrooculogram and myoelectricity. The principle of the independent component analysis is conventional technology, and is not described in detail herein. Through preprocessing, the effectiveness of the electroencephalogram signals can be effectively improved.

Step 2022, performing feature extraction on the first emotion electroencephalogram signal to obtain at least two of a non-stationary index value, a high-order intersection, a spectral entropy, a shannon entropy, a complexity of C0, a differential entropy, an absolute power ratio of a beta band and a theta band, an amplitude of an eigenmode function, an instantaneous phase of the eigenmode function, a differential entropy difference value of a space symmetric electrode, a differential entropy ratio of the space symmetric electrode and a functional connection to form a first feature matrix.

The specific extracted feature types and the specific extracted feature quantities can be set according to actual requirements, and the disclosure is not limited. Any practicable manner may be adopted for each type of feature extraction, and the disclosure is not limited thereto.

In an alternative example, the emotion recognition processing method of the present disclosure is further described by an example, which specifically includes the following steps:

1. brain-computer interface based emotion electroencephalogram signal for inducing tested user by video

(1) The same number of positive, neutral and negative emotional stimulus segments are selected from the existing video (such as a movie) as the emotional stimulus material. In the early stage of selecting the emotion-inducing material, more than 20 video segments are clipped, and in order to select the segment which can stimulate the tested emotion more, each video segment is evaluated by 15 users respectively. Finally, 12 video clips of 3-5 minutes were selected as the stimulus material.

(2) The tested user 16 is identified, such as 8 boys and 8 girls, aged between 18-25 years, right handedness. The conditions of the specific user to be tested can be set according to actual requirements, and are not limited to the above conditions.

(3) And informing the specific process of the user to be tested by playing the guide words so that the user to be tested can smoothly complete the collection of the emotion electroencephalogram signals. And each tested user sequentially watches the determined 12 video clips of 3-5 minutes, and the video clips are played in a preset sequence. In the watching process of the user to be tested, brain-computer interface acquisition equipment is adopted to acquire the electroencephalogram signals when the user to be tested watches the video clips. In this example, the brain-computer interface acquisition device may be any practicable acquisition device, such as a brain electrode cap using 128 electrodes.

(4) After watching each video segment, the user to be tried evaluates and scores from three dimensions (including the three dimensions and any dimension among the other dimensions) of the activation degree (aroma), the Valence (Valence) and the control degree (Dominance) of the emotion respectively, wherein each dimension is 1-9 minutes from weak to strong, in the emotion electroencephalogram signal acquisition process, in order to avoid the hysteresis influence of the emotion as much as possible, after each video segment is played, the user to be tried can continue to rest for a preset time, for example, rest for 30 seconds, so that the emotion of the user to be tried is restored to a stable state before watching the next video segment.

(5) Will be tried outThe score value of each dimension of the user is set to 1 when the score value is higher than 5, and is set to 0 when the score value is lower than or equal to 5, so that an emotion tag matrix Y is constructed, wherein Y is [ Y ═ Y ₁ ,y ₂ ,y ₃ ]Y satisfies the condition that Y belongs to R ^192×3 K is 3, N is 192, and T is the transpose operation of the matrix. The collected emotion electroencephalogram signals and the corresponding emotion labels form an Hded data set together.

2. And (5) extracting the features to form a feature matrix.

(1) And (3) removing physiological artifacts such as electrooculogram and myoelectricity in the emotional electroencephalogram signals by adopting independent component analysis.

(2) And respectively extracting a non-stationary index value, high-order intersection, spectral entropy, Shannon entropy, C0 complexity, differential entropy, absolute power ratio of beta wave band and theta wave band, amplitude of intrinsic mode function, instantaneous phase of intrinsic mode function, differential entropy difference of space symmetric electrode, differential entropy ratio of space symmetric electrode and functional connection from the emotional electroencephalogram signal to be used as emotional electroencephalogram characteristics.

(3) And constructing an original characteristic matrix based on the acquired emotional electroencephalogram characteristics.

3. And (4) selecting the characteristics.

(1) And carrying out normal distribution standardization on each type of electroencephalogram characteristics in the original characteristic matrix to obtain a standardized characteristic matrix X. X ═ X ₁ ,x ₂ ,…,x _d ,…,x ₇₅₆₅ ] ^T X satisfies the condition that X belongs to R ^7565×192 D-7565 is the number of features included in each sample in the normalized feature matrix, N-192 is the number of samples of the emotion electroencephalogram signal in the normalized feature matrix, T is the transpose operation on the matrix, and x _d Is the d-th feature in the normalized feature matrix. And taking the standardized feature matrix X as the input of the multi-dimensional emotion feature selection model.

(2) And constructing an objective function l of the multi-dimensional emotion feature selection model according to the standardized feature matrix X and the corresponding emotion label matrix Y.

s.t.W ^T W＝I _K ，θ ^T 1 _D ＝1，θ≥0

Wherein X represents a normalized feature matrix, and X is equal to R ^D×N D represents the number of features in the normalized feature matrix, N represents the number of samples, s.t. represents the condition that is satisfied for the following, theta represents the feature weight vector, theta belongs to R ^D×1 Theta ≧ 0 denotes that the weight value in the feature weight vector is greater than or equal to 0, theta denotes the feature weight matrix, theta ∈ R ^D×D Where Θ is a matrix obtained by diagonalizing the feature weight vector θ, W represents a mapping matrix, W is an orthogonal regression matrix, and W is within the range of R ^D×K K represents the dimension number of the emotion labels, and W satisfies the orthogonal regression constraint W ^T W＝I _K ，I _K Is an identity matrix of K multiplied by K, V represents a potential semantic matrix, and V belongs to R ^N×K Y represents an emotion tag matrix, and Y is equal to R ^N×K Alpha, eta, lambda, beta are equilibrium parameters, 1 _N A column vector of all 1, i.e. 1 _N ＝[1，1，…，1] ^T ∈R ^N×1 ，b∈R ^K×1 For the deviation vector, L is the graph Laplacian matrix of the normalized feature matrix X, L ∈ R ^N×N A represents a global feature redundancy matrix, and A is equal to R ^D×D A is determined based on a standardized feature matrix X, R represents a global label incidence matrix, and R belongs to R ^K×K R is determined based on the emotion tag matrix,

(3) And carrying out random initialization on a mapping matrix W and a potential semantic matrix V of the multi-dimensional emotional feature selection model.

(4) Each element in a characteristic weight vector theta of the multi-dimensional emotion characteristic selection model is assigned to be 1/7565, theta is larger than or equal to 0 and represents the importance of electroencephalogram characteristics to the multi-dimensional emotion recognition task, each element in the characteristic weight vector represents the weight of each characteristic, each weight value is larger than or equal to 0, and the sum is 1; and diagonalizing the characteristic weight vector theta to construct a weight matrix theta, wherein off-diagonal elements of the weight matrix theta are all zero.

(5) Calculating the partial derivative of the deviation vector for the target function l, and making the partial derivative equal to 0 to obtain the deviation vector

Substituting the deviation vector into the objective function l for calculation to obtain an updated objective function

Namely:

s.t.W ^T W＝I ₃ ，θ ^T 1 ₇₅₆₅ ＝1，θ≥0

wherein the content of the first and second substances,

(6) fixing the latent semantic matrix V and the characteristic weight matrix theta, solving the mapping matrix W, specifically, updating the target function

Is converted into

Using a generalized iterative power method pair

Calculating an orthogonal regression matrix;

wherein the content of the first and second substances,

j is a symmetric matrix, and the condition satisfied by the symmetric matrix is that J belongs to R ^7565×7565 ，B＝ΘXHY ^T Is the second placeGeneration parameters.

(7) Fixing the potential semantic matrix V and the mapping matrix W, and solving a weight matrix theta, wherein the method specifically comprises the following steps: target function after updating

Is converted into

Namely that

The condition satisfied is

And using the augmented Lagrange multiplier method to pair the updated objective function

The weight matrix Θ is calculated.

(7) Fixing the characteristic weight matrix theta and the mapping matrix W, and solving a potential semantic matrix V, which specifically comprises the following steps: for updated objective function

2[H ^T (V-X ^T ΘW)+α(V-Y)+ηLV+βVR]＝0

The latent semantic matrix V is calculated for the above-mentioned sierwist equation.

(9) According to the steps, the updated objective function is processed

Performing iterative operation, and continuously updating the characteristic weight matrix theta; iterating until the target function converges; diagonalizing the characteristic weight matrix theta to obtain an updated characteristic weight vector theta; and selecting features according to the updated feature weight vector theta, selecting features corresponding to elements with larger values in the theta, and constructing an emotion electroencephalogram feature subset for training a multi-dimensional emotion classification model.

4. And constructing a multi-dimensional emotion classification model between the emotion electroencephalogram characteristics and the multi-dimensional emotion through the classification model based on the emotion electroencephalogram characteristic subset obtained by selection. And performing emotion recognition based on the multi-dimensional emotion classification model.

(1) A multi-label-k neighbor classifier is adopted as a multi-dimensional emotion classification model, wherein the numbers of nearest neighbors and smoothness are respectively set to 10 and 1. And randomly determining 70% of samples from the obtained emotion electroencephalogram feature subset as a training set, and taking the rest 30% of samples as a test set. The method is used for training and testing the multi-dimensional emotion classification model.

(2) In the present example, seven indexes, namely Redundancy, Coverage distance, Hamming loss, Ranking loss, Average precision, Macro-F1 and Micro-F1, are used as evaluation indexes, and compared with the other 14 feature selection methods. FIG. 4 is a graph illustrating comparison results of redundancy indicators provided by an exemplary embodiment of the present disclosure; FIG. 5 is a graph illustrating a comparison of coverage distance indicators provided by an exemplary embodiment of the present disclosure; FIG. 6 is a graph illustrating a comparison of Hamming loss indicators provided by an exemplary embodiment of the present disclosure; FIG. 7 is a graphical illustration of a comparison of rank penalty indicators provided by an exemplary embodiment of the present disclosure; FIG. 8 is a graph illustrating a comparison of average accuracy indicators provided by an exemplary embodiment of the present disclosure; FIG. 9 is a graphical illustration of comparative results of the Macro-F1 metric provided by an exemplary embodiment of the present disclosure; FIG. 10 is a graphical illustration of a comparison of the Micro-F1 index provided by an exemplary embodiment of the present disclosure. Where the abscissa represents the number of feature choices and efsmder (outer) represents the method provided by the present disclosure.

Referring to fig. 4-10, it can be seen that the feature selection method based on the multidimensional emotion feature selection model provided by the present disclosure can obtain a better multidimensional emotion recognition result compared with other feature selection methods.

(3) In order to evaluate the convergence of the multidimensional emotion feature selection model, the present example further performs a convergence rate analysis on the iterative optimization algorithm, and fig. 11 is a schematic diagram of a convergence curve of the objective function value provided by an exemplary embodiment of the present disclosure. The horizontal axis represents iteration times, the vertical axis represents an objective function value, and the iterative optimization algorithm can be rapidly converged in a few iterations, so that the effectiveness of the multi-dimensional emotion feature selection algorithm is shown.

According to the processing method for emotion recognition, the original electroencephalogram feature space is mapped to the low-dimensional space through the orthogonal regression model, so that local correlation information between electroencephalogram features and multi-dimensional emotion labels is mined, label information in the low-dimensional space is constructed by combining global label correlation in the original multi-dimensional emotion label space, correlation among the features is evaluated from a global angle, redundant features are removed, a characteristic and non-redundant emotion electroencephalogram feature subset is selected, feature dimensionality is reduced, and calculation efficiency and recognition effect of emotion recognition are improved.

Any of the emotion recognition processing methods provided by the embodiments of the present disclosure may be executed by any suitable device with data processing capability, including but not limited to: terminal equipment, a server and the like. Alternatively, any of the emotion recognition processing methods provided by the embodiments of the present disclosure may be executed by a processor, for example, the processor may execute any of the emotion recognition processing methods mentioned in the embodiments of the present disclosure by calling a corresponding instruction stored in a memory. Which will not be described in detail below.

Exemplary devices

Fig. 12 is a schematic structural diagram of a processing apparatus for emotion recognition according to an exemplary embodiment of the present disclosure. The apparatus of this embodiment may be used to implement the corresponding method embodiment of the present disclosure, and the apparatus shown in fig. 12 includes: a first obtaining module 501, a first feature extraction module 502, a first processing module 503, a second processing module 504, and a third processing module 505.

The first obtaining module 501 is used for obtaining an emotion electroencephalogram signal to be identified; the first feature extraction module 502 is used for extracting at least two types of emotion electroencephalogram features based on emotion electroencephalogram signals to be identified to form a first feature matrix; the first processing module 503 is configured to determine a feature weight vector based on a multi-dimensional emotion feature selection model obtained through pre-training; the second processing module 504 is configured to perform feature selection on the first feature matrix based on the feature weight vector to obtain an electroencephalogram feature subset; and a third processing module 505, configured to obtain an emotion recognition result based on the electroencephalogram feature subset and a multi-dimensional emotion classification model obtained through pre-training.

Fig. 13 is a schematic structural diagram of a processing apparatus for emotion recognition according to another exemplary embodiment of the present disclosure.

In one optional example, the apparatus of the present disclosure further comprises: a second obtaining module 601, a first determining module 602, a first normalizing module 603 and a fourth processing module 604.

A second obtaining module 601, configured to obtain training sample data, where the training sample data includes training emotion electroencephalogram signals and corresponding emotion tag data; a first determining module 602, configured to determine a training emotion electroencephalogram feature matrix and a corresponding emotion tag matrix based on training sample data; the first standardization module 603 is used for carrying out normal distribution standardization on each type of emotion electroencephalogram characteristic in the training emotion electroencephalogram characteristic matrix to obtain a corresponding standardized characteristic matrix; the fourth processing module 604 performs iterative training on the mapping matrix, the latent semantic matrix, and the feature weight matrix of the multi-dimensional emotional feature selection model based on the standardized feature matrix, the emotion tag matrix, and the preset objective function until the value of the preset objective function converges, so as to obtain the trained multi-dimensional emotional feature selection model.

In an optional example, the first processing module 503 is specifically configured to: and determining a characteristic weight vector based on a characteristic weight matrix of the multi-dimensional emotional characteristic selection model.

In one optional example, the apparatus of the present disclosure further comprises:

a second determining module 605, configured to determine the selected training electroencephalogram feature subset and corresponding emotion tag data based on the feature weight vector; and a fifth processing module 606, configured to obtain a multi-dimensional emotion classification model through training based on the training electroencephalogram feature subset and the corresponding emotion label data, where the multi-dimensional emotion classification model adopts a multi-label k neighbor classifier.

In one alternative example, the preset objective function is:

s.t.W ^T W＝I _K ,θ ^T 1 _D ＝1,θ≥0

wherein X represents a normalized feature matrix, and X is equal to R ^D×N D represents the number of features in the normalized feature matrix, N represents the number of samples, s.t. represents the condition that follows is satisfied, theta represents the feature weight vector, theta is equal to R ^D×1 Theta ≧ 0 denotes that the weight value in the feature weight vector is greater than or equal to 0, theta denotes the feature weight matrix, theta ∈ R ^D×D Theta is a matrix obtained by diagonalizing the feature weight vector theta, W represents a mapping matrix, W is an orthogonal regression matrix, and W is an element of W ∈ R ^D×K K represents the dimension number of the emotion labels, and W satisfies the orthogonal regression constraint W ^T W＝I _K ，I _K Is an identity matrix of K multiplied by K, V represents a potential semantic matrix, and V belongs to R ^N×K Y represents an emotion tag matrix, and Y is equal to R ^N×K Alpha, eta, lambda, beta are equilibrium parameters, 1 _N A column vector of all 1, i.e. 1 _N ＝[1,1,…,1] ^T ∈R ^N×1 ，b∈R ^K×1 For the deviation vector, L is the graph Laplace matrix of the normalized feature matrix X, L ∈ R ^N×N A represents a global feature redundancy matrix, and A is equal to R ^D×D A is determined based on a standardized feature matrix X, R represents a global label incidence matrix, and R belongs to R ^K×K R, is determined based on the emotion tag matrix,

representing the Frobenius norm of the matrix.

In an optional example, the fourth processing module 604 is specifically configured to: in each iteration process, for the updating of the mapping matrix, the latent semantic matrix and the feature weight matrix, the other matrix is updated by fixing two of the matrices.

In an optional example, the second processing module 504 is specifically configured to: carrying out normal distribution standardization on each type of emotional electroencephalogram characteristics in the first characteristic matrix to obtain a corresponding standardized second characteristic matrix; determining target emotion electroencephalogram characteristics of which corresponding characteristic weight values are larger than a preset threshold value in a second characteristic matrix based on the characteristic weight vectors; and taking the target emotion electroencephalogram characteristics as an electroencephalogram characteristic subset.

In an alternative example, the first feature extraction module 502 is specifically configured to: preprocessing the emotion electroencephalogram signal to be recognized based on independent component analysis, removing physiological artifacts in the emotion electroencephalogram signal to be recognized, and obtaining a first emotion electroencephalogram signal after preprocessing; and performing feature extraction on the first emotion electroencephalogram signal to obtain at least two of a non-stationary index value, high-order intersection, spectral entropy, Shannon entropy, C0 complexity, differential entropy, absolute power ratio of beta wave band and theta wave band, amplitude of an intrinsic mode function, instantaneous phase of the intrinsic mode function, differential entropy difference of a space symmetric electrode, differential entropy ratio of the space symmetric electrode and functional connection to form a first feature matrix.

Exemplary electronic device

An embodiment of the present disclosure further provides an electronic device, including: a memory for storing a computer program;

and a processor, configured to execute the computer program stored in the memory, and when the computer program is executed, implement the processing method for emotion recognition according to any of the above embodiments of the present disclosure.

Fig. 14 is a schematic structural diagram of an application embodiment of the electronic device of the present disclosure. In this embodiment, the electronic device 10 includes one or more processors 11 and a memory 12.

The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

Memory 12 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium and executed by processor 11 to implement the methods of the various embodiments of the disclosure described above and/or other desired functionality. Various contents such as an input signal, a signal component, a noise component, etc. may also be stored in the computer-readable storage medium.

In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other form of connection mechanism (not shown).

The input means 13 may be, for example, a microphone or a microphone array as described above for capturing an input signal of a sound source.

The input device 13 may also include, for example, a keyboard, a mouse, and the like.

The output device 14 may output various information including the determined distance information, direction information, and the like to the outside. The output devices 14 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.

Of course, for simplicity, only some of the components of the electronic device 10 relevant to the present disclosure are shown in fig. 14, omitting components such as buses, input/output interfaces, and the like. In addition, the electronic device 10 may include any other suitable components depending on the particular application.

Exemplary computer program product and computer-readable storage Medium

In addition to the methods and apparatus described above, embodiments of the present disclosure may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform steps in methods according to various embodiments of the present disclosure as described in the "exemplary methods" section of this specification above.

The computer program product may write program code for carrying out operations for embodiments of the present disclosure in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform steps in methods according to various embodiments of the present disclosure as described in the "exemplary methods" section above of this specification.

The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The foregoing describes the general principles of the present disclosure in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present disclosure are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present disclosure. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the disclosure is not intended to be limited to the specific details so described.

In the present specification, the embodiments are described in a progressive manner, and each embodiment focuses on differences from other embodiments, and the same or similar parts in each embodiment are referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The block diagrams of devices, apparatuses, devices, systems involved in the present disclosure are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by one skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".

The methods and apparatus of the present disclosure may be implemented in a number of ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order for the steps of the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless specifically stated otherwise. Further, in some embodiments, the present disclosure may also be embodied as programs recorded in a recording medium, the programs including machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

It is also noted that in the devices, apparatuses, and methods of the present disclosure, each component or step can be decomposed and/or recombined. These decompositions and/or recombinations are to be considered equivalents of the present disclosure.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the disclosure to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.

Claims

1. A processing method for emotion recognition is characterized by comprising the following steps:

acquiring an emotion electroencephalogram signal to be identified;

extracting at least two types of emotional electroencephalogram characteristics based on the emotional electroencephalogram signals to be recognized to form a first characteristic matrix;

determining a characteristic weight vector based on a multi-dimensional emotional characteristic selection model obtained by pre-training, wherein an objective function of the multi-dimensional emotional characteristic selection model comprises a global characteristic redundancy matrix, a global label incidence matrix and an orthogonal regression matrix;

based on the feature weight vector, performing feature selection on the first feature matrix to obtain an electroencephalogram feature subset;

and obtaining an emotion recognition result based on the electroencephalogram feature subset and a multi-dimensional emotion classification model obtained by pre-training.

2. The method according to claim 1, wherein before determining the feature weight coefficients based on the multi-dimensional emotion feature selection model obtained by pre-training, the method further comprises:

acquiring training sample data, wherein the training sample data comprises training emotion electroencephalogram signals and corresponding emotion label data;

determining a training emotion electroencephalogram characteristic matrix and a corresponding emotion label matrix based on the training sample data;

carrying out normal distribution standardization on each type of emotion electroencephalogram characteristic in the training emotion electroencephalogram characteristic matrix to obtain a corresponding standardized characteristic matrix;

and iteratively training a mapping matrix, a potential semantic matrix and a feature weight matrix of the multi-dimensional emotional feature selection model based on the standardized feature matrix, the emotion label matrix and a preset target function until the value of the preset target function is converged, and obtaining the trained multi-dimensional emotional feature selection model.

3. The method of claim 2, wherein the determining a feature weight vector based on the multi-dimensional emotion feature selection model obtained by pre-training comprises:

determining the feature weight vector based on a feature weight matrix of the multi-dimensional emotion feature selection model;

after the trained multi-dimensional emotion feature selection model is obtained, the method further comprises the following steps:

determining the selected training electroencephalogram feature subset and corresponding emotion label data based on the feature weight vector;

and training to obtain the multi-dimensional emotion classification model based on the training electroencephalogram feature subset and the corresponding emotion label data, wherein the multi-dimensional emotion classification model adopts a multi-label k nearest neighbor classifier.

4. The method of claim 2, wherein the preset objective function is:

s.t.W ^T W＝I _K ,θ ^T 1 _D ＝1,θ≥0

wherein X represents the normalized feature matrix, X ∈ R ^D×N D represents the number of features in the normalized feature matrix, N represents the number of samples, s.t. represents the condition that follows is satisfied, theta represents a feature weight vector, theta is equal to R ^D×1 Theta ≧ 0 denotes that the weight value in the feature weight vector is greater than or equal to 0, theta denotes the feature weight matrix, theta ∈ R ^D×D Theta is a matrix obtained by diagonalizing the feature weight vector theta, W represents the mapping matrix, W is an orthogonal regression matrix, and W is an element of R ^D×K K represents the dimension number of the emotion labels, and W satisfies the orthogonal regression constraint W ^T W＝I _K ，I _K Is an identity matrix of K x K, V represents the latent semantic matrix, V is an element of R ^N×K Y represents the emotion label matrix, and Y is belonged to R ^N×K Alpha, eta, lambda, beta are equilibrium parameters, 1 _N A column vector of all 1, i.e. 1 _N ＝[1,1,…,1] ^T ∈R ^N×1 ，b∈R ^K×1 Is a deviation vector, L is a graph Laplace matrix of the normalized feature matrix X, L ∈ R ^N ^×N A represents the global feature redundancy matrix, and A is equal to R ^D×D A is determined based on the standardized feature matrix X, R represents a global label incidence matrix, and R belongs to R ^K×K R is determined based on the emotion tag matrix,

the Frobenius norm of the matrix is expressed.

5. The method according to claim 2, wherein the iteratively training a mapping matrix, a latent semantic matrix and a feature weight matrix of a multidimensional emotion feature selection model based on the normalized feature matrix, the emotion label matrix and a preset objective function until a value of the preset objective function converges to obtain the trained multidimensional emotion feature selection model comprises:

in each iteration process, for the updating of the mapping matrix, the latent semantic matrix and the feature weight matrix, updating the other matrix by fixing two of the matrices.

6. The method of claim 1, wherein the feature selection of the first feature matrix based on the feature weight vector to obtain a subset of brain electrical features comprises:

carrying out normal distribution standardization on each type of emotion electroencephalogram characteristics in the first characteristic matrix to obtain a corresponding standardized second characteristic matrix;

determining target emotion electroencephalogram characteristics of which corresponding characteristic weight values are larger than a preset threshold value in the second characteristic matrix based on the characteristic weight vector;

and taking the target emotion electroencephalogram features as the electroencephalogram feature subset.

7. The method according to any one of claims 1 to 6, wherein said extracting at least two types of emotion electroencephalogram features based on said emotion electroencephalogram signal to be recognized to form a first feature matrix, comprises:

preprocessing the emotion electroencephalogram signal to be recognized based on independent component analysis, removing physiological artifacts in the emotion electroencephalogram signal to be recognized, and obtaining a first emotion electroencephalogram signal after preprocessing;

and performing feature extraction on the first emotion electroencephalogram signal to obtain at least two of a non-stationary index value, high-order intersection, spectral entropy, Shannon entropy, complexity of C0, differential entropy, absolute power ratio of beta wave band to theta wave band, amplitude of an intrinsic mode function, instantaneous phase of the intrinsic mode function, differential entropy difference of a space symmetric electrode, differential entropy ratio of the space symmetric electrode and functional connection to form the first feature matrix.

8. An emotion recognition processing apparatus, comprising:

the first acquisition module is used for acquiring emotion electroencephalogram signals to be identified;

the first feature extraction module is used for extracting at least two types of emotional electroencephalogram features based on the emotional electroencephalogram signals to be identified to form a first feature matrix;

the first processing module is used for selecting a model based on multi-dimensional emotional characteristics obtained by pre-training and determining a characteristic weight vector;

the second processing module is used for performing feature selection on the first feature matrix based on the feature weight vector to obtain an electroencephalogram feature subset;

and the third processing module is used for obtaining an emotion recognition result based on the electroencephalogram feature subset and a multi-dimensional emotion classification model obtained through pre-training.

9. A computer-readable storage medium storing a computer program for executing the emotion recognition processing method according to any one of claims 1 to 7.

10. An electronic device, the electronic device comprising:

a processor;

a memory for storing the processor-executable instructions;

the processor is used for reading the executable instructions from the memory and executing the instructions to realize the emotion recognition processing method of any one of the claims 1-7.