CN114366122A

CN114366122A - Motor imagery analysis method and system based on EEG brain-computer interface

Info

Publication number: CN114366122A
Application number: CN202111501604.0A
Authority: CN
Inventors: 姜岩芸; 隋晓丹; 郑元杰; 赵艳娜
Original assignee: Shandong Normal University
Current assignee: Shandong Normal University
Priority date: 2021-12-09
Filing date: 2021-12-09
Publication date: 2022-04-19

Abstract

The invention provides a motor imagery analysis method based on an EEG brain-computer interface, which comprises the steps of obtaining brain electrical data; extracting features according to the acquired electroencephalogram data; according to the extracted features, feature selection and classification are carried out, so that an analysis result is obtained; the feature extraction comprises the steps of extracting time domain features through autoregressive modeling and extracting frequency domain features through fast Fourier transform. The invention uses an EEG brain-computer interface to obtain original brain-computer data, estimates the imaginary limb movement of a subject through real-time analysis of the data, and helps the subject to complete the imaginary limb movement through traction of a movement auxiliary device. The invention is helpful for assisting the limb movement recovery of the brain injury patient.

Description

Motor imagery analysis method and system based on EEG brain-computer interface

Technical Field

The invention relates to the technical field of electroencephalogram analysis, in particular to a motor imagery analysis method and system based on an EEG brain-computer interface.

Background

The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.

In recent years, brain-computer interfaces (BCIs) have been used to convert brain signals into computer data, which means that computers can be consciously controlled by monitoring the signal activity of the brain.

Compared with other methods such as functional magnetic resonance imaging (fMRI) and Magnetoencephalogram (MEG), the electroencephalogram signal is non-invasive, has the advantages of high time resolution, portability, and relatively low cost, and is widely used for recording the brain signal in the BCI system. EEG-based BCI can be divided into two main types: evoked and spontaneous, also known as exogenous and endogenous. In an evoked system, an external stimulus, such as a visual, auditory, or sensory stimulus, is required. The stimulus elicits a response in the brain, which is then recognized by the BCI system to determine the user's intent. In spontaneous BCI, no external stimulus is required and control actions are taken according to the activities produced by mental activities.

One common example of an evoked BCI is motor-image (MI), which is generated by a subject when imagining the motion of a limb. The subject receives visual, auditory or sensory stimuli that elicit a brain response that is then recognized by the BCI system to determine the user's intent. This BCI monitors the sensorimotor rhythms (SMRs), which are oscillatory events of the brain region in the EEG signal that are relevant to preparing, controlling, and performing motion. The brain-computer interface of the motor imagery data is possibly a clinical breakthrough technology. For example, the general classes of motion of the myocardial infarction EEG system include: left hand movement, right hand movement, both feet movement and tongue movement.

Motor imagery based brain-computer interfaces also present several challenges due to the complexity of EEG signals, and the close relationship of signal quality to user mental state: (1) the EEG signal may be affected by the posture and mood of the subject. For example, during recording, an upright posture tends to improve focus and electroencephalogram quality, with higher frequency content being stronger when the user is in an upright position. (2) The performance of SMR-BCI depends to a large extent on the user's neurophysiological and psychological states, and many users find control of SMR activity challenging.

In summary, the prior art is used for the motor imagery analysis problem, and a solution with high accuracy and high efficiency is not available.

Disclosure of Invention

In order to solve the problems, the invention provides a motor imagery analysis method and system based on an EEG brain-computer interface, which can help to assist the limb movement recovery of a brain injury patient.

According to some embodiments, the invention adopts the following technical scheme:

a motor imagery analysis method based on an EEG brain-computer interface, comprising:

acquiring original electroencephalogram data;

extracting features according to the acquired original electroencephalogram data;

according to the extracted features, feature selection and classification are carried out, so that an analysis result is obtained;

the feature selection and classification comprises the steps of carrying out principal component analysis on the electroencephalogram data, transmitting the most discriminative feature in the extracted features to a classifier, and estimating the body movement imagined by the subject according to the classification result.

Further, the acquiring of the brain electrical data comprises the EEG based BCI using data recorded from a plurality of EEG channels.

Further, the acquiring of the electroencephalogram data comprises preprocessing the electroencephalogram data.

Further, the feature extraction comprises extracting time domain features through autoregressive modeling and extracting frequency domain features through fast Fourier transform.

Further, the feature extraction also comprises the step of extracting discriminant and non-redundant information from the electroencephalogram data to form a group of features which can be classified.

Further, the principal component analysis includes:

(1) calculating each column X in the data matrix of the orthogonal matrix of the wavelet decomposition of each level J;

(2) for m is more than or equal to 1 and less than or equal to J, PCA of the detail matrix is executed and a proper number of important principal components are selected;

(3) PCA of the approximation matrix is performed and an appropriate number of principal components are selected,

(4) restoring a new matrix from the reduced detail matrix and the approximate matrix by inverting the wavelet transform;

(5) performing PCA of the new matrix to form

Where X is an n X p data matrix and the orthogonal matrix W contains filter coefficients G_mAnd H_JOf the matrix of (a).

Further, the feature selection and classification further includes filter bank selection, specifically including selecting the most discriminative features to input into a classifier for classification.

Further, the feature selection and classification further comprises selecting features through an evolutionary algorithm based on systematic classification accuracy.

An EEG brain-machine interface based motor imagery analysis system comprising:

a data acquisition module configured to acquire raw electroencephalogram data;

the characteristic extraction module is configured to extract characteristics according to the acquired original electroencephalogram data;

an analysis module configured to perform feature selection and classification according to the extracted features, thereby obtaining an analysis result;

A computer readable storage medium having stored therein a plurality of instructions adapted to be loaded by a processor of a terminal device and to execute a method of analysis of motor imagery based on an EEG brain-computer interface.

A terminal device comprising a processor and a computer readable storage medium, the processor being configured to implement instructions; the computer readable storage medium stores instructions adapted to be loaded by a processor and to perform a method of analysis of motor imagery based on an EEG brain-computer interface.

Compared with the prior art, the invention has the beneficial effects that:

in the practical aspect, the invention uses an EEG brain-computer interface to obtain original brain-computer data, estimates the imaginary limb movement of the testee through real-time analysis of the data, and helps the testee to complete the imaginary limb movement through traction of the motion auxiliary equipment. The invention is helpful for assisting the limb movement recovery of the brain injury patient.

In the expansibility, the method of the invention comprises several main modules of original data acquisition, data preprocessing, feature extraction, feature selection and feature classification. Each module operates independently and is easily modified and upgraded. Taking feature extraction as an example, the method provides a time domain and frequency domain technology, a time-frequency domain technology and a spatial mode. And the feature extraction method can be changed according to requirements at the later stage.

On the aspect of operation accuracy and operation speed, the method segments the acquired long-time electroencephalogram signal, and performs motor imagery analysis by taking 3 seconds as an operation period. The operation period of 3 seconds ensures the operation accuracy and simultaneously realizes the real-time analysis of the model.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application.

FIG. 1 is a flowchart of the present embodiment 1;

FIG. 2 is a diagram of a set of EEG examples of the present embodiment 1;

FIG. 3 is a filter diagram of the present embodiment 1 at scale level 2;

FIG. 4 is a schematic diagram of the multi-scale principal component analysis method of the present embodiment 1;

fig. 5 is a schematic diagram of the feature extraction and feature selection process of this embodiment 1.

The specific implementation mode is as follows:

the invention is further described with reference to the following figures and examples.

It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.

Example 1

As shown in fig. 1, a motor imagery analysis method based on an EEG brain-computer interface includes:

acquiring original electroencephalogram data;

In particular, the method comprises the following steps of,

as shown in FIG. 1, the invention provides a motor imagery analysis method based on an EEG brain-computer interface, which comprises five main parts of acquiring original brain electrical data, preprocessing the brain electrical data, extracting features, selecting features and classifying.

S1, acquiring original electroencephalogram data:

an EEG based brain-computer interface uses data recorded from multiple EEG channels rather than a single channel. 32 channels of electroencephalogram data are acquired through the brain-computer interface, and the visualized waveform of the electroencephalogram data is shown in figure 2. The subject elicits responses in the brain by receiving visual, auditory, or sensory stimuli, which are then recognized by the BCI system to determine the intent of the user. Each trial acquisition was approximately 1 hour long.

It should be noted that there are many kinds of electroencephalogram acquisition devices, and the invention is not limited to electroencephalogram acquisition devices. A32-channel brain-computer interface is used in the method, but is not limited to this brain-computer interface.

S2, preprocessing electroencephalogram data:

preprocessing the EEG data to be analyzed to reduce the influence of noise in the EEG signal.

Taking multi-scale principal component analysis as an example: the multi-scale PCA denoising method combines the functions of orthogonal wavelets and principal component analysis. PCA is used to extract the relationships between a number of variables, orthogonal wavelets separate the stochastic from the deterministic process, and perform a coarse decorrelation of the autocorrelation between measurements. For most signal processing applications, wavelet analysis is better described using a filter bank. The simplest structure produces a high-pass output signal d₁(k) Low pass output signal c₁(k) And different levels of bandpass output signals therebetween, as shown in fig. 3.

The sampling frequency of the EEG data is 256Hz, which means that the maximum frequency captured in the EEG signal is 128 Hz. Thus, a high-pass signal d₁(k) In the range of 64-128Hz, a band-pass signal d₂(k) Is 32-64Hz, low-pass signal c₂(k) Is 0-32 Hz.

With filters

Represents the projection on the scaling function with a filter h_ψThe convolution of (a) represents a projection on the wavelet. If H (z), G (z), C_m(z) and D_m(z) are each

h_ψ，c_mAnd d_mBy z transform we can write as follows:

C_m(z)＝H(z)C_m-1(z)

D_m(z)＝G(z)C_m-1(z)

generating detail coefficients d_m(k) Represents the high-pass filter h_ψAnd more low-pass filters

Is cascaded. On the other hand, coefficient c_m(k) Only by cascaded low pass filters. In addition, the original waveform can be regarded as the finest scale m ═ 0, x (k) ═ c₀(k) The vector of scaling function coefficients. Thus, the original signal x (z) is used to be represented as:

C_m(z)＝H_m(z)X(z)

D_m(z)＝G_m(z)X(z)

wherein H_m(z) is obtained by applying H (z) filter m times, and G_m(z) is obtained by applying G (z) filter once and H (z) filter (m-1) times. Similar to EEG data, the detail signal d₃(k)、d₄(k) And d₅(k) In the range of 16-32Hz, 8-16Hz and 4-8Hz, respectively, and a low-pass signal c₅(k) In the range of 0-4 Hz.

Let X be an n X p data matrix containing p signals (waveforms) of length n. Each variable in X associated with a signal column is decomposed into wavelet coefficients using an orthogonal matrix W, representing a wavelet transform operator. The matrix W contains filter coefficients G_mAnd H_JThe matrix of (a):

W＝[H_JG_JG_J-1 ... G_m ... G₁]^T

the steps of the multiscale principal component analysis are shown in FIG. 4:

calculating each column X in the data matrix of each level J wavelet decomposition WX;

for m ≦ J of 1 ≦ m, the detail matrix G is implemented_mPCA of X and selecting a proper amount of important principal components;

performing an approximation matrix H_JThe PCA of X and choosing the appropriate number of principal components,

by inverting wavelet transform W^TRecovering a new matrix from the reduced detail matrix and the approximate matrix;

finally, PCA of the new matrix is performed to form

Furthermore, the acquired electroencephalogram data are divided into a plurality of sections according to time nodes. In the method, original data of about 1 hour is intercepted in an operation period of 3s and an overlapping time of 1.5s, and a new data set is generated.

It should be noted that the operation cycle time is not fixed to 3s, and may be any time period such as 1s, 3s, 5s, and the like. In this method, in order to reduce the waiting time of model operation while ensuring the calculation accuracy, we select 3s as the operation period.

S3, feature extraction, feature selection and classification: the most discriminative features are extracted and passed to the classifier.

Feature extraction is a signal processing step, which extracts discriminant and non-redundant information from the electroencephalogram data to form a group of features which can be classified. The most basic feature extraction technique uses time domain or frequency domain analysis to extract features. The time-frequency domain analysis technique is a more advanced and complex feature extraction technique that relates spectral information to the time domain. Analysis in the spatial domain using common spectral patterns is also a popular feature extraction method.

A method for EEG electroencephalogram feature extraction, comprising:

time domain and frequency domain techniques: while the signal can be modeled as a laplacian and gaussian random process, the time domain features are calculated from the amplitude of the signal. There are two main classes of temporal features, IEEG and RMS, that are calculated and extracted from the segmented EEG. RMS is modeled as an amplitude modulated gaussian random process, while IEEG estimates the power of the EEG signal. Respectively expressed as:

frequency domain analysis is also used to extract features from MI EEG data. The power spectrum is obtained using a fast fourier transform. Or using local feature scale decomposition (LCD) to decompose the signal into inherently scaled components having characteristic instantaneous frequencies related to the original signal features.

Time-frequency domain techniques to examine time-varying features within an EEG segment, we need to transform the signal in the time-frequency domain or in the wavelet domain. Therefore, time-frequency features and wavelet-based features are also used for MI BCI. The time-frequency analysis is powerful because it correlates the spectral information about the EEG signal with the time domain, which is advantageous for BCI techniques. The spectrum of brain activity also changes as different tasks are performed during use of the system. Methods for MI EEG analysis include Short Time Fourier Transform (STFT), Wavelet Transform (WT), and Discrete Wavelet Transform (DWT). The decomposition methods, WT and DWT, are very powerful because different EEG signal bands contain different MI action information and they can be used for signals decomposed at multiple resolutions and scales. DWT and WT have the ability to derive dynamic features, which are particularly important in EEG signals because they are non-stationary, non-linear and non-gaussian.

Spatial mode: the common spatial mode (CSP) is one of the most commonly used feature extraction methods in MI EEG classification for converting EEG data to a new space. Where the variance of one class is the largest and the variance of the other class is the smallest. It is a powerful technique for MI EEG processing because different frequency bands of the signal contain different information, and the CSP is able to extract this information from a particular frequency band.

S4, the method for feature selection comprises the following steps:

and (3) main component analysis: analysis and dimensionality reduction techniques, including PCA and Independent Component Analysis (ICA), have also been applied to MI EEG. PCA has been used for both downscaling and feature selection to improve classification. In some cases, both PCA and ICA are used in feature extraction along with other signal processing techniques.

And (3) filter bank selection: to improve the conventional CSP analysis feature extraction capability, the filter bank CSP (fbcsp) solves the problem of not utilizing the intrinsic connection between the frequency bands and the CSP features by estimating the mutual information contained in the CSP features from the individual sub-bands. By selecting those features that are most discriminative, the selected features are input into the SVM for classification. Bayesian learning is used to select CSP features from a plurality of EEG subsections, which are then input into an SVM classifier. Compared with the most advanced technology, the performance of the algorithm can be improved by means of Bayesian learning.

S5, feature classification

An evolutionary algorithm: one key issue in BCI development is the high dimensional number of data during feature extraction. Typical dimension reduction and feature selection methods (e.g., PCA and ICA) involve complex feature transformations, resulting in large computational requirements and larger size feature sets. Even if the variance of the data is acceptable, these methods typically result in low classification accuracy, possibly because the basic feature extraction tends to retain some redundant features. In addition, linear transformations are often used to reduce the dimensionality of the feature set. An Evolutionary Algorithm (EA) selects features through optimization based on systematic classification accuracy.

Taking differential evolution optimization (DE) as an example, computational requirements are reduced by selecting only relevant features while improving the effectiveness of the feature set. Fig. 5 illustrates the flow of the DE-based feature extraction and feature selection process, which implements a hybrid approach in which features are extracted using CSP, an optimized feature subset is selected using the DE algorithm, and only these features are passed to the classifier.

Example 2

An EEG brain-machine interface based motor imagery analysis system comprising:

a data acquisition module configured to acquire raw electroencephalogram data;

Example 3

A computer-readable storage medium, wherein a plurality of instructions are stored, and the instructions are adapted to be loaded by a processor of a terminal device and execute a motor imagery analysis method based on an EEG brain-computer interface provided in embodiment 1.

Example 4

A terminal device comprising a processor and a computer readable storage medium, the processor being configured to implement instructions; the computer readable storage medium is used for storing a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the motor imagery analysis method based on the EEG brain-computer interface provided by the embodiment 1.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive efforts by those skilled in the art based on the technical solution of the present invention.

Claims

1. A motor imagery analysis method based on an EEG brain-computer interface is characterized by comprising the following steps:

acquiring original electroencephalogram data;

2. The method of claim 1, wherein said obtaining brain electrical data comprises EEG-based BCI using data recorded from a plurality of EEG channels.

3. The method of claim 1, wherein the acquiring of the brain electrical data further comprises preprocessing the brain electrical data.

4. The method of claim 1, wherein the feature extraction further comprises extracting discriminative and non-redundant information from the EEG data to form a set of features that can be classified.

5. The method of claim 1, wherein said feature selection and classification comprises principal component analysis of electroencephalogram data, said principal component analysis comprising:

(5) performing PCA of the new matrix to form

6. The method of claim 5, wherein the feature selection and classification further comprises filter bank selection, and in particular comprises selecting the most discriminative features to be input into a classifier for classification.

7. The method of claim 6, wherein said feature selection and classification further comprises feature selection by an evolutionary algorithm based on systematic classification accuracy.

8. An EEG brain-computer interface based motor imagery analysis system, comprising:

a data acquisition module configured to acquire raw electroencephalogram data;

9. A computer-readable storage medium having stored thereon instructions adapted to be loaded by a processor of a terminal device and to perform a method of EEG brain-computer interface based motor imagery analysis according to any one of claims 1 to 7.

10. A terminal device comprising a processor and a computer-readable storage medium, the processor being configured to implement instructions; a computer readable storage medium for storing a plurality of instructions adapted to be loaded by a processor and to perform a method of EEG brain-computer interface based motor imagery analysis according to any one of claims 1 to 7.