CN111150372B

CN111150372B - Sleep stage staging system combining rapid representation learning and semantic learning

Info

Publication number: CN111150372B
Application number: CN202010090052.8A
Authority: CN
Inventors: 向鸿鑫; 杨云
Original assignee: Yunnan University YNU
Current assignee: Yunnan University YNU
Priority date: 2020-02-13
Filing date: 2020-02-13
Publication date: 2021-03-16
Anticipated expiration: 2040-02-13
Also published as: CN111150372A

Abstract

The invention provides a sleep stage staging system combining rapid representation learning and semantic learning, which comprises a signal acquisition unit, a data preprocessing unit, a signal processing unit and a prediction result processing unit, wherein the signal acquisition unit is used for acquiring a sleep stage of a user; the signal acquisition unit comprises an EEG signal acquisition module and a knowledge data acquisition module which are respectively used for acquiring the original EEG signal and the related knowledge of the sleep stage; the data preprocessing unit comprises a feature preprocessing module and a knowledge preprocessing module which respectively preprocess the acquired EEG signals and knowledge data; the signal processing unit comprises a presentation learning module and a semantic learning module, and can effectively extract EEG signal characteristics and semantic characteristics related to sleep stages and obtain two classification results; and finally, weighting and fusing the two classification results through a prediction result processing unit to obtain a final result. The system improves the training and predicting speed, effectively utilizes the signal characteristics and the semantic characteristics, and greatly improves the accuracy of the sleep stage staging.

Description

Sleep stage staging system combining rapid representation learning and semantic learning

Technical Field

The invention relates to the field of signal processing and sleep stage staging, in particular to a sleep stage staging system combining fast representation learning and semantic learning.

Background

Traditional sleep stage scoring based on physician observations is very cumbersome, time consuming and subjective, requiring the physician to analyze the signals in the PSG recordings to arrive at a sleep score of about 8 hours. Therefore, many automatic sleep evaluation methods are proposed. These studies first extract various sleep-related features from the electroencephalogram signals, such as time domain features, frequency features, correlation features, entropy features, and the like, and then use machine learning methods (decision trees, support vector machines, and the like) to classify the extracted features. However, these feature extraction methods are subjective, and many potential sleep features are not mined, which can play a key role in automatic sleep stage assessment.

More and more deep learning methods are currently applied to sleep classification. Convolutional Neural Networks (CNN) are used to extract time-invariant local features. Recurrent Neural Networks (RNNs) are used to learn time-series related information, for example to mine temporal associations between different sequences in EEG within the same 30 s. However, in order to ensure the performance of the models, the models are often very complex, which results in that the training speed and the prediction speed of the deep learning methods are very slow and are difficult to use in an EEG data online learning scenario, such as the deep sleepnet model proposed by a. However, the network structure is relatively complex, and the training time of the method is as long as 3 hours; phan, which is the simplest structure of one-to-one 1-maxCNN, extracts features using CNN only and classifies them using softmax layer. But the training time still needs 1 hour, and there is a lot of knowledge about sleep stage stages in the network, which can be used as auxiliary information for sleep stage evaluation. However, no method is available for effectively utilizing the knowledge to improve the classification accuracy. Therefore, the invention provides a sleep staging method with high accuracy and low computational complexity to solve the problem to be solved urgently in the prior art.

Disclosure of Invention

The invention provides a sleep stage staging system combining rapid expression learning and semantic learning, which utilizes a shallow network for improving training and prediction speed by utilizing the rapid expression learning and ensures accuracy at the same time, and also provides a semantic feature extraction method for effectively extracting and applying a large amount of sleep staging semantic information to improve the performance of the system.

The system firstly uses a deep learning method to extract EEG signal characteristics in the sleep process and obtains a classification result 1, secondly uses a generator in ACGAN to extract signal characteristics with semantic information, uses the signal characteristics with the semantic information to train any machine learning algorithm and obtain a classification result 2, and finally weights and fuses the result 1 and the result 2 to obtain a final classification result, and the system comprises the following unit modules:

1. a signal acquisition unit: the method comprises an EEG signal acquisition module and a knowledge data acquisition module, wherein the original EEG signal and the sleep stage related knowledge are acquired;

2. a data preprocessing unit: the EEG feature preprocessing module is used for preprocessing the acquired EEG signal and the acquired text information;

3. a signal processing unit: the EEG prediction method comprises a representation learning module and a semantic learning module, wherein preprocessed EEG signals and semantic features are respectively input into the representation learning module and the semantic learning module to obtain a prediction result 1 and a prediction result 2;

4. a prediction result processing unit: and performing weighted fusion on the obtained prediction result 1 and the prediction result 2 to obtain a final result.

Preferably, the raw EEG signals are collected from public sleep databases such as SleepEDF, MASS, etc., and knowledge of sleep staging decisions is collected from the manual for sleep and related event interpretation published by AASM, and descriptions about individual sleep staging characteristics in wikipedia.

Preferably, the EEG signal is acquired at a frequency of 100Hz, 30s is used as a period for evaluating a sleep stage, a data point with a length of 3000 determines a result of the sleep stage, the EEG data is divided into 25 or any segment, and the divided data is longitudinally spliced to obtain (25,120) two-dimensional data as an input for representing the learning module.

Preferably, stop words are removed from the acquired text information, a jieba word segmentation device is used for word segmentation, and finally the semantic features are extracted by using a TF-IDF method.

Preferably, the representation learning module comprises a first layer consisting of 3 parallel Convolutional Neural Networks (CNN) and 2 serial bidirectional long and short memory networks (BilSTM), the CNN is used for extracting time invariance characteristics, the BilSTM is used for extracting time domain characteristics, the sizes of filters of the three parallel CNN are respectively from small to large, the small filter can better capture electroencephalogram characteristics of a specific mode, and the large filter can better capture frequency information of global electroencephalogram signals.

Preferably, after each convolutional layer of the learning module is represented, training of the convolutional layer is accelerated and performance of the convolutional layer is optimized through Batch Normalization (BN), a linear rectification unit (ReLU) is used as an activation function, sampling is carried out through a maximum pool layer, and finally features extracted through CNNs and BilSTM are transversely spliced and spread to a softmax layer to obtain a classification result.

Preferably, the semantic learning module extracts features through two fully-connected layers to reduce text noise, the extracted features are connected with 10-dimensional Gaussian random noise in series, the feature dimensions are raised to 3000 dimensions by using two FC layers with LeakyReLU activators, a discriminator is trained through 3000-dimensional data generated and real 3000-dimensional EEG data, loss of the generator is calculated to guide the training of the generator, and a prediction result is obtained through predicting original EEG signals of a test set.

Preferably, the result data output by the presentation learning module and the result output by the semantic learning module are weighted and fused to obtain a final result.

Preferably, the prediction result obtained in the semantic learning module can be fused with any other machine learning model trained by the EEG signal to obtain the final result.

The method enables the network structure to be smaller and smaller, reduces parameters, better captures EEG signal characteristics of a specific mode through filters with different sizes, also better captures frequency information of global EEG signals, shortens training time and prediction time, ensures effectiveness and accuracy of the characteristics, simultaneously uses semantic characteristics and EEG signal characteristics to carry out antagonistic training, fully utilizes knowledge about sleep stage stages, and improves classification accuracy.

Drawings

FIG. 1 is an automatic sleep stage staging flow diagram;

FIG. 2 is a network architecture diagram illustrating a learning model;

FIG. 3 is a flow diagram of a semantic learning module;

Detailed Description

In the following description, technical solutions in the embodiments of the present invention are clearly and completely described, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The embodiment of the invention provides a sleep stage staging system combining rapid representation learning and semantic learning, which comprises the following unit modules as shown in figure 1: the signal acquisition unit comprises an EEG signal acquisition module and a knowledge data acquisition module and is used for acquiring related knowledge of an original EEG signal and a sleep stage, the data preprocessing unit comprises a feature preprocessing module and a knowledge preprocessing module and preprocesses the acquired EEG signal and text information, the signal processing unit comprises a representation learning module and a semantic learning module and respectively inputs the preprocessed EEG signal and semantic features into the representation learning module and the semantic learning module to obtain a prediction result 1 and a prediction result 2, and the prediction result processing unit performs weighted fusion on the obtained prediction result 1 and the prediction result 2 to obtain a final result.

In this example, EEG signals were obtained from SleepEDF public sleep databases, and the target subjects were white males and females aged 21-35 years without any sleep-related drugs and diseases, and the data included horizontal electrooculogram, electroencephalogram of Fpz-Cz channel, and electroencephalogram of Pz-Oz channel, and data for each channel were sampled at 100Hz for 30 seconds as a cycle, and each sample was 3000 data points.

As shown in fig. 2, the EEG signal is feature preprocessed, 3000 data points are equally divided into 25 segments with 120 data points, then the two segments are longitudinally spliced (25,120) into a two-dimensional matrix, the two-dimensional matrix is input into a representation learning module to perform feature extraction through 3 parallel convolutional neural networks CNN and 2 serial bidirectional long and short memory networks BiLSTM, three sequence operations of batch normalization BN, linear rectification with reluctnt as an activation function and maximum pool layer sampling are performed after each convolutional layer, and finally the features extracted by the convolutional layers are transversely spliced and input into a softmax layer to obtain a classification result 1 representing the learning module.

As shown in fig. 3, the method includes collecting knowledge for sleep stage determination from a manual for sleep and related event interpretation published by AASM and text information obtained from wikipedia about the description of each sleep stage feature, removing stop words, performing word segmentation by using a jieba word splitter, extracting features by using two full-connection layers, connecting the extracted features with 10-dimensional gaussian random noise in series, completing further feature extraction operation by using two FC layers with a leakyreu activator, increasing feature dimensions to 3000 dimensions, receiving 3000-dimensional data generated by a discriminator and real 3000-dimensional EEG data by the discriminator, training the generator by calculating loss of the generator, generating signal features with semantic information according to semantic features, and training an AdaBoost classifier to obtain a classification result 2.

As shown in fig. 1, the two classification results output by the presentation learning module and the semantic learning module are weighted and fused to obtain the final sleep stage staging result.

The invention is not limited to the foregoing embodiments. The invention extends to any novel feature or any novel combination of features disclosed in this specification and any novel method or process steps or any novel combination of features disclosed.

Claims

1. A sleep stage staging system that combines rapid presentation learning and semantic learning, comprising:

the signal acquisition unit comprises an EEG signal acquisition module and a knowledge data acquisition module, wherein the EEG signal acquisition module acquires an original EEG signal from a public sleep database, and the knowledge data acquisition module acquires knowledge for sleep stage judgment and description about each sleep stage characteristic from the knowledge database;

the data preprocessing unit is used for dividing the EEG signals obtained from the public sleep database into a training set, a verification set and a test set, carrying out equivalent segmentation processing on the EEG signals, and splicing and expanding the segmented signals;

the signal processing unit comprises a representation learning module and a semantic learning module, a training set which is obtained by dividing in the data preprocessing unit and is processed in a segmented mode is input into the representation learning module for training, a model is used for predicting a test set to obtain a prediction result 1, meanwhile, sleep stage knowledge in the signal acquisition unit is input into a generator in the semantic learning module, semantic features are extracted, and an EEG signal of the test set is predicted by using a semantic feature training classifier to obtain a prediction result 2;

and the prediction result processing unit obtains a final prediction result by weighting and fusing the two results predicted in the signal processing unit.

2. The sleep stage staging system for combined rapid representation learning and semantic learning according to claim 1, wherein the data preprocessing unit includes a feature preprocessing module and a knowledge preprocessing module, the feature preprocessing module divides the EEG signal into 25 equal parts and concatenates the 25 parts longitudinally into a two-dimensional matrix (25,120), the knowledge preprocessing module removes stop words from the text, performs word segmentation using a jieba tokenizer, and extracts semantic features using a TF-IDF method.

3. The sleep stage staging system for combined fast representation learning and semantic learning according to claim 2, wherein the representation learning module includes a convolutional layer and a softmax layer, the convolutional layer includes 3 parallel convolutional neural networks and 2 serial bidirectional long and short memory networks, the two-dimensional data with the shape of (25,120) processed by the data preprocessing unit is input into the convolutional layer for feature extraction, and the extracted features are transversely spliced and input into the softmax layer to obtain a prediction result 1.

4. The sleep stage staging system for combined fast representation learning and semantic learning according to claim 3, wherein the data in the representation learning module is output from the convolutional layer and then processed through quasi-normalization, linear rectification unit and max-pool-layer sampling.

5. The sleep stage staging system for combined rapid representation learning and semantic learning according to claim 4, wherein the semantic learning module includes two fully-connected layers and two FC layers with LeakyReLU activators, to be passed through the data

The semantic features extracted by the preprocessing unit are input into two full-connection layers and are connected with 10-dimensional Gaussian random noise obtained from Gaussian distribution sampling in series, and then the semantic features with 3000-dimensional feature dimensions are obtained by inputting the semantic features into two FC layers with LeakyReLU activators.

6. The sleep stage staging system for combined fast representation learning and semantic learning according to claim 5, wherein the semantic features have dimensions that are the same as the feature dimensions of the EEG signal, the classifier is trained on the semantic features, and a prediction result 2 is predicted from the test set of raw EEG signals.

7. The sleep stage staging system according to claim 6, wherein the prediction result 1 obtained in the representation learning module and the prediction result 2 obtained in the semantic learning module are weighted and fused to obtain the final prediction result.

8. The sleep stage staging system for combined rapid representation learning and semantic learning according to claim 6, wherein the prediction result 2 obtained in the semantic learning module can be fused with the prediction result 1 output by any other machine learning model trained using EEG signals to obtain the final result.