CN113111697B - Electroencephalogram signal identification system and method for three-dimensional static image in virtual reality environment - Google Patents
Electroencephalogram signal identification system and method for three-dimensional static image in virtual reality environment Download PDFInfo
- Publication number
- CN113111697B CN113111697B CN202011493152.1A CN202011493152A CN113111697B CN 113111697 B CN113111697 B CN 113111697B CN 202011493152 A CN202011493152 A CN 202011493152A CN 113111697 B CN113111697 B CN 113111697B
- Authority
- CN
- China
- Prior art keywords
- dimensional static
- dimensional
- machine learning
- learning model
- electroencephalogram
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/02—Preprocessing
- G06F2218/04—Denoising
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Signal Processing (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses an electroencephalogram signal identification system and method for a three-dimensional static image in a virtual reality environment. The system comprises a three-dimensional static image construction module, an electroencephalogram signal acquisition module, a machine learning model training module and an electroencephalogram signal identification module. The specific method comprises the following steps: three-dimensional static images comprising a stimulated target and a non-stimulated target are constructed, the brain electrical signals of a tested person in a three-dimensional virtual reality environment are collected, a machine learning model is trained, and the brain electrical signals are identified. The invention solves the problems that the collected brain electrical signals contain noise due to the fact that the testee is easy to be interfered by other things in the prior art, and the collected brain electrical signals are invalid brain electrical signals due to the fact that the stimulating picture is distorted, so that the collected brain electrical signals contain less noise and are effective brain electrical signals, and the accuracy of brain electrical signal identification results and the brain electrical signal collection efficiency are improved.
Description
Technical Field
The invention belongs to the technical field of information processing, and further relates to an electroencephalogram signal identification system and method for a three-dimensional static image in a virtual reality environment in the technical field of brain-computer interfaces. The invention can identify the brain electrical signal of the person in the virtual reality environment.
Background
The Brain-computer interface (BCI) technology realizes a new way of communicating and controlling external information which makes thought put into action by using engineering technical means, and is a cross technology related to a plurality of fields of medicine, neurology, signal detection, signal processing, pattern recognition and the like. The BCI system can realize communication between a person and the outside and control the external equipment by collecting information of the brain and directly converting the information into a command capable of driving the external equipment instead of limbs or language organs of the person.
The university of Tianjin discloses a brain-controlled rehabilitation system motor imagery recognition system integrating a complex network and graph convolution in the patent literature (patent application number CN202010364649.7, application publication number CN 111584027A) applied by Tianjin university. The system comprises a movement intention recognition module, an electromyographic signal acquisition module and a multichannel electric stimulation output module. The motion intention recognition module is used for the testee to respectively watch the hand fist making and hand stretching videos and simultaneously imagine corresponding actions of the videos, and the electroencephalogram signals of the testee are collected through the electroencephalogram signal collection equipment. The electromyographic signal acquisition module is used for applying electrical stimulation to the upper limb of the tested person and acquiring electromyographic signals after the electrical stimulation. The multichannel electric stimulation output module is used for generating a control instruction according to the classification result received from the exercise intention recognition module and the electromyographic signals after the electric stimulation, and controlling the multichannel electric stimulation output module to stimulate the upper limbs of the tested person. The system has the following defects: in the movement intention recognition module, the tested person is easy to be interfered by other things except the video content when watching the video, so that the acquired electroencephalogram signal contains noise.
The Beijing university of post discloses a brain-machine combined target detection method and system based on the RSVP paradigm in the patent literature (patent application number CN202010412626.9, application publication number CN 111597990A) applied by the university of post, namely, the RSVP paradigm. The system disclosed in the patent application comprises an aerial photo receiving module, a stimulation photo sequence generating module, an electroencephalogram signal acquisition module and an electroencephalogram signal identification module. The aerial photo receiving module is used for receiving the aerial photo returned by the unmanned aerial vehicle. The stimulation picture sequence generation module is used for cutting the aerial photo by adopting a picture cutting algorithm to obtain a stimulation picture sequence. And the electroencephalogram signal acquisition module is used for continuously presenting the stimulation picture sequence to a tested person and acquiring the electroencephalogram signal of the tested person. The electroencephalogram signal identification module is used for respectively identifying the electroencephalogram signal corresponding to each stimulation picture by utilizing the trained HDCA algorithm model, and the identification result is a target picture and a non-target picture. The method disclosed in this patent application comprises the steps of: the method comprises the steps of (1) receiving an aerial photograph returned by an unmanned aerial vehicle; (2) Cutting the aerial photo by adopting a grid method to obtain a stimulated photo sequence; (3) And continuously presenting a stimulation picture sequence to the tested person to acquire the brain electrical signal of the tested person. The method has the following defects: because the stimulating picture sequence is obtained by cutting the aerial photo by adopting a grid method, the stimulating picture is easy to distort, the acquired electroencephalogram signal is an invalid electroencephalogram signal, and the accuracy of electroencephalogram signal identification is reduced.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, and provides an electroencephalogram signal identification system and method for three-dimensional static images in a virtual reality environment, which are used for solving the problems that acquired electroencephalogram signals contain noise caused by interference of other objects on a tested person and the identification accuracy of the electroencephalogram signals is reduced caused by image distortion stimulation.
The idea for realizing the purpose of the invention is as follows: the electroencephalogram signals of the tested person are collected in the three-dimensional virtual reality environment, the tested person can only see the three-dimensional static target image, and noise contained in the collected electroencephalogram signals is reduced; at least 60 images including a stimulation target and a non-stimulation target are constructed, each image is a three-dimensional static image, and the target images are not easy to generate distortion, so that the acquired brain electrical signals are effective brain electrical signals, and the accuracy of brain electrical signal identification corresponding to the three-dimensional static stimulation target images is improved.
The electroencephalogram signal recognition system comprises a three-dimensional static image construction module, an electroencephalogram signal recognition module, a machine learning model training module and an electroencephalogram signal acquisition module, wherein:
the three-dimensional static image construction module is used for constructing at least 60 images comprising a stimulation target and a non-stimulation target, wherein each image is a three-dimensional static image, and the ratio of the number of the three-dimensional static images of the stimulation target to the number of the three-dimensional static images of the non-stimulation target is 1:5;
the electroencephalogram signal acquisition module is used for calling all three-dimensional static images in a three-dimensional virtual reality environment, randomly presenting a three-dimensional static image to a tested person each time, and acquiring electroencephalogram signals of the tested person after watching all the three-dimensional static images by the electroencephalogram signal acquisition system;
the machine learning model training module is used for forming a training set from all acquired electroencephalogram signals, constructing a machine learning model by utilizing a gradient enhancement decision tree method, inputting the training set into the machine learning model, and performing iterative updating training on the machine learning model by using an Adam algorithm to obtain a trained machine learning model;
the electroencephalogram signal recognition module is used for collecting electroencephalogram signals of a tested person to be recognized in a three-dimensional virtual reality environment, inputting the electroencephalogram signals to be recognized into a trained machine learning model, and obtaining an electroencephalogram signal corresponding to a three-dimensional static stimulation target image or an electroencephalogram signal corresponding to a three-dimensional static non-stimulation target image as a recognition result.
The method comprises the following steps:
step 1, constructing a three-dimensional static image:
constructing at least 60 images comprising a stimulated target and a non-stimulated target, wherein each image is a three-dimensional static image, and the ratio of the number of the three-dimensional static images of the stimulated target to the number of the three-dimensional static images of the non-stimulated target is 1:5;
step 2, acquiring brain electrical signals of a tested person in a three-dimensional virtual reality environment:
in a three-dimensional virtual reality environment, all three-dimensional static images are called, each time, a three-dimensional static image is randomly presented to a tested person, and an electroencephalogram signal acquisition system acquires electroencephalogram signals of the tested person after watching all the three-dimensional static images;
step 3, training a machine learning model:
all acquired electroencephalogram signals form a training set, a machine learning model is built by utilizing a gradient enhancement decision tree method, the training set is input into the machine learning model, and after iterative updating training is carried out on the machine learning model by using an Adam algorithm, a trained machine learning model is obtained;
step 4, recognizing an electroencephalogram signal:
and acquiring brain electrical signals of a tested person to be identified in a three-dimensional virtual reality environment, inputting the brain electrical signals to be identified into a trained machine learning model, and obtaining the brain electrical signals with identification results corresponding to the three-dimensional static stimulation target images or brain electrical signals corresponding to the three-dimensional static non-stimulation target images.
Compared with the prior art, the invention has the following advantages:
firstly, because the electroencephalogram signal acquisition module in the system can acquire the electroencephalogram signal of the tested person in the three-dimensional virtual reality environment, the tested person can only see the three-dimensional static target image in the process of acquiring the electroencephalogram signal, the problem that the acquired electroencephalogram signal contains a large amount of noise due to the fact that the tested person is easily interfered by other things in the prior art is solved, the electroencephalogram signal acquired by the system is more sufficient, and noise contained in the electroencephalogram signal corresponding to the three-dimensional static target image is reduced.
Secondly, as the method constructs at least 60 images comprising the stimulation target and the non-stimulation target, each image is a three-dimensional static image, the problem that the stimulation image obtained by cutting the images in the prior art is easy to distort, and the acquired brain electrical signals are invalid brain electrical signals is solved, so that the brain electrical signals acquired by the method are valid brain electrical signals, and the accuracy of brain electrical signal identification corresponding to the stimulation target image is improved.
Drawings
FIG. 1 is a block diagram of a system of the present invention;
fig. 2 is a flow chart of the method of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
The system of the present invention is further described with reference to fig. 1.
The electroencephalogram signal recognition system comprises a three-dimensional static image construction module, an electroencephalogram signal recognition module, a machine learning model training module and an electroencephalogram signal acquisition module.
The three-dimensional static image construction module is used for constructing 60 images comprising a stimulation target and a non-stimulation target by utilizing a unit 3D technology, wherein each image is a three-dimensional static image, and the ratio of the number of the three-dimensional static images of the stimulation target to the number of the three-dimensional static images of the non-stimulation target is 1:5.
The electroencephalogram signal acquisition module is used for calling all three-dimensional static images in a three-dimensional virtual reality environment after the HTC head-mounted virtual reality display is worn by a person to be tested, randomly presenting a three-dimensional static image to the person to be tested each time, and acquiring electroencephalogram signals of the person to be tested after watching all the three-dimensional static images by using the gtec electroencephalogram signal acquisition system.
The machine learning model training module is used for forming a training set by all acquired electroencephalogram signals, constructing a machine learning model by utilizing a gradient enhancement decision tree method, inputting the training set into the machine learning model, and obtaining a trained machine learning model after iterative updating training of the machine learning model by using an Adam algorithm.
The electroencephalogram signal recognition module is used for collecting electroencephalogram signals of a tested person to be recognized in a three-dimensional virtual reality environment, inputting the electroencephalogram signals to be recognized into a trained machine learning model, and obtaining an electroencephalogram signal corresponding to a three-dimensional static stimulation target image or an electroencephalogram signal corresponding to a three-dimensional static non-stimulation target image as a recognition result.
The steps of implementing the method of the present invention are further described with reference to fig. 2.
And 1, constructing a three-dimensional static image.
The embodiment of the method adopts a unit 3D technology to construct at least 60 images comprising a stimulated target and a non-stimulated target, wherein each image is a three-dimensional static image and is stored on a computer disk.
And 2, acquiring an electroencephalogram signal of the tested person in a three-dimensional virtual reality environment.
After the HTC head-mounted virtual reality display is worn by a tested person, in a three-dimensional virtual reality environment, all three-dimensional static images are called from a computer disk, each time, a three-dimensional static image is randomly presented to the tested person, and the brain electrical signals of the tested person after watching all the three-dimensional static images are collected by using the gtec brain electrical signal collecting system with the sampling frequency of 250 Hz.
And 3, training a machine learning model.
All acquired electroencephalogram signals are formed into a data set, the data set is randomly divided into a training set and a testing set according to the proportion of 7:3, a machine learning model is built by utilizing a gradient enhancement decision tree method, the training set is input into the machine learning model, and after the machine learning model is subjected to iterative update training 300 times by using an Adam algorithm, a trained machine learning model is obtained.
And 4, recognizing the brain electrical signals.
Inputting the test set into a trained machine learning model to obtain an electroencephalogram signal corresponding to the three-dimensional static stimulation target image or an electroencephalogram signal corresponding to the three-dimensional static non-stimulation target image as a recognition result, wherein the recognition accuracy of the electroencephalogram signal on the test set is 87.6%.
Claims (2)
1. The electroencephalogram signal identification system for the three-dimensional static image in the virtual reality environment comprises a three-dimensional static image construction module, a machine learning model training module and an electroencephalogram signal identification module, and is characterized by further comprising an electroencephalogram signal acquisition module, wherein:
the three-dimensional static image construction module is used for constructing at least 60 images comprising a stimulation target and a non-stimulation target, wherein each image is a three-dimensional static image, and the ratio of the number of the three-dimensional static images of the stimulation target to the number of the three-dimensional static images of the non-stimulation target is 1:5;
the electroencephalogram signal acquisition module is used for calling all three-dimensional static images in a three-dimensional virtual reality environment, randomly presenting a three-dimensional static image to a tested person each time, and acquiring electroencephalogram signals of the tested person after watching all the three-dimensional static images by the electroencephalogram signal acquisition system;
the machine learning model training module is used for forming a training set from all acquired electroencephalogram signals, constructing a machine learning model by utilizing a gradient enhancement decision tree method, inputting the training set into the machine learning model, and performing iterative updating training on the machine learning model by using an Adam algorithm to obtain a trained machine learning model;
the electroencephalogram signal recognition module is used for collecting electroencephalogram signals of a tested person to be recognized in a three-dimensional virtual reality environment, inputting the electroencephalogram signals to be recognized into a trained machine learning model, and obtaining an electroencephalogram signal corresponding to a three-dimensional static stimulation target image or an electroencephalogram signal corresponding to a three-dimensional static non-stimulation target image as a recognition result.
2. The method for recognizing the electroencephalogram signals of the three-dimensional static image in the virtual reality environment of the system according to claim 1 is characterized by constructing a three-dimensional static image, collecting the electroencephalogram signals of each three-dimensional static image which are randomly presented by a tested person in the three-dimensional virtual reality environment, forming a training set, and recognizing the electroencephalogram signals by utilizing a trained machine learning model; the method comprises the following steps:
step 1, constructing a three-dimensional static image:
constructing at least 60 images comprising a stimulated target and a non-stimulated target, wherein each image is a three-dimensional static image, and the ratio of the number of the three-dimensional static images of the stimulated target to the number of the three-dimensional static images of the non-stimulated target is 1:5;
step 2, acquiring brain electrical signals of a tested person in a three-dimensional virtual reality environment:
in a three-dimensional virtual reality environment, all three-dimensional static images are called, each time, a three-dimensional static image is randomly presented to a tested person, and an electroencephalogram signal acquisition system acquires electroencephalogram signals of the tested person after watching all the three-dimensional static images;
step 3, training a machine learning model:
all acquired electroencephalogram signals form a training set, a machine learning model is built by using a gradient enhancement decision tree method, the training set is input into the machine learning model, and after iterative updating training is carried out on the machine learning model by using an Adam algorithm, a trained machine learning model is obtained;
step 4, recognizing an electroencephalogram signal:
and acquiring brain electrical signals of a tested person to be identified in a three-dimensional virtual reality environment, inputting the brain electrical signals to be identified into a trained machine learning model, and obtaining the brain electrical signals with identification results corresponding to the three-dimensional static stimulation target images or brain electrical signals corresponding to the three-dimensional static non-stimulation target images.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011493152.1A CN113111697B (en) | 2020-12-17 | 2020-12-17 | Electroencephalogram signal identification system and method for three-dimensional static image in virtual reality environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011493152.1A CN113111697B (en) | 2020-12-17 | 2020-12-17 | Electroencephalogram signal identification system and method for three-dimensional static image in virtual reality environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113111697A CN113111697A (en) | 2021-07-13 |
| CN113111697B true CN113111697B (en) | 2023-07-28 |
Family
ID=76710182
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011493152.1A Active CN113111697B (en) | 2020-12-17 | 2020-12-17 | Electroencephalogram signal identification system and method for three-dimensional static image in virtual reality environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113111697B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111028919A (en) * | 2019-12-03 | 2020-04-17 | 北方工业大学 | Phobia self-diagnosis and treatment system based on artificial intelligence algorithm |
| CN111026267A (en) * | 2019-11-29 | 2020-04-17 | 北方工业大学 | VR electroencephalogram idea control interface system |
| CN111062250A (en) * | 2019-11-12 | 2020-04-24 | 西安理工大学 | Multi-subject motor imagery electroencephalogram signal identification method based on depth feature learning |
| CN111597990A (en) * | 2020-05-15 | 2020-08-28 | 北京邮电大学 | RSVP-model-based brain-computer combined target detection method and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10795441B2 (en) * | 2017-10-23 | 2020-10-06 | Korea University Research And Business Foundation | Method of recognizing user intention by estimating brain signals, and brain-computer interface apparatus based on head mounted display implementing the method |
-
2020
- 2020-12-17 CN CN202011493152.1A patent/CN113111697B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111062250A (en) * | 2019-11-12 | 2020-04-24 | 西安理工大学 | Multi-subject motor imagery electroencephalogram signal identification method based on depth feature learning |
| CN111026267A (en) * | 2019-11-29 | 2020-04-17 | 北方工业大学 | VR electroencephalogram idea control interface system |
| CN111028919A (en) * | 2019-12-03 | 2020-04-17 | 北方工业大学 | Phobia self-diagnosis and treatment system based on artificial intelligence algorithm |
| CN111597990A (en) * | 2020-05-15 | 2020-08-28 | 北京邮电大学 | RSVP-model-based brain-computer combined target detection method and system |
Non-Patent Citations (1)
| Title |
|---|
| 基于脑电信号的情绪分类;李娟;刘国忠;高洁;;北京信息科技大学学报(自然科学版)(第02期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113111697A (en) | 2021-07-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110069958B (en) | Electroencephalogram signal rapid identification method of dense deep convolutional neural network | |
| CN112120694B (en) | Motor imagery electroencephalogram signal classification method based on neural network | |
| CN109784023B (en) | Steady-state vision-evoked electroencephalogram identity recognition method and system based on deep learning | |
| CN109299751B (en) | EMD data enhancement-based SSVEP electroencephalogram classification method of convolutional neural model | |
| CN106529476B (en) | A kind of EEG feature extraction and classification method stacking network based on deep layer | |
| CN112022153B (en) | Electroencephalogram signal detection method based on convolutional neural network | |
| CN110555468A (en) | Electroencephalogram signal identification method and system combining recursion graph and CNN | |
| CN111783942A (en) | Brain cognition process simulation method based on convolution cyclic neural network | |
| CN111930238B (en) | Brain-computer interface system implementation method and device based on dynamic SSVEP (secure Shell-and-Play) paradigm | |
| CN111584032B (en) | Novel brain-controlled intelligent rehabilitation method based on capsule network and application | |
| CN111714118A (en) | Brain cognition model fusion method based on ensemble learning | |
| CN109816630A (en) | FMRI visual coding model building method based on transfer learning | |
| CN112732092B (en) | Surface electromyogram signal identification method based on double-view multi-scale convolution neural network | |
| CN108523883A (en) | A kind of continuous Mental imagery identifying system of left and right index finger based on actual act modeling | |
| CN117520891A (en) | Motor imagery electroencephalogram signal classification method and system | |
| CN111966218A (en) | Two-dimensional cursor control method and device | |
| CN111772629A (en) | Brain cognitive skill transplantation method | |
| CN113128353B (en) | Emotion perception method and system oriented to natural man-machine interaction | |
| CN114601476A (en) | EEG signal emotion recognition method based on video stimulation | |
| CN113111697B (en) | Electroencephalogram signal identification system and method for three-dimensional static image in virtual reality environment | |
| CN113974627A (en) | Emotion recognition method based on brain-computer generated confrontation | |
| CN116541751B (en) | Electroencephalogram signal classification method based on brain function connection network characteristics | |
| CN116421200A (en) | Brain electricity emotion analysis method of multi-task mixed model based on parallel training | |
| CN102764474B (en) | Auxiliary exercise training system and method | |
| CN115392287A (en) | Electroencephalogram signal online self-adaptive classification method based on self-supervision learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |