CN112617831B - Alzheimer's disease auxiliary treatment device based on virtual reality and brain-computer interface - Google Patents
Alzheimer's disease auxiliary treatment device based on virtual reality and brain-computer interface Download PDFInfo
- Publication number
- CN112617831B CN112617831B CN202010985826.3A CN202010985826A CN112617831B CN 112617831 B CN112617831 B CN 112617831B CN 202010985826 A CN202010985826 A CN 202010985826A CN 112617831 B CN112617831 B CN 112617831B
- Authority
- CN
- China
- Prior art keywords
- patient
- module
- signal
- brain
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 208000024827 Alzheimer disease Diseases 0.000 title claims abstract description 23
- 238000012545 processing Methods 0.000 claims abstract description 43
- 238000004458 analytical method Methods 0.000 claims abstract description 11
- 230000000694 effects Effects 0.000 claims abstract description 7
- 230000003213 activating effect Effects 0.000 claims abstract description 5
- 210000004958 brain cell Anatomy 0.000 claims abstract description 5
- 230000000638 stimulation Effects 0.000 claims abstract description 5
- 210000004556 brain Anatomy 0.000 claims description 31
- 238000000034 method Methods 0.000 claims description 20
- 238000013079 data visualisation Methods 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 15
- 238000007405 data analysis Methods 0.000 claims description 12
- 230000002996 emotional effect Effects 0.000 claims description 8
- 230000002093 peripheral effect Effects 0.000 claims description 8
- 230000000474 nursing effect Effects 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 claims description 3
- 230000006996 mental state Effects 0.000 claims description 3
- 230000036651 mood Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 238000002560 therapeutic procedure Methods 0.000 claims description 3
- 238000011360 adjunctive therapy Methods 0.000 claims 3
- 238000013461 design Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 208000000044 Amnesia Diseases 0.000 description 1
- 208000026139 Memory disease Diseases 0.000 description 1
- 206010039966 Senile dementia Diseases 0.000 description 1
- 208000003443 Unconsciousness Diseases 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 208000010877 cognitive disease Diseases 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000001976 improved effect Effects 0.000 description 1
- 230000006984 memory degeneration Effects 0.000 description 1
- 208000023060 memory loss Diseases 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000002040 relaxant effect Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 208000027765 speech disease Diseases 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
- 230000001225 therapeutic effect Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7253—Details of waveform analysis characterised by using transforms
- A61B5/7257—Details of waveform analysis characterised by using transforms using Fourier transforms
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
- A61M21/00—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
- A61M21/00—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
- A61M21/02—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis for inducing sleep or relaxation, e.g. by direct nerve stimulation, hypnosis, analgesia
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
- G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
- A61M21/00—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
- A61M2021/0005—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus
- A61M2021/0044—Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis by the use of a particular sense, or stimulus by the sight sense
Landscapes
- Health & Medical Sciences (AREA)
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Artificial Intelligence (AREA)
- Veterinary Medicine (AREA)
- Heart & Thoracic Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Medical Informatics (AREA)
- Biophysics (AREA)
- Psychiatry (AREA)
- Molecular Biology (AREA)
- Theoretical Computer Science (AREA)
- Surgery (AREA)
- Pathology (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Anesthesiology (AREA)
- General Engineering & Computer Science (AREA)
- Psychology (AREA)
- Signal Processing (AREA)
- Physiology (AREA)
- Acoustics & Sound (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Hematology (AREA)
- Developmental Disabilities (AREA)
- Hospice & Palliative Care (AREA)
- Social Psychology (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Educational Technology (AREA)
- Fuzzy Systems (AREA)
- Software Systems (AREA)
- Child & Adolescent Psychology (AREA)
- Pain & Pain Management (AREA)
Abstract
The invention provides an Alzheimer's disease auxiliary treatment device based on virtual reality and brain-computer interface, and relates to the technical field of medical auxiliary treatment. The device can help the treating personnel to correctly wear the external equipment for the patient by playing a course through the upper computer; and providing a plurality of scene contents for the treatment personnel to select; the treating personnel selects picture groups with different themes according to the memory condition of the patient; then displaying different types of picture groups and virtual scenes selected by the treating personnel; collecting an EEG signal when the patient is temporarily moved by a signal collecting device, processing and analyzing the EEG signal, marking a picture which is interested by the patient, and visually displaying an analysis result of data; finally, according to the marking result, pictures which are selected from the picture group and are interesting to the patient in the current treatment and pictures which are not interesting to the patient are obtained; in the second round of treatment, only pictures which are interesting for the patient are displayed, and the effect of activating the brain cells of the patient is achieved through repeated stimulation.
Description
Technical Field
The invention relates to the technical field of medical auxiliary treatment, in particular to an Alzheimer's disease auxiliary treatment device based on virtual reality and a brain-computer interface.
Background
Alzheimer's disease, also known as senile dementia, has the main symptoms of memory loss, cognitive dysfunction, speech disorder, etc. It has serious effects on the health and life of patients, the economic burden, the burden of families and caregivers, and the like. Since the direct cause of Alzheimer's disease is unknown, it has been a major difficulty in medical science. In recent years, with the aging of the chinese society, the number of patients with alzheimer's disease in china is rising to the peak due to the inverted pyramid structure of the population age distribution, which is really worried about. Therefore, in addition to the research on the disease in medical aspect, a scientific and technological approach should be found to design an auxiliary therapeutic device for auxiliary treatment.
Disclosure of Invention
The invention aims to solve the technical problem of providing an assistant treatment device for Alzheimer's disease based on virtual reality and brain-computer interface, aiming at the defects of the prior art, and realizing assistant treatment of Alzheimer's disease.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows: the Alzheimer disease auxiliary treatment device based on the virtual reality and brain-computer interface comprises an upper computer, external equipment, an external management module, a user operation management module, a virtual content display module, a data analysis module and an image playback module, wherein the external management module, the user operation management module, the virtual content display module, the data analysis module and the image playback module run on the upper computer;
the peripheral management module is used for connecting the upper computer and the external equipment, judging whether the external equipment is correctly connected or not and realizing the communication between the upper computer and the external equipment;
the external equipment comprises virtual reality equipment and EEG signal acquisition equipment; the virtual reality equipment adopts HTC Vive, and the EEG signal acquisition equipment adopts brain wave reading equipment MindWave;
the user operation management module comprises a course playing management submodule, a scene content selection submodule and an image content selection submodule;
the course playing management submodule assists the treating personnel to correctly wear the external equipment for the patient through playing the course; the scene content selection submodule provides a plurality of scene contents for the treatment personnel to select, and the treatment personnel select different treatment scenes for the treatment personnel according to the personal preference of the patient;
the picture content selection submodule comprises a plurality of groups of preset pictures, and a therapist selects picture groups with different themes for the patient according to the memory condition of the patient and formulates a next treatment plan according to data displayed by the data visualization module in the treatment process of the patient;
the virtual content display module displays different types of picture groups and virtual scenes selected by a therapist in the upper computer and the virtual reality equipment, and controls the movement of a patient in the virtual scenes and the opening and closing of the data processing module; when the picture selected by the treating personnel appears, the movement of the patient is paused for S seconds, and the data analysis module is started at the same time; when the patient continues to move, the data analysis module is closed;
the data analysis module comprises a signal acquisition sub-module, a signal processing sub-module and a data visualization sub-module;
the signal acquisition sub-module acquires EEG signals when the movement of the patient is suspended through EEG signal acquisition equipment and transmits the signals to the signal processing sub-module;
the signal processing submodule processes and analyzes data transmitted by external equipment, marks pictures in which a patient is interested, and simultaneously visually displays the analysis result of the data through the data visualization submodule, so that a therapist can know the emotional state of the patient in the nursing process;
the data visualization sub-module displays the analysis result of the data processing module on a screen interface of the upper computer, so that a therapist can visually see the real-time emotional feedback of the patient;
the picture playback module obtains pictures which are selected in the picture group and are interesting to the patient and pictures which are not interesting to the patient in the current treatment according to the marking result of the signal processing submodule; in the second round of treatment, according to the classification condition of the previous round, the pictures which are not interested by the patient are removed, only the pictures which are interested by the patient are displayed, and the effect of activating the brain cells of the patient is achieved through repeated stimulation.
Preferably, the tutorial assists the treating person in correctly wearing the external device for the patient through the tutorial, and the tutorial is placed in the virtual environment; after determining that the external device has been worn correctly, the tutorial will issue a prompt to keep the patient in a relaxed mental state; the tutorial is set as a 2D cartoon character.
Preferably, the specific method for the data processing module to process and analyze the data transmitted from the external device is as follows:
s1, firstly, processing an EEG signal transmitted by external equipment;
firstly, carrying out fast Fourier transform on a received EEG signal, decomposing a complex signal into small Delta, Theta, Alpha and Beta wave signals, filtering out brain wave signals with specified frequency by setting different passband frequencies, and reconstructing the brain wave signals through the Delta, Theta, Alpha and Beta wave signals after frequency division, wherein the following formula is shown as follows:
F(t)＝A1·sin(α)+A2·sin(β)+A3·sin(θ)+A4·sin(Δ)
wherein F (t) is the reconstructed EEG signal, Delta, Theta, Alpha and Beta represent Delta, Theta, Alpha and Beta wave signals respectively, A1、A2、A3、A4Respectively represent Delta, Theta, Alpha and Beta wave signal amplitudes;
multiplying the reconstructed electroencephalogram signal by a discrete sine signal to obtain a finally processed discrete electroencephalogram signal, wherein the formula is as follows:
Fs(t)＝G(Tn)×F(t)
wherein, Fs(T) is the final processed brain electrical signal, G (T)n) For discrete sinusoidal signals, A stands for continuousSinusoidal signal waveform, Δ (t-nT)s) Representing the pulse function, every TsGenerating a pulse signal in time, wherein n is the sampling times;
s2, analyzing the processed discretized electroencephalogram signals, and judging whether the wave band signals express that the patient is interested or not interested;
the processed brain electrical signal is subjected to binarization processing by taking 50 as a threshold value, wherein 50 represents the brain electrical signal value when the mood of the patient is calm, and the brain electrical signal is converted into a numerical code of 01, and the following formula is shown as follows:
wherein, InIs the value of the brain electrical signal sampled for the nth time after binarization processing;
then, the brain electrical signals in the binary form are input into a BP network, and whether the brain electrical signals in the wave band express that the patient is interested or not is judged.
Preferably, the BP network comprises an input layer, a hidden layer 1, a hidden layer 2, and an output layer; the input layer has four binary inputs and the weight array size is 4 x 5, followed by a first hidden layer 1 containing four nodes and a hidden layer 2 containing two nodes, and the final output layer contains only one node.
The design of the Alzheimer's disease auxiliary treatment device based on the virtual reality and the brain-computer interface mainly comprises three factors:
(1) and inputting a signal. These signals represent the patient's perception of the image as it is viewed, received by using the Mind Wave equipment, extracted and filtered in a program and decoded, processed to classify the positive and negative signals and passed to the analysis section of the system. The thought state information is then transmitted to project engineering files for analysis.
(2) And (3) an algorithm for promoting the classification precision of the brain wave signals. The first design step is to use mathematical techniques to roughly classify the EEG signal based on its classification. In the design of the second step, a deep learning algorithm is adopted to analyze signals, and a large number of EEG signals are used for training a deep learning algorithm model in the early stage to realize the classification of the EEG signals. After the system receives the classified signals, whether the images are repeated or not is determined according to the types of the signals.
(3) And designing a virtual reality environment suitable for treating the patients with the Alzheimer's disease. The environment needs to bring relaxation feeling to the patient, needs to research a proper human-computer interaction mode, ensures that the environment can enable the patient to be in an unconscious state, keeps a relaxed state of mind, and is not mixed with other factors which may influence the emotion of the patient. Through the construction of the virtual reality scene, the treatment environment can be ensured to be controllable, and the patient can not be influenced by external factors.
Adopt the produced beneficial effect of above-mentioned technical scheme to lie in: according to the Alzheimer's disease auxiliary treatment device based on the virtual reality and the brain-computer interface, the virtual reality and brain-computer interface (BCI) technology is combined, the input mode of a traditional virtual reality system can be changed through the introduction of the BCI, a user can interact and control with the virtual reality system through an EEG signal, and the operability is improved. Meanwhile, a feedback module of the BCI technology is innovated by utilizing virtual reality, so that the BCI system is not a simple two-dimensional feedback mode any more, but becomes a more active and colorful situation application for a user. The system combining virtual reality and BCI can bring a real, reliable and safe training scene for a user, greatly reduce the nursing cost for families and hospitals of patients with Alzheimer's disease, reduce the social pressure, and simultaneously improve the life quality of the patients and the families.
Drawings
Fig. 1 is a block diagram of an alzheimer's disease auxiliary treatment device based on virtual reality and brain-computer interface according to an embodiment of the present invention;
fig. 2 is a schematic process diagram of a data processing module processing and analyzing data transmitted from an external device according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating a data processing module processing and analyzing data transmitted from an external device according to an embodiment of the present invention;
fig. 4 is a flowchart of an assisted therapy performed by the alzheimer's disease assisted therapy apparatus based on virtual reality and brain-computer interface according to the embodiment of the present invention.
Detailed Description
The following detailed description of embodiments of the present invention is provided in connection with the accompanying drawings and examples. The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention.
In this embodiment, the alzheimer's disease auxiliary treatment device based on virtual reality and brain-computer interface, as shown in fig. 1, includes an upper computer, an external device, and a peripheral management module, a user operation management module, a virtual content display module, a data analysis module, and an image playback module running on the upper computer;
the peripheral management module is used for connecting the upper computer and the external equipment, judging whether the external equipment is correctly connected or not and realizing the communication between the upper computer and the external equipment;
the external equipment comprises virtual reality equipment and EEG signal acquisition equipment; the virtual reality equipment adopts HTC Vive, and the EEG signal acquisition equipment adopts brain wave reading equipment MindWave;
according to the requirements of the device on virtual reality technology and EEG signal receiving, the HTC Vive is adopted as virtual reality equipment, and the MindWave is adopted as EEG signal acquisition equipment; the peripheral management module is mainly responsible for detecting whether the external equipment is correctly connected with the device. The MindWave equipment needs to manually input an equipment port number and needs to be correctly input by a user, and if an input error or a connection fault occurs, the peripheral management module can give a prompt on an interface to allow the user to input again or check whether the equipment is correctly worn;
the user operation module provides a proper man-machine interaction mode for a user, so that the user can conveniently and correctly use the device, and specifically comprises a course playing management sub-module, a scene content selection sub-module and a picture content selection sub-module;
the course playing management submodule assists the treating personnel to correctly wear the external equipment for the patient through playing the course; the tutorial assists the treating personnel to correctly wear the external equipment for the patient through the tutorial; meanwhile, in order to relieve the tension of the patient entering the virtual reality system for the first time and avoid influencing the subsequent signal detection, the course is placed in a relaxing virtual environment; after determining that the external device has been worn correctly, the tutorial will issue a prompt to keep the patient in a relaxed mental state; since the effect of the 3D character as a guide in the scene is less than ideal, the tutorial is set as a 2D cartoon character;
in order to avoid the patient from generating boring feeling due to single scene of the nursing process, the scene content selection submodule provides a plurality of scene contents for the treatment personnel to select, and the treatment personnel select different treatment scenes for the patient according to personal preference of the patient;
the picture content selection submodule comprises a plurality of groups of preset pictures, and a therapist selects picture groups with different themes for the patient according to the memory condition of the patient and formulates a next treatment plan according to data displayed by the data visualization module in the treatment process of the patient;
the virtual content display module displays different types of picture groups and virtual scenes selected by a therapist in the upper computer and the virtual reality equipment, and controls the movement of a patient in the virtual scenes and the opening and closing of the signal processing sub-modules; after the treatment plan of the patient is selected by the treating person, the patient can roam in the virtual reality scene selected by the treating person and see the successively selected pictures appearing in front of the eyes. When the picture selected by the treating personnel appears, the movement of the patient is paused for S seconds, and the data analysis module is started at the same time; when the patient continues to move, the data analysis module is closed; such processing facilitates more accurate collection of EEG signals, ensures that the input EEG signals are required for treatment, enables more accurate processing and analysis of subsequent EEG signals by avoiding interference of extraneous signals, and provides accurate feedback to the treating person.
The data analysis module comprises a signal acquisition sub-module, a signal processing sub-module and a data visualization sub-module;
the signal acquisition sub-module acquires EEG signals when the movement of the patient is suspended through EEG signal acquisition equipment and transmits the signals to the signal processing sub-module;
the signal processing submodule processes and analyzes data transmitted by external equipment, marks pictures in which a patient is interested, and simultaneously visually displays the analysis result of the data through the data visualization module, so that a therapist can know the emotional state of the patient in the nursing process;
the data visualization submodule displays the analysis result of the signal processing submodule on a screen interface of an upper computer, so that a therapist can visually see real-time emotional feedback of a patient;
the picture playback module is used for obtaining pictures which are selected in the picture group and are interesting to the patient and pictures which are not interesting to the patient in the current treatment according to the mark classification of the signal processing submodule; in the second round of treatment, according to the classification condition of the previous round, the pictures which are not interested by the patient are removed, only the pictures which are interested by the patient are displayed, and the effect of activating the brain cells of the patient is achieved through repeated stimulation.
The signal processing sub-module processes and analyzes data transmitted by an external device as shown in fig. 2 and 3, and the specific method comprises the following steps:
s1, firstly, processing an EEG signal transmitted by external equipment;
firstly, carrying out fast Fourier transform on a received EEG signal, decomposing a complex signal into small Delta, Theta, Alpha and Beta wave signals, filtering out brain wave signals with specified frequency by setting different passband frequencies, and reconstructing the brain wave signals through the Delta, Theta, Alpha and Beta wave signals after frequency division, wherein the following formula is shown as follows:
F(t)＝A1·sin(α)+A2·sin(β)+A3·sin(θ)+A4·sin(Δ)
wherein F (t) is the reconstructed EEG signal, Delta, Theta, Alpha and Beta represent Delta, Theta, Alpha and Beta wave signals respectively, A1、A2、A3、A4Respectively represent Delta, Theta, Alpha andbeta wave signal amplitude;
multiplying the reconstructed electroencephalogram signal by a discrete sine signal to obtain a finally processed discrete electroencephalogram signal, wherein the formula is as follows:
Fs(t)＝G(Tn)×F(t)
wherein, Fs(T) is the final processed brain electrical signal, G (T)n) For a discrete sinusoidal signal, A represents a continuous sinusoidal signal waveform, Δ (t-nT)s) Representing the pulse function, every TsGenerating a pulse signal in time, wherein n is the sampling times;
s2, analyzing the processed discretized electroencephalogram signals, and judging whether the wave band signals express that the patient is interested or not interested;
the processed brain electrical signal is subjected to binarization processing by taking 50 as a threshold value, wherein 50 represents the brain electrical signal value when the mood of the patient is calm, and the brain electrical signal is converted into 01 numerical code, and the following formula is shown as follows:
wherein, InIs the value of the brain electrical signal sampled for the nth time after binarization processing;
then, inputting the brain wave signals in the binary form into a BP network, and judging whether the brain wave signals in the wave band express that the patient is interested or not interested;
the BP network comprises an input layer, a hidden layer 1, a hidden layer 2 and an output layer; the input layer has four binary inputs and the weight array size is 4 x 5, followed by a first hidden layer 1 containing four nodes and a hidden layer 2 containing two nodes, and the final output layer contains only one node.
In this embodiment, the method of using the alzheimer's disease auxiliary treatment device based on the virtual reality and the brain-computer interface is as shown in fig. 4, and firstly, a doctor who is a treatment staff wears external equipment on a patient according to a course; then, the treating personnel selects a treating scene for the patient through the scene content selection submodule according to the personal preference of the patient, and selects picture groups with different themes for the patient through the picture content selection submodule according to the memory condition of the patient; and simultaneously, the virtual content display module displays different types of picture groups and virtual scenes selected by the treating personnel, and after the treating personnel selects the treatment scheme of the patient, the patient can roam in the virtual reality scene selected by the treating personnel and see the successively selected pictures appearing in front of the patient. The virtual content display module controls the movement of the patient in the virtual scene and the opening and closing of the signal processing sub-module at the same time; when the picture selected by the treating personnel appears, the movement of the patient is paused for S seconds, and the signal acquisition submodule is started at the same time; when the patient continues to move, the signal acquisition submodule is closed; the signal acquisition sub-module acquires EEG signals when the movement of the patient is suspended through EEG signal acquisition equipment and transmits the signals to the signal processing sub-module; the signal processing submodule processes and analyzes data transmitted by external equipment, marks pictures in which a patient is interested, and simultaneously visually displays the analysis result of the data through the data visualization submodule, so that a therapist can visually see real-time emotional feedback of the patient; thereby enabling the therapist to know the emotional state of the patient in the nursing process; in the second round of treatment, according to the classification condition of the previous round, the picture content which is not interesting to the patient is removed, only the picture which is interesting to the patient is displayed, and the effect of activating the brain cells of the patient is achieved through repeated stimulation.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions and scope of the present invention as defined in the appended claims.
Claims (4)
1. The utility model provides an alzheimer's disease adjunctie therapy device based on virtual reality and brain-machine interface which characterized in that: the system comprises an upper computer, external equipment, a peripheral management module, a user operation management module, a virtual content display module, a data acquisition module, a data processing module, a data visualization module and a picture playback module, wherein the peripheral management module, the user operation management module, the virtual content display module, the data acquisition module, the data processing module, the data visualization module and the picture playback module run on the upper computer;
the peripheral management module is used for connecting the upper computer and the external equipment, judging whether the external equipment is correctly connected or not and realizing the communication between the upper computer and the external equipment;
the external equipment comprises virtual reality equipment and EEG signal acquisition equipment; the virtual reality equipment adopts HTC Vive, and the EEG signal acquisition equipment adopts brain wave reading equipment MindWave;
the user operation management module comprises a course playing management submodule, a scene content selection submodule and an image content selection submodule;
the course playing management submodule assists the treating personnel to correctly wear the external equipment for the patient through playing the course; after determining that the external device has been worn correctly, the tutorial will issue a prompt to keep the patient in a relaxed mental state;
the scene content selection submodule provides a plurality of scene contents for the treatment personnel to select, and the treatment personnel select different treatment scenes for the treatment personnel according to the personal preference of the patient;
the picture content selection submodule comprises a plurality of groups of preset pictures, and a therapist selects picture groups with different themes for the patient according to the memory condition of the patient and formulates a next treatment plan according to data displayed by the data visualization module in the treatment process of the patient;
the virtual content display module displays different types of picture groups and virtual scenes selected by a therapist in the upper computer and the virtual reality equipment, and controls the movement of a patient in the virtual scenes and the opening and closing of the data processing module; when the picture selected by the treating personnel appears, the movement of the patient is paused for S seconds, and the data analysis module is started at the same time; when the patient continues to move, the data analysis module is closed;
the data analysis module comprises a signal acquisition sub-module, a signal processing sub-module and a data visualization sub-module;
the signal acquisition sub-module acquires EEG signals when the movement of the patient is suspended through EEG signal acquisition equipment and transmits the signals to the signal processing sub-module;
the signal processing submodule processes and analyzes data transmitted by external equipment, marks pictures in which a patient is interested, and simultaneously visually displays the analysis result of the data through the data visualization submodule, so that a therapist can know the emotional state of the patient in the nursing process;
the data visualization sub-module displays the analysis result of the data processing module on a screen interface of the upper computer, so that a therapist can visually see the real-time emotional feedback of the patient;
the picture playback module obtains pictures which are selected in the picture group and are interesting to the patient and pictures which are not interesting to the patient in the current treatment according to the marking result of the signal processing submodule; in the second round of treatment, according to the classification condition of the previous round, the pictures which are not interested by the patient are removed, only the pictures which are interested by the patient are displayed, and the effect of activating the brain cells of the patient is achieved through repeated stimulation.
2. The Alzheimer's disease adjunctive therapy device based on virtual reality and brain-computer interface of claim 1, characterized in that: the course guides the staff to assist the treating staff to correctly wear the external equipment for the patient through the course, and the course is placed in a virtual environment; the tutorial is set as a 2D cartoon character.
3. The Alzheimer's disease adjunctive therapy device based on virtual reality and brain-computer interface of claim 1, characterized in that: the specific method for processing and analyzing the data transmitted by the external equipment by the data processing module is as follows:
s1, firstly, processing an EEG signal transmitted by external equipment;
firstly, carrying out fast Fourier transform on a received EEG signal, decomposing a complex signal into small Delta, Theta, Alpha and Beta wave signals, filtering out brain wave signals with specified frequency by setting different passband frequencies, and reconstructing the brain wave signals through the Delta, Theta, Alpha and Beta wave signals after frequency division, wherein the following formula is shown as follows:
F(t)＝A1·sin(α)+A2·sin(β)+A3·sin(θ)+A4·sin(Δ)
wherein F (t) is the reconstructed EEG signal, Delta, Theta, Alpha and Beta represent Delta, Theta, Alpha and Beta wave signals respectively, A1、A2、A3、A4Respectively represent Delta, Theta, Alpha and Beta wave signal amplitudes;
multiplying the reconstructed electroencephalogram signal by a discrete sine signal to obtain a finally processed discrete electroencephalogram signal, wherein the formula is as follows:
Fs(t)＝G(Tn)×F(t)
wherein, Fs(T) is the final processed brain electrical signal, G (T)n) For a discrete sinusoidal signal, A represents a continuous sinusoidal signal waveform, Δ (t-nT)s) Representing the pulse function, every TsGenerating a pulse signal in time, wherein n is the sampling times;
s2, analyzing the processed discretized electroencephalogram signals, and judging whether the wave band signals express that the patient is interested or not interested;
the processed brain electrical signal is subjected to binarization processing by taking 50 as a threshold value, wherein 50 represents the brain electrical signal value when the mood of the patient is calm, and the brain electrical signal is converted into a numerical code of 01, and the following formula is shown as follows:
wherein, InIs the value of the brain electrical signal sampled for the nth time after binarization processing;
then, the brain electrical signals in the binary form are input into a BP network, and whether the brain electrical signals in the wave band express that the patient is interested or not is judged.
4. The Alzheimer's disease adjunctive therapy device based on virtual reality and brain-computer interface of claim 3, characterized in that: the BP network comprises an input layer, a hidden layer 1, a hidden layer 2 and an output layer; the input layer has four binary inputs and the weight array size is 4 x 5, followed by a first hidden layer 1 containing four nodes and a hidden layer 2 containing two nodes, and the final output layer contains only one node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010985826.3A CN112617831B (en) | 2020-09-18 | 2020-09-18 | Alzheimer's disease auxiliary treatment device based on virtual reality and brain-computer interface |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010985826.3A CN112617831B (en) | 2020-09-18 | 2020-09-18 | Alzheimer's disease auxiliary treatment device based on virtual reality and brain-computer interface |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112617831A CN112617831A (en) | 2021-04-09 |
| CN112617831B true CN112617831B (en) | 2021-08-31 |
Family
ID=75300172
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010985826.3A Active CN112617831B (en) | 2020-09-18 | 2020-09-18 | Alzheimer's disease auxiliary treatment device based on virtual reality and brain-computer interface |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112617831B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012183215A (en) * | 2011-03-07 | 2012-09-27 | Seiko Epson Corp | Analysis system, electroencephalogram analysis apparatus and program |
| JP2013121119A (en) * | 2011-12-08 | 2013-06-17 | Renesas Electronics Corp | Ad converter and solid-state image pickup device using the same |
| KR20190041081A (en) * | 2017-10-12 | 2019-04-22 | 주식회사 라스테크 | Evaluation system of cognitive ability based on virtual reality for diagnosis of cognitive impairment |
| CN109875509A (en) * | 2019-02-27 | 2019-06-14 | 京东方科技集团股份有限公司 | The test macro and method of Alzheimer Disease patient rehabilitation training effect |
| CN110292514A (en) * | 2019-06-03 | 2019-10-01 | 东莞佰和生物科技有限公司 | The method for serving the intelligent robot rehabilitation training of old dementia patients |
| CN110313895A (en) * | 2019-04-28 | 2019-10-11 | 江南大学 | Training of cognitive function method |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040230549A1 (en) * | 2003-02-03 | 2004-11-18 | Unique Logic And Technology, Inc. | Systems and methods for behavioral modification and behavioral task training integrated with biofeedback and cognitive skills training |
| US20160196765A1 (en) * | 2014-12-24 | 2016-07-07 | NeuroSpire, Inc. | System and method for attention training using electroencephalography (EEG) based neurofeedback and motion-based feedback |
| IL310060A (en) * | 2016-05-09 | 2024-03-01 | Magic Leap Inc | Augmented reality systems and methods for user health analysis |
| US20170352283A1 (en) * | 2016-06-07 | 2017-12-07 | Cerekinetic, Inc. | Self-administered evaluation and training method to improve mental state |
-
2020
- 2020-09-18 CN CN202010985826.3A patent/CN112617831B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012183215A (en) * | 2011-03-07 | 2012-09-27 | Seiko Epson Corp | Analysis system, electroencephalogram analysis apparatus and program |
| JP2013121119A (en) * | 2011-12-08 | 2013-06-17 | Renesas Electronics Corp | Ad converter and solid-state image pickup device using the same |
| KR20190041081A (en) * | 2017-10-12 | 2019-04-22 | 주식회사 라스테크 | Evaluation system of cognitive ability based on virtual reality for diagnosis of cognitive impairment |
| CN109875509A (en) * | 2019-02-27 | 2019-06-14 | 京东方科技集团股份有限公司 | The test macro and method of Alzheimer Disease patient rehabilitation training effect |
| CN110313895A (en) * | 2019-04-28 | 2019-10-11 | 江南大学 | Training of cognitive function method |
| CN110292514A (en) * | 2019-06-03 | 2019-10-01 | 东莞佰和生物科技有限公司 | The method for serving the intelligent robot rehabilitation training of old dementia patients |
Non-Patent Citations (1)
| Title |
|---|
| 虚拟现实技术在老年痴呆患者认知功能训练中的研究进展;陈妞等;《护理学杂志》;20170228;第32卷(第3期);第106-109页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112617831A (en) | 2021-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Citi et al. | P300-based BCI mouse with genetically-optimized analogue control | |
| Wang et al. | The visual magnocellular pathway in Chinese-speaking children with developmental dyslexia | |
| CN103699226A (en) | Tri-modal serial brain-computer interface method based on multi-information fusion | |
| CN1555243A (en) | System and method for vision examination | |
| CN110141258A (en) | A kind of emotional state detection method, equipment and terminal | |
| CN112328072A (en) | Multi-mode character input system and method based on electroencephalogram and electrooculogram | |
| CN110737331B (en) | Individualized cognitive training and rehabilitation method, device and equipment based on multi-task brain-computer interface | |
| Li et al. | Multi-modal emotion recognition based on deep learning of EEG and audio signals | |
| CN113974627B (en) | Emotion recognition method based on brain-computer generated confrontation | |
| Abibullaev et al. | Deep learning in EEG-based BCIs: a comprehensive review of transformer models, advantages, challenges, and applications | |
| Parmar et al. | A novel and efficient Wavelet Scattering Transform approach for primitive-stage dyslexia-detection using electroencephalogram signals | |
| Perakakis et al. | Affective evaluation of a mobile multimodal dialogue system using brain signals | |
| Hernández et al. | The contribution of event-related potentials to the understanding of sign language processing and production in the brain: Experimental evidence and future directions | |
| CN112617831B (en) | Alzheimer's disease auxiliary treatment device based on virtual reality and brain-computer interface | |
| Aldhaheri et al. | Brain–Computer Interfaces and Neurolinguistics: A Short Review | |
| Razi et al. | Detecting learning disabilities based on neuro-physiological interface of affect (NPIoA) | |
| CN103300849A (en) | Electroencephalogram signal processing method | |
| Ramakuri et al. | Performance analysis of eye-state charecterization through single electrode EEG device for medical application | |
| Xing et al. | The development of EEG-based brain computer interfaces: potential and challenges | |
| Wang et al. | Temporal-spatial representation learning transformer for EEG-based emotion recognition | |
| Zhao et al. | Research on steady state visual evoked potential based on FBCCA | |
| Hu et al. | TDD: Auxiliary framework for recognizing people with depression based on physiological and emotional characteristics of EEG signals | |
| AU2021102564A4 (en) | HCT based Near infrared Brain Function Imager 3D Analysis System | |
| Anh et al. | Application of portable electroencephalograph device in controlling and identifying emotion | |
| Afsharyan et al. | A Hardware in Loop Simulation Robot Control by Weareable Electroencephalography (EEG)-Based Brain Computer Interface (BCI) |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230925 Address after: No. 8, 6th Floor, 1st Floor, Boguangyuan, High tech Industrial Park, Dalian, Liaoning Province, 116086 Patentee after: Dalian Zeyuan Technology Co.,Ltd. Address before: 110819 No. 3 lane, Heping Road, Heping District, Shenyang, Liaoning 11 Patentee before: Northeastern University |
|
| TR01 | Transfer of patent right |