CN115227264A

CN115227264A - Method for displaying brain wave signals by using Cranib graph

Info

Publication number: CN115227264A
Application number: CN202210833637.3A
Authority: CN
Inventors: 高军晖; 李伟明; 谭润东; 何熲
Original assignee: Shanghai Nuanhenao Science And Technology Co ltd
Current assignee: Shanghai Nuanhenao Science And Technology Co ltd
Priority date: 2022-07-15
Filing date: 2022-07-15
Publication date: 2022-10-25

Abstract

The invention provides a method for displaying brain wave signals by using a Claanib graph, which comprises the following steps: respectively generating corresponding Clarnian graphs according to different waveform oscillation frequencies; acquiring and processing an electroencephalogram signal to obtain a neural feedback index; mapping the nerve feedback index to the waveform oscillation frequency of the Clarnian graph to present a corresponding Clarnian graph; keeping the Claani pattern synchronized with the brain electrical signal or differing by a fixed delay. The method for displaying the brain wave signals by using the Clarniy graph utilizes the visual stimulation of the Clarniy graph, can intuitively and simply represent the energy of the brain waves of the subject in different frequency bands and corresponding nerve feedback indexes, has immersion feeling, cannot disperse the attention of the subject and influence the current brain state, and therefore influences the effect of the method for displaying the brain wave signals by using the Clarniy graph.

Description

Method for displaying brain wave signals by using Cranib graph

Technical Field

The invention belongs to the fields of brain-computer interfaces, signal visualization, nerve feedback and the like, and particularly relates to a method for displaying brain wave signals by using a Cranib graph.

Background

Scalp Electroencephalography (EEG) is a non-invasive electrophysiological monitoring method for recording brain electrical signals by placing electrodes on the surface of a person's scalp and measuring the sum of the weights of postsynaptic potentials at the electrode placement points when a large number of pyramidal cells are excited.

At present, brain-computer interface systems based on electroencephalogram are divided into two categories, namely an exogenous brain-computer interface and an endogenous brain-computer interface.

In the external brain-computer interface, it is necessary to sequentially and continuously present the external device with a specific visual or auditory stimulus to the subject, generate corresponding brain responses such as event-related potential P300 and steady-state visual evoked potential (SSVEP), etc., and control the external device after identification. This is a discrete pattern with a low information transfer rate. This discrete mode is inconvenient for brain-computer interface applications that require fast and real-time.

The endogenous brain-computer interface does not need any external stimulation, and the subject adjusts the state of the brain according to the intention of the subject, such as motor imagery and attention intensity, and controls external equipment. The spontaneous mode does not need to be induced by external visual stimulation, so that the mode is continuous and the information transmission rate is high. However, for example, motor imagery and attention intensity control have different control effects of different subjects, and have large individual difference for endogenous electroencephalograms, so that a large amount of data training algorithm models in the early stage are required to identify common electroencephalograms of different people for model training, and the effectiveness is inconsistent.

For a closed-loop brain-computer interface system, i.e., generating graphics based on characteristics of electroencephalograms, on one hand, relatively simple graphics such as amplitude diagrams, frequency spectrum diagrams, bar charts based on brain wave energy, hue and scale (Wood and Kober, 2018), etc., which lack aesthetics, such that the above conventional methods lack immersion, easily distract the subject, and affect the current brain state, thereby affecting the brain-computer interface effect; another aspect may be relatively complex graphics such as a gyroscopic image or the like. Grechko and Gontar proposed in 2009 to convert brain electrical signals into simulated time series using a discrete chaotic dynamics (discrete chaotic dynamics) mathematical model, and then into a gyroscopic image. According to the curative theory of the psychological phenomenon of the mannuro image, the visual stimulation of the mannuro image can promote the training effect of the nerve feedback. However, the study only uses the time sequence characteristics of the brain electrical signals to generate pictures, which represent the mental state of the current subject, and ignores the abundant frequency spectrum characteristics of the brain electrical signals, which can better represent the cognitive and emotional states of the subject. Thus, current closed-loop brain-computer interface techniques reflect the state of the brain less comprehensively or accurately.

In the eighteenth century, the german physicist entster clandeni made an experiment in which a wide piece of foil was placed on a violin and sand was evenly spread over it. Then, the fiddle is drawn by the bow, so that the fine sands are automatically arranged in different beautiful patterns, and the patterns are continuously changed and become more complicated with the difference of the melody drawn by the strings and the increase of the frequency, which is the craini pattern.

Therefore, based on the prior art, a closed-loop brain-computer interface system can be designed to reflect the brain state more comprehensively and accurately.

Disclosure of Invention

The invention aims to provide a method for displaying brain wave signals by using a Cranib graph so as to reflect brain states more comprehensively and accurately.

In order to achieve the above object, the present invention provides a method for displaying brain wave signals using a kraani graph, comprising:

s1: respectively generating corresponding Clarnian graphs according to different waveform oscillation frequencies;

s2: acquiring and processing an electroencephalogram signal to obtain a neural feedback index;

s3: mapping the nerve feedback index to the waveform oscillation frequency of the Clarnian graph to present a corresponding Clarnian graph;

s4: the Clarnib pattern is kept synchronous with the brain electrical signal or differs from the brain electrical signal by a fixed delay by executing steps S1-S3 continuously in real time.

In the step S1, a cratanib graph corresponding to different waveform oscillation frequencies is generated by using matplotlib visualization.

Utilize matplotlib visualization to generate the clarnia graph that different waveform oscillation frequencies correspond to, specifically include: digitally simulating the Kranit board, and setting the natural frequency of the Kranit board; and simulating sound, and obtaining Clarni graphs corresponding to different waveform oscillation frequencies respectively by adjusting sound frequency parameters.

The step S2 includes:

s21: selecting a recording electrode channel, placing a reference electrode and a grounding electrode on an ear, and starting to acquire electroencephalogram signals;

s22: and carrying out on-line preprocessing on the electroencephalogram signal to obtain a nerve feedback index of the electroencephalogram signal.

The step S22 includes:

s221: filtering with a finite impulse response filter with 0.5Hz high pass and 45Hz low pass;

s222: calculating power spectral density through Welch;

s223: and calculating the average power in each brain wave frequency range according to each brain wave frequency range, and determining the neural feedback index.

In step S222, every 0.05 second, the last 4 seconds of electroencephalogram signals are selected, and power spectral density is calculated by Welch; in step S223, the range of each brain wave frequency includes delta, theta, low alpha, high alpha, low beta, high beta, and gamma.

In the step S2, a single-channel electroencephalogram acquisition device is used to acquire and process electroencephalogram signals, so as to obtain a neural feedback index.

In the step S2, multi-channel electroencephalogram equipment is adopted to collect and process electroencephalogram signals to obtain a neural feedback index; for multi-channel electroencephalogram acquisition equipment, the neural feedback indexes are respectively calculated in each recording electrode channel, and then the average value of the neural feedback indexes of different recording electrode channels is taken as the final neural feedback index.

The neurofeedback index is an attention index, a relaxation index, a stress index, or an impulse control index.

In said step S3, the corresponding kranib graphic is selected for presentation with pygame.

The brain-computer interface system based on the closed loop is used for real-time graphic display to further influence the brain reaction of a subject by extracting the electroencephalogram characteristics, and the attention index is measured by adopting the ratio of different brain wave energies, so that a large amount of training at the early stage is not needed; meanwhile, the current brain state of the subject is reflected in real time through the signal characteristics of the continuous visual stimulation, and the induction of the visual stimulation is not required to be provided from the outside, so that the continuous mode is realized, the information transmission rate is high, and the real-time performance is good. Therefore, the advantages of the two brain-computer interfaces are combined, the current brain state of the subject can be intuitively reflected, the brain state is influenced and changed, the purpose of rapidness and real-time is realized, and the disadvantages of the two brain-computer interfaces are avoided.

The method for displaying the brain wave signals by using the Clarniy graph utilizes the visual stimulation of the Clarniy graph, can intuitively and simply represent the energy of the brain waves of the subject in different frequency bands and corresponding nerve feedback indexes, has immersion feeling, cannot disperse the attention of the subject and influence the current brain state, and therefore influences the effect of the method for displaying the brain wave signals by using the Clarniy graph.

Due to the plasticity mechanism of the brain, the method for displaying the brain wave signals by the Clarnian graph can train the testee to improve various nerve feedback indexes (such as attention) and enhance the brain self-regulation capability.

Drawings

FIG. 1 is a flow chart of a method of displaying brain wave signals using a Cranib graph according to the present invention.

Detailed Description

The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

As shown in fig. 1, the method for displaying brain wave signals by using a kraani graph of the present invention comprises the following steps:

step S1: respectively generating corresponding Clarnian graphs according to different waveform oscillation frequencies;

all objects, including the Claanib plate, have a set natural frequency of vibration. A system, such as a standing wave in a musical instrument (a standing wave is a superposition of two rows of waves with opposite propagation directions and the same amplitude and frequency). After the structure vibrates with a certain waveform oscillation frequency, the structure can be deformed into a corresponding shape: a characteristic mode.

In the present embodiment, matplotlib visualization is used to generate clarnia graphs corresponding to different waveform oscillation frequencies.

The generating of the Claani graph by utilizing matplotlib visualization specifically comprises the following steps: digitally simulating the Claanib board, and setting the natural frequency of the Claanib board; and simulating sound, and obtaining Clarni graphs corresponding to different waveform oscillation frequencies respectively by adjusting sound frequency parameters.

The value of the natural frequency of the Claanib board can be defined by itself, and has no specific requirement. The kranib phenomenon is that the frequency of sound (i.e., the waveform oscillation frequency) affects the kranib sheet, so that the sand on the kranib sheet presents a certain pattern, and therefore, in the process of simulating sound, only the frequency of sound is changed, and different types of kranib patterns can be generated. The clarniy patterns corresponding to the different waveform oscillation frequencies have different complexities, and the clarniy patterns change complicatedly along with the increase of the waveform oscillation frequency, namely the complexities are higher.

According to the literature [ para jiangsong, approximate solution of lateral vibration of a four-sided free rectangular plate and experimental studies thereof, 2016, the tenth academic conference on dynamics and controls ], the calculation of the natural frequency of the kraani plate comprises the following steps:

a1: setting a control differential equation of the rectangular plate according to a thin plate vibration theory;

a2: and solving an approximate solution of the vibration mode function of the four-side free rectangular plate and the corresponding natural frequency by adopting a Claanitan pattern experimental method, and obtaining a two-dimensional standing wave theoretical figure of the four-side free rectangular plate according to an expression of the approximate solution.

Among them, the method of the Clarithroman test is described in the literature [ ROSSING T D.Chladni's law for visualization plants [ J ]. American Journal of Physics,1982, 50 (3): 271-274). By using a differential equation, an expression of the mode shape function at each order of the natural frequency can be obtained.

A3: and obtaining a two-dimensional standing wave experimental pattern of the rectangular plate by using a Clarnian speckle pattern experimental method.

By approximate solution expression, we can obtain the two-dimensional standing wave theoretical figure of the four-side free rectangular plate. By comparing the experimental graph with the theoretical graph, the conclusion is reached: the approximate solution obtained in step A2 is substantially identical to the experimental solution.

Step S2: acquiring and processing an electroencephalogram signal to obtain a neural feedback index;

the step S2 includes:

step S21: selecting a recording electrode channel, placing a reference electrode and a grounding electrode on the ear, and starting to acquire electroencephalogram signals;

in the present embodiment, the recording electrode channel is selected to be FPz, and in addition to FPz, channels located in the frontal lobe region of the brain, such as Fp1, fp2, fz, F3, F4, F7, and F8, may be selected as the recording electrode channel.

When electroencephalogram signals are collected, the impedance of a recording electrode of electroencephalogram collecting equipment is controlled to be below 10k omega (the contact area of the electrode and the scalp is increased by adjusting the angle of the electrode, so that the impedance is controlled to be below 10k omega), the sampling frequency is 512Hz, and the collected electroencephalogram signals are transmitted through Bluetooth.

During the acquisition of the electroencephalogram signals, the subject is required to remain as still as possible, avoiding excessive blinking and head movement.

Step S22: and carrying out on-line preprocessing on the electroencephalogram signal to obtain a nerve feedback index of the electroencephalogram signal.

Step S22 includes:

step S221: filtering is performed using a Finite Impulse Response (FIR) filter with a 0.5Hz high pass and a 45Hz low pass to remove low frequency drift and unwanted frequency bands, removing low frequency artifacts of eye movement and high frequency noise of the muscles.

Step S222: calculating power spectral density through Welch;

in the embodiment, every 0.05 second, the last 4 seconds of electroencephalogram signals are selected, and the power spectral density is calculated through Welch, wherein the frequency resolution is 0.25Hz.

Step S223: and calculating the average power in each brain wave frequency range according to each brain wave frequency range, and determining the neural feedback index.

The range of individual brain wave frequencies includes delta (0.5-4 Hz), theta (4-8 Hz), low alpha (8-10 Hz), high alpha (10-12 Hz), low beta (12-20 Hz), high beta (20-30 Hz), and gamma (30-45 Hz).

In this embodiment, the neurofeedback index is an attention index, and the attention index is:

attention metric = high beta average power/(theta average power + low alpha average power),

since the beta rhythm is strong in a state of concentration. In the state of drowsiness and inattention, theta and alpha rhythms are strong. Therefore, the attention index is obtained by calculating the average power of high beta/(the average power of theta + the average power of low alpha).

It should be noted that, in the step S2, a single-channel electroencephalogram acquisition device may be adopted, and a multi-channel electroencephalogram device may also be adopted to acquire and process an electroencephalogram signal, so as to obtain a neural feedback index. As described above, in step S21, the recording electrode channels may be channels located in the frontal lobe region of the brain, such as Fp1, fp2, fz, F3, F4, F7, and F8, in addition to Fpz. In step S22, for a multi-channel electroencephalogram acquisition device, a neural feedback index (such as an attention index) is calculated in each recording electrode channel, and then an average value of the neural feedback indexes of different recording electrode channels is taken as a final neural feedback index.

In addition, in other embodiments, the neurofeedback index may be a relaxation index, a stress index, and an impulse control index, in addition to the attention index, for training relaxation, stress reduction, and impulse control capability, respectively. These indices can be expressed by correspondence of brain waves of different frequency bands of the electroencephalogram signal. Specific references [ Marzbani, H., marateb, H.R., & Mansourian, M. (2016.) Neurodeep: a comprehensive view on system design, methodology and clinical applications, basic and clinical neuroscience,7 (2), 143 ].

And step S3: mapping the nerve feedback index to the waveform oscillation frequency of the Clarnian graph to present a corresponding Clarnian graph;

in this example, the corresponding Clarnian graphic is selected for presentation with pygame.

In this embodiment, the relationship between the waveform oscillation frequency and the neurofeedback index is:

waveform oscillation frequency = empirical coefficient × neurofeedback index.

Thus, the brain attention state is represented by the waveform oscillation frequency (complexity of the picture) of the clarnian pattern. The larger the waveform oscillation frequency of the clarnia pattern is, the more complex the picture is, and the larger the brain attention index is.

And step S4: the method comprises the steps of S1-S3, wherein real-time animation is realized by executing the steps S1-S3 in real time and continuously, and the Clarnib graph and the electroencephalogram signal are kept synchronous (or the difference between the Clarnib graph and the electroencephalogram signal is small and fixed delay is small, is less than 10 milliseconds and is stable, and can be ignored.

In addition, in other embodiments, the pattern of the electroencephalogram signal frequency distribution and the neural feedback index such as the attention index under different Claani graphic visual stimuli can be identified in a machine learning manner. The specific steps of machine learning are to input a deep learning model for training by taking three parameters of the waveform oscillation frequency of the Clarnian graph, the pattern of electroencephalogram signal frequency distribution and the attention index as input parameters. The method can more accurately correspond the Clarnian graph, the brain electrical signals and the attention indexes of the brain. Specifically, the frequency distribution of the electroencephalogram signal is calculated in a statistical manner, thereby classifying the pattern of the frequency distribution of the electroencephalogram signal.

Besides generating the Cranib graphic visual stimulus, the audio stimulus can be generated according to the electroencephalogram signal, the brain state can be regulated and controlled together, the immersion feeling of the brain-computer interface system is improved, and therefore the attention distribution of the testee is improved. The audio stimulation is to select proper music with different emotion types to stimulate the brain according to the current state of the brain.

Results of the experiment

The existing graph generating method based on electroencephalogram signals or related features can be divided into 4 types of graph generating amplitude diagrams based on electroencephalogram signal features, frequency spectrum diagrams based on electroencephalogram signal features, bar graphs, tones and scales based on electroencephalogram signal features and generated based on electroencephalogram wave energy, and Datura images generated based on electroencephalogram signal features and provided by Grechko and Gontar, and the 4 types of graph generating method are compared with the method provided by the invention in the following table.

TABLE 1 comparison of the four methods

The amplitude diagram cannot reflect brain wave energy, cannot reflect brain waves of different frequency bands according to the graph refreshing rate, and is lack of immersion; the spectrogram can reflect brain wave energy, but cannot reflect brain waves of different frequency bands according to the graph refresh rate, and is lack of immersion; based on bar graphs, tones and scales of brain wave energy, brain wave energy can be reflected, and brain waves of different frequency bands can be reflected according to a graph refresh rate, but the brain waves lack immersion feeling. Grechko & Gontar cannot reflect brain wave energy, cannot reflect brain waves of different frequency bands according to the pattern refresh rate, and has immersion feeling. The invention can reflect brain wave energy, and can also reflect brain waves of different frequency bands according to the graph refresh rate, and has immersion feeling.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and various changes may be made in the above embodiment of the present invention. All simple and equivalent changes and modifications made according to the claims and the content of the specification of the present application fall within the scope of the claims of the present patent application.

Claims

1. A method of displaying brain wave signals using a kraani graph, comprising:

and step S4: the Claanib pattern is kept synchronous with the brain electrical signal or differs from the brain electrical signal by a fixed delay by executing the steps S1-S3 continuously in real time.

2. The method according to claim 1, wherein in step S1, a matplotlib visualization is used to generate clarnia graphs corresponding to different oscillation frequencies of the waveform.

3. The method according to claim 2, wherein the clarnia graphs corresponding to different waveform oscillation frequencies are generated by utilizing matplotlib visualization, and the method specifically comprises: digitally simulating the Kranit board, and setting the natural frequency of the Kranit board; and simulating sound, and obtaining Clarni graphs corresponding to different waveform oscillation frequencies respectively by adjusting sound frequency parameters.

4. The method for displaying brainwave signals with a Claanib diagram according to claim 1, wherein said step S2 comprises:

step S21: selecting a recording electrode channel, placing a reference electrode and a grounding electrode on an ear, and starting to acquire electroencephalogram signals;

step S22: and carrying out on-line preprocessing on the electroencephalogram signal to obtain a neural feedback index of the electroencephalogram signal.

5. The method for displaying brainwave signals with a Claanib diagram according to claim 4, wherein said step S22 includes:

step S221: filtering with a finite impulse response filter with 0.5Hz high pass and 45Hz low pass;

step S222: calculating the power spectral density by Welch;

step S223: and calculating the average power in each brain wave frequency range according to the range of each brain wave frequency, and determining the neural feedback index.

6. The method for displaying the brain wave signals according to the claim 5, wherein in the step S222, every 0.05 seconds, the last 4 seconds of brain electrical signals are selected, and the power spectral density is calculated by Welch;

in step S223, the range of each brain wave frequency includes delta, theta, low alpha, high alpha, low beta, high beta, and gamma.

7. The method for displaying the brain wave signals with the Clarniy graphs as claimed in claim 1, wherein in the step S2, a single-channel electroencephalogram acquisition device is adopted to acquire and process the electroencephalogram signals, so as to obtain the neural feedback indexes.

8. The method for displaying brain wave signals by using Claanib figures as claimed in claim 1, wherein in step S2, a multichannel electroencephalogram device is used for acquiring and processing electroencephalogram signals to obtain a neural feedback index;

for multi-channel electroencephalogram acquisition equipment, a neural feedback index is calculated in each recording electrode channel, and then the average value of the neural feedback indexes of different recording electrode channels is taken as the final neural feedback index.

9. The method of displaying brain wave signals with a kraani graph according to claim 1, wherein the neurofeedback index is an attention index, a relaxation index, a stress index, or an impulse control index.

10. The method for displaying brainwave signals with a Clarnike diagram according to claim 1, wherein in said step S3, the corresponding Clarnike diagram is selected for presentation by pygame.