WO2022257329A1 - 一种基于脉冲神经网络的脑机接口解码方法 - Google Patents

一种基于脉冲神经网络的脑机接口解码方法 Download PDF

Info

Publication number
WO2022257329A1
WO2022257329A1 PCT/CN2021/126579 CN2021126579W WO2022257329A1 WO 2022257329 A1 WO2022257329 A1 WO 2022257329A1 CN 2021126579 W CN2021126579 W CN 2021126579W WO 2022257329 A1 WO2022257329 A1 WO 2022257329A1
Authority
WO
WIPO (PCT)
Prior art keywords
layer
weight
connection
neuron
pulse
Prior art date
Application number
PCT/CN2021/126579
Other languages
English (en)
French (fr)
Inventor
祁玉
方涛
潘纲
王跃明
Original Assignee
浙江大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浙江大学 filed Critical 浙江大学
Priority to US17/913,079 priority Critical patent/US11948068B2/en
Publication of WO2022257329A1 publication Critical patent/WO2022257329A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/061—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using biological neurons, e.g. biological neurons connected to an integrated circuit
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning

Definitions

  • the invention belongs to the field of invasive action potential brain signal analysis, in particular to a brain-computer interface decoding method based on a pulse neural network.
  • the Brain Machine Interface (BMI) system is a method that does not rely on the human muscle system to directly establish a connection path between the brain and external devices.
  • BMI is responsible for collecting and analyzing the electrical signals of neural activity output by the brain, and converting them into control signals for external devices or prosthetics (computer cursors, robotic arms, etc.), so that the human brain can directly interact with devices without bypassing the nervous and muscular systems.
  • the ultimate goal of BMI development is to restore the loss of human motor function in patients due to accidental injury.
  • the BMI system has achieved high performance in decoding pulse electrical signals to predict motion information.
  • Many BMI decoding studies use shallow models to decode neural signals, including the use of Wiener filters (WF) and Kalman filters (KF), which have high decoding efficiency.
  • WF Wiener filters
  • KF Kalman filters
  • Ahmadi et al. used long-term short-term memory network (LSTM) to decode the finger movement trajectory from the neural signals of the monkey brain motor area.
  • LSTM long-term short-term memory network
  • RNN Recurrent Neural Networks
  • Gated Recurrent Unit networks Gated Recurrent Unit
  • Spiking Neural Networks are known as the third generation of artificial neural networks (Artificial Neural Networks, ANNs). Integrate-And-Fire, LIF) and other bionic neurons, and represent and transmit information according to the neuron pulse firing time, have been widely used in image classification, music recognition and other issues.
  • the Chinese patent document whose publication number is CN112906828A discloses a kind of image classification method based on time-domain coding and impulse neural network. Including: S1. Constructing a sample set based on time-domain encoding and category labeling of images; S2. Constructing a spiking neural network as a classification model; S3. Using the constructed sample set to train the spiking neural network to obtain a mature trained spiking neural network; S4 .For the image to be recognized, it is encoded in the time domain and then input into the well-trained spiking neural network to obtain the classification result of the image.
  • SNN Compared with ANN, SNN has stronger computing power, noise resistance and better biological interpretability, and the spike neuron can directly process the time information carried by the pulse signal, preventing the loss of time information caused by ANN frequency encoding, predicting more accurate.
  • SNN does not have a unified training algorithm.
  • the known classic algorithms include Tempotron, SpikeProp, ReSuMe and other single-layer or multi-layer SNN training algorithms.
  • the present invention provides a brain-computer interface decoding method based on a pulsed neural network, which can quickly train a model in a relatively short period of time, predict the trajectory of the arm movement in real time, and improve efficiency and accuracy compared with classical methods.
  • a brain-computer interface decoding method based on a spiking neural network comprising the following steps:
  • step (2-1) STDP optimizes the connection weight W hx from the input layer to the middle layer by using the traces of the synapses of front and back neurons.
  • the trace is the accumulation of attenuated pulse signals, and the formula is:
  • U and X are the pulses emitted by the input layer and the middle layer respectively;
  • T pre (t) and T post (t) are the traces accumulated by the pre-synapse and post-synapse at time t due to the pulse emission, respectively, and the parameter ⁇ is the attenuation
  • the factor controls the speed of trace attenuation during trace accumulation;
  • the STDP algorithm using trace optimization is:
  • W hx (i,j) W hx (i,j)-lr ⁇ W hx (i,j)
  • lr is the learning rate
  • the weight update amount ⁇ W hx (i,j) between neuron i and j is:
  • T up and T down are the upper and lower target values of the presynaptic trace respectively;
  • W max and W min are the maximum/minimum values of weight changes caused by STDP optimization, and the maximum value and 0.0 of W hx are taken respectively;
  • the post-synaptic neuron When the post-synaptic neuron emits a pulse, detect the pre-synaptic trace; when the current synaptic cumulative trace T pre (t) is greater than T up , it is considered that the pre-synaptic pulse sequence and post-synaptic pulse firing exist in the current period of time Significant causal association, and enhance its corresponding weight; when T pre (t) is less than T down , it is considered that the pre-synapse is not directly related to the firing of the post-synapse in the current period of time, and the weight is weakened or disconnected.
  • step (2-2) is:
  • connection weight W hh initializes the connection weight W hh as a two-dimensional matrix that obeys the standard normal distribution:
  • the pool in the middle layer is set as a three-dimensional structure.
  • the pool is composed of multiple cubes with a side length of 1.
  • Each neuron is distributed at the apex of the cube. Then, based on the Euclidean distance between two neurons, it is judged whether there is a connection, and the disconnection distance is relatively small. Connections between distant neurons.
  • W hh is a square matrix in [N,N] format, where N is the number of neurons in the middle layer, and element w ij defines the connection size between neuron j and neuron i; whether the connection between neuron i and j There is an obedience probability:
  • the parameter ⁇ defaults to 2
  • C defaults to 1.0
  • the D(i,j) function is a distance function, which measures the distance between neurons i and j, using the square of the Euclidean distance, and the formula is:
  • step (2-3) is:
  • connection weight W yh When training the connection weight W yh from the middle layer to the output layer, fix the connection between the generated input layer W hx and the middle layer W hh , and when the data arrives, calculate and update the membrane potential of the middle layer neurons in real time:
  • V(t) V(t-1)+f(W hx U(t))+f(W hh X(t-1))
  • f(W hx U(t)) is the current contribution of the input layer pulse to the post-synaptic neuron
  • f(W hh X(t-1)) is the pulse pair sent out in the middle layer cycle synapse in the past Current contribution at the current moment
  • V thres is the membrane potential release threshold
  • ⁇ is the attenuation factor, which controls the decay speed of the past trace.
  • T ⁇ ⁇ 3.21* ⁇ that is, the current liquid state is affected by at most 3.21 ⁇ time window duration, and ⁇ is taken as four times the time window value in the present invention.
  • the neurons in the middle layer adopt the cumulative emission model IF, which is the accumulation and characteristic expression of the short-term input pulse signal, and the trajectory data is highly correlated with the past neural signal, so the trace-based model with a higher ⁇ decay variable is used.
  • the accumulative liquid expression method of is a supplement to the long-term information expression, so it is sufficient to obtain similar results to the deep regression model on such a simple shallow model.
  • the optimization objective function for supervised training with ridge regression is:
  • the ⁇ t (Y(t)-W yh R(t)) term reduces the deviation between the task target and the prediction
  • the item is a penalty item, which is responsible for minimizing the variance of the parameter W yh
  • the trained connection weight W yh is fixed for real-time motion signal prediction:
  • the present invention has the following beneficial effects:
  • the present invention adopts the liquid state machine model based on the pulse neural network to directly analyze the discrete pulse signal and predict the motion trajectory according to the state of the middle layer of the network.
  • the decoding accuracy of the model algorithm, and compared with deep models such as RNN, it has a smaller amount of calculation but has a similar decoding effect.
  • the training strategy of the present invention while ensuring the model training efficiency (the maximum amount of calculation is only the supervised training of the output layer), makes the model fully optimize the connection weights for the two data domains. The results show that the method can obtain higher Accuracy.
  • Fig. 1 is the schematic diagram of animal experiment of motion signal decoding in the embodiment of the present invention.
  • Fig. 2 is a structure diagram of the brain-computer interface decoding method based on the spiking neural network of the present invention
  • Fig. 3 is the decoding effect of the embodiment of the present invention on multiple test data
  • Fig. 4 is a comparison of decoding effects of various classic models and depth models of the method of the present invention.
  • the invention adopts the invasive electrode to capture the action potential activity of the neuron in the motor area of the monkey brain, and records it as a pulse sequence signal. By analyzing brain pulse signals, it is possible to decode the animal's corresponding movement state.
  • the invention adopts the liquid state machine model based on the pulse neural network, directly analyzes the discrete pulse signal and predicts the motion track according to the state of the middle layer of the network.
  • Liquid State Machine (LSM) and echo state network both belong to the pool computing model, and are a type of cyclic neural network computing framework suitable for time series data processing.
  • the basic principle is to map the input data to a higher-dimensional intermediate layer (pool) to achieve feature expression, and train a readout weight (Readout Weight) to read and analyze the liquid state from this high-dimensional space to achieve regression or classification.
  • the liquid state machine is a single hidden layer impulse neural network.
  • the network is mainly composed of input neurons U, middle layer (pool, also called liquid layer) X, and output layer neurons Y.
  • the neurons are all bionic neurons (such as LIF or IF neurons).
  • this neuron membrane potential accumulation formula is:
  • V(t) V(t-1)+I(t)
  • I(t) is the current contribution of the pre-synaptic neuron connected to the neuron to the post-synaptic neuron at time t.
  • the main connection weights involved in the model are the weight W hx from the input layer to the middle layer, the internal loop connection weight W hh of the middle layer, and the read weight W yh from the middle layer to the output layer.
  • the input layer current contribution is expressed as:
  • the middle layer circulating current is expressed as:
  • U and X are the pulse sequences issued by the input layer and the middle layer respectively.
  • f(x) is the activation function applied to the output (current contribution) of each neuron.
  • the tanh function is adopted in the present invention, which is responsible for regularizing the current contribution to the range of (-1,1).
  • the tanh function is expressed as:
  • V(t) V(t-1)+I hx (t)+I hh (t-1)
  • I hx (t) is the current stimulation caused by the input layer pulse
  • I hh (t) is the mutual stimulation current caused by the internal interconnection of the middle layer, that is, the current middle layer membrane potential and past potential accumulation and the input layer, middle layer current Correlation, so the neurons in the middle layer have a certain short-term memory ability for past pulse signals.
  • the present invention adopts the literature "Li, H., Hao, Y., Zhang, S., Wang, Y., Chen, W., & Zheng, X. (2017). Prior Knowledge of Target Direction and Intended Movement Selection Improves Indirect Experimental data collected in Reaching Movement Decoding. Behavioral Neurology, 2017."
  • the experiment used a 96-channel microelectrode array (10 ⁇ 10 matrix arrangement, 4.2 ⁇ 4.2mm in size), and the array was implanted in the motor area PMd of the monkey cerebral cortex.
  • the monkeys carried out the above experiments and collected data after at least one week of recovery.
  • the acquired analog waveform is first amplified (amplified), band-pass filtered (0.3Hz to 7.5kHz) by a Butterworth filter, then digitized (30kHz frequency) and high-pass filtered (250Hz). Subsequently, the threshold method (minus 4.5 times the root mean square of the reference data) was used to collect the action potential pulse signal.
  • the Spike sort method is not used for preprocessing, but the single electrode channel is regarded as one or more neuron activity signals for processing.
  • the original high-frequency pulse signal is down-sampled to 300Hz data as the input of the liquid state machine.
  • the motion status can be known by observing the state of the middle layer of the liquid state machine at any time, in order to compare with the classical algorithm (which needs to count the number of pulses in a fixed time window as a feature), we set the detection frequency to 3Hz, that is, every 0.33s to read the predicted motion state from the liquid state machine.
  • the middle layer cyclic connection W hh is set in the way of distance model plus random generation.
  • connection matrix W hh is initialized as a two-dimensional matrix that obeys a standard normal distribution (mean is 0 and variance is 1):
  • the neuron connection at a distance to generate a 01 matrix mask (mask).
  • the pool in the middle layer is composed of multiple cubes with a side length of 1.
  • Each neuron is distributed at the vertex of the cube.
  • it is judged whether there is a connection or not.
  • the number N of neurons in the middle layer is set to 250
  • the connection matrix W hh of the middle layer is a square matrix in [N, N] format, wherein the element w ij defines the connection size between neuron j and neuron i. Whether the connection between neuron i and j has obedience probability:
  • the D(i, j) function is a distance function, which measures the distance between neurons i and j, using the square of the Euclidean distance.
  • the weights in the middle layer are randomly connected to reduce the amount of calculation, and the optimization for data and task goals is handed over to W hx and W yh respectively.
  • the input layer weight mainly optimizes the input connection weight for the input pulse sequence, because different data pulse frequencies and channel distributions are inconsistent, and the STDP (Spike-timing-dependent plasticity) algorithm is used in the present invention to quickly optimize the input connection without supervision.
  • STDP Spike-timing-dependent plasticity
  • the STDP algorithm is a Hebb learning rule that adjusts the connection weights based on the peak time difference between the front and rear synaptic neurons.
  • the STDP algorithm if the current synaptic neuron fires before the post-synaptic neuron, it is considered that there is a causal connection between the two and the connection is strengthened (Long-Term Potentiation, LTP), otherwise, the connection between the two is weakened (Long-Term Depression, LTD), the degree of enhancement or weakening is related to the time difference between the front and rear synaptic pulses.
  • the pulse time point is not directly used to optimize the synapse weight, because the adjacent pulse judgment LTD/LTP will ignore the pulse information continuously issued by the pre-synapse, and we use the trace of the front and rear synapses to realize STDP optimization. Trace is accumulated for the attenuated pulse signal:
  • T pre (t) and T post (t) are the traces accumulated by the pre-synaptic and post-synaptic pulses at time t, respectively.
  • the traces reflect the intensity of synaptic pulses in the most recent period. It can be explained scientifically as the accumulation of neurotransmitter release.
  • the STDP algorithm using trace optimization is:
  • W hx (i,j) W hx (i,j)-lr ⁇ W hx (i,j)
  • the weight update amount ⁇ W hx (i,j) between neuron i and j is:
  • T up and T down are the upper target and lower target values of the presynaptic trace respectively, only when the post synaptic neuron fires a pulse, we detect the presynaptic trace, and the current synaptic cumulative trace T pre (t) is greater than
  • T up we think that there is a significant causal relationship between the pulse sequence of the pre-synapse and the pulse of the post-synapse in the current period of time, and when T pre (t) is less than T down , we think that the pre-synapse is in the current period of time Not directly related to postsynaptic firing.
  • W max and W min are the maximum/minimum values of weight changes caused by STDP optimization, and take the maximum value of W hx and 0.0 respectively.
  • T up and T down in the present invention is related to specific data.
  • a relatively conservative value selection method to prevent STDP training from causing excessive polarization of neuron connections (a large number of connections are in the maximum/minimum state).
  • the pulse decoding and pulse frequency in the motor area are highly correlated (with the degree of neuron activity Correlation), so we think that the high-frequency pulse channel carries more motion information than the low-frequency channel, and we focus on strengthening the connection between the high-frequency channel and the frequently firing neurons in the middle layer, while weakening the corresponding connection of the low-frequency channel, so we calculate W hx
  • the average value of the presynaptic traces under the window size WIN stdp is sorted in ascending order, the value of the 3000th trace is taken as T down , and the value of the penultimate 200th trace is taken as T up .
  • the signal whose trace is larger than T up is judged as a short-term high-frequency pulse, and STDP will enhance its corresponding weight, and vice versa.
  • connection weight of the output layer is related to the task goal, and the output layer outputs the predicted motion trajectory Y. Therefore, W yh needs to be trained to establish the mapping between the intermediate layer liquid state information R(t) and the output motion information Y(t).
  • V(t) V(t-1)+f(W hx Y(t))+f(W hh X(t-1))
  • f(W hx Y(t)) is the current contribution of the input layer pulse to the post-synaptic neuron
  • f(W hh X(t-1)) is the pulse pair sent out in the middle layer cycle synapse in the past Current contribution at the current moment.
  • V thr is the membrane potential release threshold
  • V thre 0.15 is set.
  • expressing the liquid state of the middle layer is realized by accumulating the pulse distribution of the middle layer in the latest period of time, and the expression formula of the liquid state R (t) of the middle layer is:
  • ⁇ is the attenuation factor, which controls the decay speed of the past trace.
  • ⁇ is set to 400.
  • the ⁇ t (Y(t)-W yh R(t)) term reduces the deviation between the task target and the prediction
  • the item is a penalty item, which is responsible for minimizing the variance of the parameter W yh , where ⁇ R controls the proportion of the penalty item, and is responsible for balancing the prediction deviation and variance.
  • ⁇ R 20 is selected.
  • the trained connection weight W yh is fixed for real-time motion signal prediction:
  • ridge regression can significantly enhance the robustness of the model to the noise of pulsed nerve signals and outliers (excessive traces). Compared with the linear regression model, it has a higher accuracy rate and ensures a lower amount of training calculations.
  • the performance of this model is tested in the monkey arm movement obstacle avoidance task experiment, and the two-dimensional trajectory of the monkey arm movement is predicted.
  • invasive equipment was used to capture the dorsal premotor cortex (PMd) signal of the monkey brain with a rectangular electrode array, and the acquisition was a multi-channel discrete pulse sequence.
  • PMd dorsal premotor cortex
  • the verification uses a total of five segments of motion data, and each segment of motion data intercepts data with the same duration and the same number of detection windows, which is convenient for comparison with the fixed time window method in classic frequency coding, directly predicting the arm coordinates (x, y), and the predicted Results Pearson correlation was used to measure the decoding effect.
  • the experimental results are shown in Figures 3 and 4. The experimental results show that, on the basis of efficient training of the shallow model, this method achieves a decoding accuracy rate that exceeds that of the classic shallow model algorithm, and at the same time has a smaller The amount of calculation has a similar decoding effect.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Image Analysis (AREA)

Abstract

一种基于脉冲神经网络的脑机接口解码方法，包括：(1)构建基于脉冲神经网络的液体状态机模型，液体状态机模型由输入层、中间层和输出层构成；其中，输入层至中间层的连接权重为Whx，中间层内部的循环连接权重为Whh，中间层到输出层的读取权重为Wyh；(2)输入脉冲脑电信号，并采用以下策略训练各个权重：(2-1)采用STDP无监督训练连接权重Whx；(2-2)采用距离模型加随机连接的方式设置中间层循环连接权重Whh；(2-3)采用岭回归有监督训练连接权重Wyh，并建立中间层液态信息R(t)和输出运动信息Y(t)之间的映射，最终输出预测的运动轨迹。利用本方法，可以在较短时间内快速训练模型，实时预测手臂运动轨迹，提升效率和准确率。

Description

一种基于脉冲神经网络的脑机接口解码方法 技术领域
本发明属于侵入式动作电位脑信号分析领域，尤其是涉及一种基于脉冲神经网络的脑机接口解码方法。
背景技术
脑机接口(Brain Machine Interface，BMI)***是一种不依赖于人体肌肉***直接建立大脑和外界设备连接通路的方法。BMI负责采集和分析大脑输出的神经活动电信号，并转换为外部设备或假肢(电脑光标、机械臂等)的控制信号，从而实现人脑直接与设备交互而绕过神经和肌肉***。BMI发展的最终目的是恢复患者由于意外损伤导致的人体运动功能的缺失。
目前随着侵入式多电极阵列技术的发展，BMI***已经在解码脉冲电信号预测运动信息上取得了较高的性能表现。很多BMI解码研究采用浅层模型实现神经信号解码，包括使用维纳滤波器(Wiener filter，WF)和卡尔曼滤波器(Kalman filter，KF)等，具有较高的解码效率。为了取得更高的运动信号解码准确率，越来越多的深层模型被应用到运动信号分析中，Ahmadi等人利用长短期记忆网络(LSTM)从猴子大脑运动区神经信号中解码其手指运动轨迹，其他深度模型包括循环神经网络(Recurrent Neural Networks,RNN)和门控循环单元网络(Gated Recurrent Unit)等也被应用到运动分析中。
脉冲神经网络(Spiking Neural Networks，SNNs)被誉为第三代人工神经网络(Artificial Neural Networks，ANNs)，它由累积-发放(Integrate-And-Fire，IF)或带泄漏的累积-发放(Leaky Integrate-And-Fire，LIF)等仿生神经元构成，并依据神经元脉冲发放时间来表征和传递信息，目前已经在图像分类、音乐识别等问题上有了较多的应用。
如公开号为CN112906828A的中国专利文献公开了一种基于时域编 码和脉冲神经网络的图像分类方法。包括：S1.基于图像的时域编码和类别标记构建样本集；S2.构建脉冲神经网络作为分类模型；S3.利用构建的样本集对脉冲神经网络进行训练，得到训练成熟的脉冲神经网络；S4.对于待识别的图像，将其进行时域编码后输入训练成熟的脉冲神经网络中，得到图像的分类结果。
相比于ANN，SNN具有更强的计算能力、抗噪性和更好的生物可解释性，且脉冲神经元能够直接处理脉冲信号携带的时间信息，防止ANN频率编码对时间信息的损失，预测更为准确。目前SNN并没有统一的训练算法，已知经典算法有Tempotron，SpikeProp，ReSuMe等单层或多层SNN训练算法。
目前尚未有基于脉冲神经网络的运动神经信号解码动物运动轨迹的算法。
发明内容
本发明提供了一种基于脉冲神经网络的脑机接口解码方法，可以在较短时间内快速训练模型，实时预测手臂运动轨迹，并相比经典方法实现效率和准确率上的提升。
一种基于脉冲神经网络的脑机接口解码方法，包括以下步骤：
(1)构建基于脉冲神经网络的液体状态机模型，所述的液体状态机模型由输入层、中间层和输出层构成；其中，输入层至中间层的连接权重为W hx，中间层内部的循环连接权重为W hh，中间层到输出层的读取权重为W yh；
(2)输入脉冲脑电信号，并采用以下策略训练各个权重：
(2-1)采用STDP无监督训练输入层到中间层的连接权重W hx；
(2-2)采用距离模型加随机连接的方式设置中间层循环连接权重W hh，得到中间层液态信息R(t)；
(2-3)以中间层液态信息作为输入，采用岭回归有监督训练中间层到输出层的连接权重W yh，并建立中间层液态信息R(t)和输出运动信息Y(t)之间的映射，最终输出预测的运动轨迹。
进一步地，步骤(2-1)中，采用前后神经元突触的迹的方式实现STDP 优化输入层到中间层连接权重W hx，迹为衰减的脉冲信号累积，公式为：
Figure PCTCN2021126579-appb-000001
Figure PCTCN2021126579-appb-000002
其中，U和X分别为输入层和中间层发放脉冲；T pre(t)和T post(t)分别为前突触和后突触在t时刻由于脉冲发放而累积的迹，τ参数为衰减因子，控制了迹累积时过去迹衰减的速度；采用迹优化的STDP算法为：
W hx(i,j)＝W hx(i,j)-lr×ΔW hx(i,j)
其中，lr为学习率；神经元i和j之间权值更新量ΔW hx(i,j)为：
Figure PCTCN2021126579-appb-000003
其中，T up和T down分别为前突触迹的上目标和下目标值；W max和W min为STDP优化导致权值变化的最大/最小值，分别取W hx的最大值和0.0；
当后突触神经元发放脉冲时检测前突触的迹；当前突触累积迹T pre(t)大于T up时认为前突触的脉冲序列在当前一段时间内和后突触的脉冲发放存在显著因果关联，并增强其对应权重；而当T pre(t)小于T down时认为前突触在当前一段时间内和后突触的发放没有直接关联，并弱化该权重或断开连接。
进一步地，步骤(2-2)的具体过程为：
将连接权重W hh初始化为服从标准正态分布的二维矩阵：
W hh～N(0,1)
对标准化后的W hh进行缩放；
设置中间层水池为三维立体结构，水池由多个边长为1的立方体构成，每个神经元分布于立方体顶点处，随后基于神经元两点间欧式距离判断是否存在连接，并断开距离较远的神经元之间的连接。
进一步地，基于神经元两点间欧式距离判断是否存在连接的具体方式为：
W hh为[N,N]格式方阵，其中，N为中间层神经元数目，元素w ij定义 了神经元j到神经元i之间的连接大小；神经元i和j之间的连接是否存在服从概率：
Figure PCTCN2021126579-appb-000004
其中，参数λ默认为2，C默认为1.0；D(i,j)函数为距离函数，衡量神经元i和j之间的距离，采用欧式距离的平方，公式为：
D(i,j)＝(pos(i)-pos(j)) 2。
进一步地，步骤(2-3)的具体过程为：
训练中间层到输出层的连接权重W yh时，固定已经生成的输入层W hx和中间层W hh连接，当数据到来时，计算实时更新中间层神经元膜电位：
V(t)＝V(t-1)+f(W hxU(t))+f(W hhX(t-1))
其中，f(W hxU(t))为输入层脉冲对后突触神经元带来的电流贡献，f(W hhX(t-1))为中间层循环突触中过去发放的脉冲对当前时刻的电流贡献；
计算中间层第i个神经元的输出脉冲X i(t)：
Figure PCTCN2021126579-appb-000005
其中，V thres为膜电位发放阈值，V i(t)为中间层第i神经元t时刻膜电位，当神经元膜电位超过阈值时发放脉冲，随后V i(t)＝V rest设置回静息电压，等待下一次积累-发放活动；本发明中，膜电位发放阈值V thre＝0.15。
中间层液态信息R(t)表达公式为：
Figure PCTCN2021126579-appb-000006
其中，τ为衰减因子，其控制了过去迹衰减速度，τ越小，过去发放的脉冲对当前时刻的影响越小。
假设当t i时刻脉冲对当前时刻的影响已经衰减到小于K时视作影响消失，则τ参数对应的影响窗口长度为：
T τ＝-τlnK
当取K＝0.05时，T τ≈3.21*τ，即当前时刻液态至多受到3.21τ个时间窗口时长影响，本发明中取τ为四倍时间窗口值。
中间层神经元采用累积发放模型IF，是对短时程(short-term)输入脉 冲信号的累积与特征表达，而运动轨迹数据和过去神经信号高度相关，因此采用较高τ衰减变量的基于迹的累积的液态表达方法，是对长时程(long-term)信息表达的补充，因此足以在如此简单的浅层模型上得到和深度回归模型相近的结果。
采用岭回归有监督训练的优化目标函数为：
Figure PCTCN2021126579-appb-000007
其中，∑ t(Y(t)-W yhR(t))项降低任务目标和预测之间的偏差，而
Figure PCTCN2021126579-appb-000008
项为惩罚项，负责最小化参数W yh的方差；λ R为权重系数，用于控制惩罚项比重，负责在预测偏差和方差之间平衡；本发明中，权重系数λ R＝20。
训练过后的连接权重W yh被固定下来用于实时运动信号预测：
Y(t)＝W yhR(t)。
与现有技术相比，本发明具有以下有益效果：
1、本发明采用基于脉冲神经网络的液体状态机模型，直接分析离散脉冲信号并根据网络中间层状态预测运动轨迹，本发明的方法在浅层模型高效训练的基础上，实现了超越经典浅层模型算法的解码准确率，同时相比RNN等深度模型具有更小的计算量却具有接近的解码效果。
2、本发明的训练策略，在保证了模型训练效率(最大的计算量只在于输出层监督训练)的同时，使得模型针对两个数据域充分优化连接权重，结果表明该方法能够取得较高的准确率。
附图说明
图1为本发明实施例中运动信号解码动物实验示意图；
图2为本发明基于脉冲神经网络的脑机接口解码方法的架构图；
图3为本发明实施例在多次测试数据上的解码效果；
图4为本发明方法多种经典模型和深度模型解码效果对比。
具体实施方式
下面结合附图和实施例对本发明做进一步详细描述，需要指出的是，以下所述实施例旨在便于对本发明的理解，而对其不起任何限定作用。
本发明在实验中采用侵入式电极捕获猴脑运动区神经元动作电位活动，并记录为脉冲序列信号。通过分析大脑脉冲信号，能够解码动物对应的运动状态。
本发明采用基于脉冲神经网络的液体状态机模型，直接分析离散脉冲信号并根据网络中间层状态预测运动轨迹。
液体状态机(Liquid State Machine，LSM)和回声状态网络同属于水池计算模型，是一类适用于时间序列数据处理的循环神经网络计算框架。其基本原理是将输入数据映射到更高维度的中间层(水池)中实现特征表达，并训练一个读取权重(Readout Weight)从该高维空间中读取并分析液态，实现回归或分类。
液体状态机为单隐藏层脉冲神经网络，网络主要由输入神经元U、中间层(水池，也叫液态层)X和输出层神经元Y构成，其中神经元均为仿生神经元(如LIF或IF神经元)。本发明中作为优选，采用IF仿生神经元，该神经元膜电位累积公式为：
V(t)＝V(t-1)+I(t)
其中，I(t)为与该神经元连接的前突触神经元对后突触神经元在t时刻的电流贡献。当神经元膜电位累积超过阈值V thres时，该神经元发放脉冲并且膜电位V(t)降低为V thres。
模型中主要涉及的连接权重有输入层至中间层权重W hx、中间层内部循环连接权重W hh、中间层到输出层的读取权重W yh。
输入层电流贡献表达为：
I hx(t)＝f(W hxU(t))
中间层循环电流表达为：
I hh(t)＝f(W hhX(y-1))
其中U和X分别为输入层和中间层发放脉冲序列。f(x)为激活函数，适用于每个神经元的输出(电流贡献)上。本发明中采用tanh函数，负责规 整电流贡献到(-1,1)范围内。tanh函数表达为：
Figure PCTCN2021126579-appb-000009
因此，中间层神经元膜电位状态为：
V(t)＝V(t-1)+I hx(t)+I hh(t-1)
其中，I hx(t)为输入层脉冲引起的电流刺激，I hh(t)为中间层内部互连接引起的互相刺激电流，即当前中间层膜电位和过去电位累积以及输入层、中间层电流相关，因此中间层神经元对过去脉冲信号有一定短时记忆的能力。
如图2所示，为本发明方法的架构图，下面介绍本发明实施的具体过程：
(1)侵入式设备采集猴脑运动区数据
本发明采用文献“Li,H.,Hao,Y.,Zhang,S.,Wang,Y.,Chen,W.,& Zheng,X.(2017).Prior Knowledge of Target Direction and Intended Movement Selection Improves Indirect Reaching Movement Decoding.Behavioural Neurology,2017.”中采集的实验数据。
实验中训练两只成年雄性恒河猴，使用上肢实现障碍回避任务。过程中猴子被固定在椅子上，正前方放置显示器，猴子需要控制二维平面上的光标从初始位置绕过障碍物(绿色条)到达终点，才能获得奖励(水)，具体如图1所示。
实验采用96通道微电极阵列(10×10矩阵排列，4.2×4.2mm尺寸)，阵列被植入猴子大脑皮层运动区PMd，猴子在恢复至少一周后开展上述实验并采集数据。
采集到的模拟波形首先被放大(amplified)，通过巴特沃兹滤波器带通滤波(0.3Hz到7.5kHz)，随后被数字化(频率为30kHz)并通过高通滤波(250Hz)。随后采用阈值方法(负4.5倍基准数据均方根)采集动作电位脉冲信号。
本发明中，使用其中五天数据测试，且不使用Spike sort方法预处理，而将单电极通道视作一个或多个神经元活动信号处理。对原高频脉冲信号降采样为300Hz数据作为液体状态机的输入。虽然在任意时刻观察液体状 态机中间层状态即可得知运动状况，但为了和经典算法(需要统计固定时间窗口内的脉冲个数作为特征)对比，我们设定检测频率为3Hz，即每隔0.33s从液体状态机读取预测的运动状态。
(2)生成液态层循环连接
采用距离模型加随机生成的方式设置中间层循环连接W hh。
连接矩阵W hh初始化为服从标准正态分布的二维矩阵(均值为0，方差为1)：
W hh～N(0,1),
为避免W hhX(t)数值过大导致在激活函数f(x)中趋于饱和而失去区分度，且避免电流贡献过大导致中间层过度密集发放，因此需要对标准化后的W hh缩放(比例为0.004)。
随后依据三维模型断开距离较远神经元连接，生成01矩阵蒙板(mask)。设置中间层水池为三维立体结构，水池由多个边长为1的立方体构成，每个神经元分布于立方体顶点处，随后基于神经元两点间欧式距离判断是否存在/断开连接。本发明中中间层神经元数目N设为250，中间层连接矩阵W hh为[N,N]格式方阵，其中元素w ij定义了神经元j到神经元i之间的连接大小。神经元i和j之间的连接是否存在服从概率：
Figure PCTCN2021126579-appb-000010
其中，D(i,j)函数为距离函数，衡量神经元i和j之间的距离，采用欧式距离的平方。
D(i,j)＝(pos(i)-pos(j)) 2
本发明中选择λ＝2，并出于简化模型对所有突触选择相同的C＝1.0。随后应用上述方法断开距离较远的神经元之间的连接，从而满足稳定发放。
在中间层权重采用随机连接的方式减少计算量，而将针对数据和任务目标的优化分别交由W hx和W yh完成。
(3)无监督预训练输入层连接
输入层权重主要针对输入的脉冲序列优化输入连接权重，因为不同数据脉冲频率、通道分布不一致，本发明中采用STDP(Spike-timing-dependent plasticity)算法无监督地快速优化输入连接。
设M为输入电信号通道数，输入W hx连接初始化为服从标准正态分布的尺寸为[N,M]的矩阵。为了防止由于输入层权重过大导致中间层神经元过度密集发放的问题，同样需要对W hx进行缩放(比例为0.02)。
STDP算法是一种基于前后突触神经元峰值时间差调整连接权重的Hebb学习规则。在STDP算法中，当前突触神经元在后突触神经元之前发放，则认为两者存在因果联系并增强连接(Long-Term Potentiation，LTP)，反之则削弱两者连接(Long-Term Depression，LTD)，增强或削弱的程度与前后突触脉冲发放时间差大小相关。
本发明中不直接使用脉冲时间点优化突触权值，因为近邻脉冲判断LTD/LTP会忽略前突触连续发放的脉冲信息，我们采用前后突触的迹(trace)的方式实现STDP优化。Trace为衰减的脉冲信号累积：
Figure PCTCN2021126579-appb-000011
Figure PCTCN2021126579-appb-000012
其中，T pre(t)和T post(t)分别为前突触和后突触在t时刻由于脉冲发放而累积的迹，迹反映了当前最近一段时刻内突触脉冲发放的密集程度，生物学上可解释为神经递质释放的累积。τ参数为衰减因子，控制了迹累积时过去迹衰减的速度，τ越小则过去时刻对当前时刻迹影响越小。本发明中选择τ＝400。
采用迹优化的STDP算法为：
W hx(i,j)＝W hx(i,j)-lr×ΔW hx(i,j)
其中，神经元i和j之间权值更新量ΔW hx(i,j)为：
Figure PCTCN2021126579-appb-000013
其中T up和T down分别为前突触迹的上目标和下目标值，只有当后突触神经元发放脉冲时，我们检测前突触的迹，当前突触累积迹T pre(t)大于T up 时我们才认为前突触的脉冲序列在当前一段时间内和后突触的脉冲发放存在显著因果关联，而当T pre(t)小于T down时我们认为前突触在当前一段时间内和后突触的发放没有直接关联。W max和W min为STDP优化导致权值变化的最大/最小值，分别取W hx的最大值和0.0。采用该STDP逻辑，使得前后突触存在明显关联的连接被强化，而前后神经元发放不存在明显关联的连接则被弱化甚至断开连接。
本发明中T up和T down的选择和具体的数据相关，这里我们采用较为保守的数值选择方法，防止STDP训练导致神经元连接过度两极化(大量连接处于最大值/最小值状态)。通过我们对真实神经元信号的分析，发现其中大部分通道处于稳定的、较低发放频率状态，较少通道具有高频脉冲信号，由于运动区脉冲解码和脉冲频率高度相关(与神经元活跃程度相关)，因此我们认为高频脉冲通道相比低频通道携带更多的运动信息，着重加强高频通道与中间层频繁发放神经元之间的连接，而削弱低频通道对应连接，因此我们计算W hx突触连接矩阵中前突触迹在窗口大小为WIN stdp下的平均值按升序排序，取第3000个迹数值为T down，取倒数第200个迹数值为T up。迹大于T up的信号被判断为短时内高频脉冲，STDP会增强其对应的权重，反之亦然。
(4)有监督训练输出层连接
输出层连接权重的训练和任务目标相关，输出层输出预测的运动轨迹Y，因此，需要训练W yh建立中间层液态信息R(t)和输出运动信息Y(t)之间的映射。
训练输出层连接时，固定已经生成的输入层W hx和中间层W hh连接，当数据到来时，计算实时更新中间层神经元膜电位：
V(t)＝V(t-1)+f(W hxY(t))+f(W hhX(t-1))
其中，f(W hxY(t))为输入层脉冲对后突触神经元带来的电流贡献，f(W hhX(t-1))为中间层循环突触中过去发放的脉冲对当前时刻的电流贡献。计算中间层第i个神经元的输出脉冲X i(t)：
Figure PCTCN2021126579-appb-000014
其中，V thr为膜电位发放阈值，V i(t)为中间层第i神经元t时刻膜电位，当神经元膜电位超过阈值时发放脉冲，随后V i(t)＝V rest设置回静息电压，等待下一次积累-发放活动。本发明中设置V thre＝0.15。
本发明中表达中间层液态是通过累计中间层最近一段时间内的脉冲发放实现的，中间层液态R(t)表达公式：
Figure PCTCN2021126579-appb-000015
其中，τ为衰减因子，其控制了过去迹衰减速度，τ越小，过去发放的脉冲对当前时刻的影响越小。本发明中设τ为400。
经典液体状态机本发明中将输出层连接优化视为线性回归问题处理，采用最小二乘法优化目标函数。本发明中采用岭回归(Ridge Regression)，优化目标函数：
Figure PCTCN2021126579-appb-000016
其中，∑ t(Y(t)-W yhR(t))项降低任务目标和预测之间的偏差，而
Figure PCTCN2021126579-appb-000017
项为惩罚项，负责最小化参数W yh的方差，其中λ R控制惩罚项比重，负责在预测偏差和方差之间平衡，本发明中选择λ R＝20。
训练过后的连接权重W yh被固定下来用于实时运动信号预测：
Y(t)＝W yhR(t)
采用岭回归，能够明显增强模型对于脉冲神经信号噪声及异常值(过高的迹)的鲁棒性，相比于线性回归模型具有更高的准确率且保证了较低的训练计算量。
为了验证本方法在真实运动区神经信号解码任务上的可行性，作为特例将本模型在猴子手臂运动避障任务实验中测试性能，预测猴子手臂运动的二维运动轨迹。实验中采用侵入式设备，用矩形电极阵列捕获猴脑背侧运动前区(dorsal premotor cortex，PMd)信号，采集为多通道离散的脉冲序列。
验证采用共五段运动数据，每段运动数据截取相同时长、相同检测窗口数目的数据，便于和经典频率编码中的固定时间窗口方法相比较，直接 预测手臂坐标(x,y)，预测后的结果采用皮尔逊相关性衡量解码效果。实验结果如图3和图4所示，实验结果表明，本方法在浅层模型高效训练的基础上，实现了超越经典浅层模型算法的解码准确率，同时相比RNN等深度模型具有更小的计算量却具有接近的解码效果。
以上所述的实施例对本发明的技术方案和有益效果进行了详细说明，应理解的是以上所述仅为本发明的具体实施例，并不用于限制本发明，凡在本发明的原则范围内所做的任何修改、补充和等同替换，均应包含在本发明的保护范围之内。

Claims (8)

  1. 一种基于脉冲神经网络的脑机接口解码方法，其特征在于，包括以下步骤：
    (1)构建基于脉冲神经网络的液体状态机模型，所述的液体状态机模型由输入层、中间层和输出层构成；其中，输入层至中间层的连接权重为W hx，中间层内部的循环连接权重为W hh，中间层到输出层的读取权重为W yh；
    (2)输入脉冲脑电信号，并采用以下策略训练各个权重：
    (2-1)采用STDP无监督训练输入层到中间层的连接权重W hx；
    (2-2)采用距离模型加随机连接的方式设置中间层循环连接权重W hh，得到中间层液态信息R(t)；
    (2-3)以中间层液态信息作为输入，采用岭回归有监督训练中间层到输出层的连接权重W yh，并建立中间层液态信息R(t)和输出运动信息Y(t)之间的映射，最终输出预测的运动轨迹。
  2. 根据权利要求1所述的基于脉冲神经网络的脑机接口解码方法，其特征在于，步骤(2-1)中，采用前后神经元突触的迹的方式实现STDP优化输入层到中间层连接权重W hx，迹为衰减的脉冲信号累积，公式为：
    Figure PCTCN2021126579-appb-100001
    Figure PCTCN2021126579-appb-100002
    其中，U和X分别为输入层和中间层发放脉冲；T pre(t)和T post(t)分别为前突触和后突触在t时刻由于脉冲发放而累积的迹，τ参数为衰减因子，控制了迹累积时过去迹衰减的速度；采用迹优化的STDP算法为：
    W hx(i,j)＝W hx(i,j)-lr×ΔW hx(i,j)
    其中，lr为学习率；神经元i和j之间权值更新量ΔW hx(i,j)为：
    Figure PCTCN2021126579-appb-100003
    其中，T up和T down分别为前突触迹的上目标和下目标值；W max和W min为STDP优化导致权值变化的最大/最小值，分别取W hx的最大值和0.0；
    当后突触神经元发放脉冲时检测前突触的迹；当前突触累积迹T pre(t)大于T up时认为前突触的脉冲序列在当前一段时间内和后突触的脉冲发放存在显著因果关联，并增强其对应权重；而当T pre(t)小于T down时认为前突触在当前一段时间内和后突触的发放没有直接关联，并弱化该权重或断开连接。
  3. 根据权利要求2所述的基于脉冲神经网络的脑机接口解码方法，其特征在于，衰减因子τ的值为4倍的时间窗口值。
  4. 根据权利要求1所述的基于脉冲神经网络的脑机接口解码方法，其特征在于，步骤(2-2)的具体过程为：
    将连接权重W hh初始化为服从标准正态分布的二维矩阵：
    W hh～N(0,1)
    对标准化后的W hh进行缩放；
    设置中间层水池为三维立体结构，水池由多个边长为1的立方体构成，每个神经元分布于立方体顶点处，随后基于神经元两点间欧式距离判断是否存在连接，并断开距离较远的神经元之间的连接。
  5. 根据权利要求4所述的基于脉冲神经网络的脑机接口解码方法，其特征在于，基于神经元两点间欧式距离判断是否存在连接的具体方式为：
    W hh为[N,N]格式方阵，其中，N为中间层神经元数目，元素w ij定义了神经元j到神经元i之间的连接大小；神经元i和j之间的连接是否存在服从概率：
    Figure PCTCN2021126579-appb-100004
    其中，参数λ默认为2，C默认为1.0；D(i,j)函数为距离函数，衡量神经元i和j之间的距离，采用欧式距离的平方，公式为：
    D(i,j)＝(pos(i)-pos(j)) 2。
  6. 根据权利要求1所述的基于脉冲神经网络的脑机接口解码方法，其特征在于，步骤(2-2)中，中间层液态信息R(t)表达公式为：
    Figure PCTCN2021126579-appb-100005
    其中，τ为衰减因子，其控制了过去迹衰减速度，τ越小，过去发放的脉冲对当前时刻的影响越小。
  7. 根据权利要求1所述的基于脉冲神经网络的脑机接口解码方法，其特征在于，步骤(2-3)的具体过程为：
    训练中间层到输出层的连接权重W yh时，固定已经生成的输入层W hx和中间层W hh连接，当数据到来时，计算实时更新中间层神经元膜电位：
    V(t)＝V(t-1)+f(W hxU(t))+f(W hhX(t-1))
    其中，f(W hxU(t))为输入层脉冲对后突触神经元带来的电流贡献，f(W hhX(t-1))为中间层循环突触中过去发放的脉冲对当前时刻的电流贡献；
    计算中间层第i个神经元的输出脉冲X i(t)：
    Figure PCTCN2021126579-appb-100006
    其中，V thr为膜电位发放阈值，V i(t)为中间层第i神经元t时刻膜电位，当神经元膜电位超过阈值时发放脉冲，随后V i(t)＝V rest设置回静息电压，等待下一次积累-发放活动；
    中间层液态信息R(t)表达公式为：
    Figure PCTCN2021126579-appb-100007
    其中，τ为衰减因子，其控制了过去迹衰减速度，τ越小，过去发放的脉冲对当前时刻的影响越小；
    采用岭回归有监督训练的优化目标函数为：
    Figure PCTCN2021126579-appb-100008
    其中，∑ t(Y(t)-W yhR(t))项降低任务目标和预测之间的偏差，而
    Figure PCTCN2021126579-appb-100009
    项为惩罚项，负责最小化参数W yh的方差；λ R为权重系数，用于控制惩罚项比重，负责在预测偏差和方差之间平衡；
    训练过后的连接权重W yh被固定下来用于实时运动信号预测：
    Y(t)＝W yhR(t)。
  8. 根据权利要求7所述的基于脉冲神经网络的脑机接口解码方法，其特征在于，膜电位发放阈值V thres＝0.15，权重系数λ R＝20。
PCT/CN2021/126579 2021-06-08 2021-10-27 一种基于脉冲神经网络的脑机接口解码方法 WO2022257329A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/913,079 US11948068B2 (en) 2021-06-08 2021-10-27 Brain machine interface decoding method based on spiking neural network

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110636399.2 2021-06-08
CN202110636399.2A CN113298242B (zh) 2021-06-08 2021-06-08 一种基于脉冲神经网络的脑机接口解码方法

Publications (1)

Publication Number Publication Date
WO2022257329A1 true WO2022257329A1 (zh) 2022-12-15

Family

ID=77327578

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/126579 WO2022257329A1 (zh) 2021-06-08 2021-10-27 一种基于脉冲神经网络的脑机接口解码方法

Country Status (3)

Country Link
US (1) US11948068B2 (zh)
CN (1) CN113298242B (zh)
WO (1) WO2022257329A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116548980A (zh) * 2023-07-07 2023-08-08 广东技术师范大学 基于脉冲神经网络的长时程心电分类方法及***
CN117437382A (zh) * 2023-12-19 2024-01-23 成都电科星拓科技有限公司 一种数据中心部件的更新方法及***

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113298242B (zh) * 2021-06-08 2022-08-05 浙江大学 一种基于脉冲神经网络的脑机接口解码方法
CN114202068B (zh) * 2022-02-17 2022-06-28 浙江大学 面向类脑计算芯片的自学习实现***
CN114527878B (zh) * 2022-02-24 2024-05-17 温州医科大学 量化脑机接口控制动机的分析方法
CN115358367B (zh) * 2022-08-09 2023-04-18 浙江大学 一种基于多模型学习集成的动态自适应脑机接口解码方法
CN116304676B (zh) * 2023-02-01 2023-11-17 瑞鞍星医疗科技(苏州)有限公司 脑活动状态分类模型训练方法、装置和设备
CN117037287B (zh) * 2023-10-08 2023-12-29 武汉理工大学 一种基于3d脉冲神经网络的行为识别方法、***及装置
CN117493066B (zh) * 2023-12-28 2024-03-15 苏州元脑智能科技有限公司 一种服务器的故障预测方法、装置、设备及介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110751032A (zh) * 2019-09-20 2020-02-04 华中科技大学 一种无需校准的脑机接口模型的训练方法
WO2020115746A1 (en) * 2018-12-04 2020-06-11 Technion Research & Development Foundation Limited Delta-sigma modulation neurons for high-precision training of memristive synapses in deep neural networks
CN111783973A (zh) * 2020-08-03 2020-10-16 中国人民解放军国防科技大学 一种面向液体状态机计算的神经形态处理器及设备
CN112405542A (zh) * 2020-11-17 2021-02-26 中国科学院自动化研究所 基于脑启发多任务学习的肌肉骨骼机器人控制方法及***
CN112784976A (zh) * 2021-01-15 2021-05-11 中山大学 一种基于脉冲神经网络的图像识别***及方法
CN113298242A (zh) * 2021-06-08 2021-08-24 浙江大学 一种基于脉冲神经网络的脑机接口解码方法

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8433665B2 (en) * 2010-07-07 2013-04-30 Qualcomm Incorporated Methods and systems for three-memristor synapse with STDP and dopamine signaling
US10311375B2 (en) * 2014-10-16 2019-06-04 Nanyang Technological University Systems and methods for classifying electrical signals
US11195079B2 (en) * 2017-11-22 2021-12-07 Intel Corporation Reconfigurable neuro-synaptic cores for spiking neural network
CN108304767B (zh) * 2017-12-12 2020-06-30 中国科学院自动化研究所 基于多脑区协同计算的人类动作意图识别训练方法
US20220198245A1 (en) * 2019-04-10 2022-06-23 Cornell University Neuromorphic algorithm for rapid online learning and signal restoration
CN112329327B (zh) * 2020-11-26 2022-06-07 中国人民解放军国防科技大学 一种硬件感知的液体状态机网络生成方法及***

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020115746A1 (en) * 2018-12-04 2020-06-11 Technion Research & Development Foundation Limited Delta-sigma modulation neurons for high-precision training of memristive synapses in deep neural networks
CN110751032A (zh) * 2019-09-20 2020-02-04 华中科技大学 一种无需校准的脑机接口模型的训练方法
CN111783973A (zh) * 2020-08-03 2020-10-16 中国人民解放军国防科技大学 一种面向液体状态机计算的神经形态处理器及设备
CN112405542A (zh) * 2020-11-17 2021-02-26 中国科学院自动化研究所 基于脑启发多任务学习的肌肉骨骼机器人控制方法及***
CN112784976A (zh) * 2021-01-15 2021-05-11 中山大学 一种基于脉冲神经网络的图像识别***及方法
CN113298242A (zh) * 2021-06-08 2021-08-24 浙江大学 一种基于脉冲神经网络的脑机接口解码方法

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116548980A (zh) * 2023-07-07 2023-08-08 广东技术师范大学 基于脉冲神经网络的长时程心电分类方法及***
CN116548980B (zh) * 2023-07-07 2023-09-01 广东技术师范大学 基于脉冲神经网络的长时程心电分类方法及***
CN117437382A (zh) * 2023-12-19 2024-01-23 成都电科星拓科技有限公司 一种数据中心部件的更新方法及***
CN117437382B (zh) * 2023-12-19 2024-03-19 成都电科星拓科技有限公司 一种数据中心部件的更新方法及***

Also Published As

Publication number Publication date
CN113298242B (zh) 2022-08-05
CN113298242A (zh) 2021-08-24
US11948068B2 (en) 2024-04-02
US20230289575A1 (en) 2023-09-14

Similar Documents

Publication Publication Date Title
WO2022257329A1 (zh) 一种基于脉冲神经网络的脑机接口解码方法
Yu et al. A brain-inspired spiking neural network model with temporal encoding and learning
Jimenez Rezende et al. Stochastic variational learning in recurrent spiking networks
Sporea et al. Supervised learning in multilayer spiking neural networks
Pham et al. Control chart pattern recognition using spiking neural networks
Paugam-Moisy et al. Delay learning and polychronization for reservoir computing
Nuntalid et al. EEG classification with BSA spike encoding algorithm and evolving probabilistic spiking neural network
Bernert et al. An attention-based spiking neural network for unsupervised spike-sorting
US20170337469A1 (en) Anomaly detection using spiking neural networks
CN108992066A (zh) 基于肌电信号的便携式下肢行为模式实时识别***及方法
Thenmozhi et al. Feature selection using extreme gradient boosting Bayesian optimization to upgrade the classification performance of motor imagery signals for BCI
Zheng et al. Image segmentation method based on spiking neural network with adaptive synaptic weights
Chen et al. Unsupervised learning method for sar image classification based on spiking neural network
Wang et al. Bio-inspired model based on global-local hybrid learning in spiking neural network
Fasel An introduction to bio-inspired artificial neural network architectures
Amin et al. Hybrid spiking neural model for clustering smart environment activities
Zhang et al. The Mechanism of Orientation Detection Based on Dendritic Neuron
Lv et al. A novel spiking neural network with the learning strategy of biomimetic structure
Li et al. Maximum correntropy based attention-gated reinforcement learning designed for brain machine interface
CN116530996B (zh) 一种面向低测量负荷心电数据的异常预警方法及***
Vaila et al. Spiking CNNs with PYNN and NEURON
CN113313236B (zh) 基于时序神经通路的深度强化学习模型中毒检测方法及其装置
Michler Self-Organization of Spiking Neural Networks for Visual Object Recognition
Oudjail et al. Meta-parameters exploration for unsupervised event-based motion analysis
Li LSTM Boosted Multi-layer Residual Spiking Neural Network with Application to Motor Cortical Activities

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21944828

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21944828

Country of ref document: EP

Kind code of ref document: A1