WO2022027822A1 - Electromyographic signal-based intelligent gesture action generation method - Google Patents

Electromyographic signal-based intelligent gesture action generation method Download PDF

Info

Publication number
WO2022027822A1
WO2022027822A1 PCT/CN2020/120831 CN2020120831W WO2022027822A1 WO 2022027822 A1 WO2022027822 A1 WO 2022027822A1 CN 2020120831 W CN2020120831 W CN 2020120831W WO 2022027822 A1 WO2022027822 A1 WO 2022027822A1
Authority
WO
WIPO (PCT)
Prior art keywords
layer
emg
gesture
neural network
network model
Prior art date
Application number
PCT/CN2020/120831
Other languages
French (fr)
Chinese (zh)
Inventor
徐小龙
徐浩严
Original Assignee
南京邮电大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 南京邮电大学 filed Critical 南京邮电大学
Publication of WO2022027822A1 publication Critical patent/WO2022027822A1/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the invention relates to the technical field of artificial intelligence, in particular to a method for generating an intelligent gesture action based on an electromyography signal.
  • the purpose of the present invention is to provide a method for generating intelligent gesture actions based on myoelectric signals with high practicability, high flexibility, high accuracy and wide application range.
  • the present invention adopts following technical scheme for realizing above-mentioned purpose of invention:
  • the present invention provides a method for generating intelligent gesture actions based on myoelectric signals, including:
  • the EMG signal of N channels is acquired by the EMG signal acquisition equipment;
  • the gesture-related neural network model is trained through the real-time EMG signals
  • Gestures are generated from unknown EMG signals through a trained neural network model.
  • the method for training a gesture-related neural network model through the electromyographic signal acquired in real time specifically includes:
  • the neural network model is trained through the EMG signals collected by the EMG acquisition device and the corresponding gestures.
  • the method for constructing the neural network model specifically includes:
  • the input layer is constructed to obtain the input EMG signal.
  • each convolution kernel is regenerated
  • the inner product operation is performed at the beginning of the sequence until the end of the sequence to obtain a new feature layer;
  • SoftMax layer is constructed, and the output value of the fully connected layer is input into SoftMax to obtain the probability of each category.
  • the result of the feature layer output is activated through the Relu activation function to obtain the activation layer.
  • the formula of the Relu activation function is as follows:
  • max represents the largest value among all input values
  • f represents the input data
  • Ac represents the output result of the activation function.
  • the maximum pooling is performed on the Ac output from the activation layer to obtain the result of the maximum pooling.
  • the formula is as follows:
  • ma is the length of the maximum pooling
  • Po i is the result of the maximum pooling
  • i represents the ith element in the Ac matrix.
  • the fully connected layer connects Po i obtained by the pooling layer into a one-dimensional vector PO, and then connects all the values in the one-dimensional vector M to n neurons to form the output, and the value of n is the category of gesture generation.
  • the total, the expression is as follows:
  • w 1 ,w 2 ,...,w n are random weights
  • w 1 ,w 2 ,...,w n and Po i are multiplied by bits and summed to obtain multiple values
  • fc 1 ,fc 2 , ...,fc n is the output result of the fully connected layer.
  • ph is the output of each fch via SoftMax, and the obtained ph is the probability that the current target is the h -th type of gesture.
  • the method for training the neural network model by using the EMG signals collected by the EMG acquisition device and the corresponding gestures includes:
  • EMG signal in real time through the EMG acquisition device, record the electrode placement position and the corresponding gesture label of the EMG acquisition device, and use the EMG signal and gesture label to train the neural network model, and use the adaptive moment estimation optimizer to optimize the network parameters.
  • a neural network model built using gradient descent strategy training.
  • the gradient vector is first calculated for the parameter vector W of the model based on the penalty function loss(W) computed over the entire dataset. Then update the parameter W: subtract the value of the gradient value multiplied by the learning rate from the parameter W, that is, in the reverse gradient direction, update the parameter.
  • generating gestures for unknown EMG signals through the trained neural network model specifically includes:
  • the acquired EMG signal is wirelessly transmitted to the computing device, and the computing device inputs the EMG signal into the trained neural network model, and generates corresponding gestures.
  • the invention provides an electromyographic signal-based intelligent gesture action generation method with high practicability, high flexibility, high accuracy and wide application range.
  • the present invention can realize intelligent control through EMG signals in daily life, and simplify some life operations, such as furniture control in smart home. Aiming at the handicapped, the present invention can help the handicapped to restore the movement of the palm through myoelectric signals, and is no longer just a prosthetic limb that plays an aesthetic role.
  • the enhancement of the human body can be achieved, and exoskeleton equipment can help soldiers have stronger strength, and at the same time, the operation can be made more flexible through electromyographic signals.
  • the position of the present invention is not fixed when collecting EMG signals, as long as the data collected by the training model and the data collected when using the model are kept at the same electrode placement position. This makes the product accessible to persons with disabilities in different conditions.
  • the acquisition equipment is very portable, and the number of channels for collecting EMG signals is small and easy to install.
  • the device is lightweight and has low energy consumption and long battery life.
  • FIG. 1 is a structural diagram of a neural network model provided according to an embodiment of the present invention.
  • FIG. 2 is a schematic diagram of electromyography acquisition provided according to an embodiment of the present invention.
  • FIG. 3 is an overall flowchart of an intelligent prosthesis provided according to an embodiment of the present invention.
  • the main idea of this embodiment is to construct a neural network model.
  • the neural network model is trained through the EMG signals collected by the EMG acquisition equipment and the corresponding gestures, and the adaptive moment estimation optimizer is used to optimize the network parameters.
  • This case trains six gestures, namely: snapping fingers, making a fist, opening, waving to the right, waving to the left, and resting.
  • the data is acquired through an 8-channel electromyography acquisition device, and gestures are generated through the trained model, which is then transmitted to the robotic arm for gesture generation.
  • the 8-channel light-weight EMG acquisition device collects EMG signals at the forearm to generate gestures in 6 states, namely, snapping fingers, clenching fist, opening, and right. Waving, waving to the left, still. After a large number of experiments, the accuracy rate can reach 98.3%.
  • the 8-channel OpenBCI bioelectricity acquisition device and the single-layer convolutional neural network model are used to ensure the detection accuracy and reduce the computational overhead. Include the following steps:
  • Step 1 This case uses python to implement the code.
  • the neural network model is built through the Tensorflow framework, using 32 one-dimensional convolution kernels to perform convolution operations, and each convolution kernel in the convolution layer and the boundary in the input layer are filled with 0
  • the input sequence of performs the inner product operation from the beginning of the sequence to the end of the sequence to obtain a new feature layer;
  • Step 2 Activate the result of the output layer through the Relu activation function, and the Relu activation function is shown in formula (1):
  • the max function represents the largest value among all input values, where x is the input data, and Y is the output data of the activation layer.
  • Step 3 Perform maximum pooling on the Ac output from the activation layer to obtain the result of maximum pooling.
  • the formula is as follows:
  • Step 4 Input the result of pooling into the fully connected layer for classification, and map the distributed features to the sample label space; the Po i obtained by the pooling layer is connected into a one-dimensional vector PO, and finally connected to the output of 6 neurons. ;
  • w 1 ,w 2 ,...,w 6 are random weights, and w 1 ,w 2 ,...,w 6 and Po i are multiplied and summed to obtain multiple values, fc 1 ,fc 2 , ..., fc 6 is the output result of the fully connected layer.
  • Step 5 Input the output value of the fully connected layer into the SoftMax layer to obtain the probability of each gesture category.
  • the calculation method is as follows:
  • ph is the output of each fch via SoftMax, and the obtained ph is the probability that the current target is the h -th type of gesture.
  • Step 1 to Step 5 the construction of the neural network model is completed through the python language and the tensorflow framework.
  • the specific model structure diagram is shown in Figure 1.
  • Step 6 In this case, OpenBci's 8-channel bioelectric signal acquisition device is used as a portable EMG acquisition device to connect to the forearm, record the position electrode, and make corresponding actions according to six instructions: snap your fingers, make a fist, open, Wave your hand to the right, wave your hand to the left, stand still and other actions to complete the 30-second data collection of the EMG signal. Divide the collected data into 1 second length and 0.5 second step (3-second EEG samples will be divided into: 0-1 seconds, 0.5-1.5 seconds, 1-2 seconds, 1.5-2.5 seconds, 2- 3 seconds), used to train the constructed neural network model.
  • Figure 2 is a schematic diagram of EMG acquisition.
  • Step 7 Assuming that the sampling frequency of each channel of the EMG signal data X is 1000 Hz, the data format of the sampling of the n-th channel in the ith second is ⁇ x n,i,1 ,x n,i,2 ,x n,i,3 ,x n,i,3 ,...,x n,i,v-2 ,x n,i,v-1 ,x n,i,v ⁇ ; the nth channel in the ith second
  • Step 8 Input X into M(x), and get the output of M( x ) (p 1 , p 2 , .
  • the labeled data is used as the training sample data, and the neural network model constructed by the gradient descent strategy is trained.
  • the gradient vector is first calculated for the parameter vector W of the model based on the penalty function loss(W) calculated over the entire dataset.
  • update the parameter W subtract the value of the gradient value multiplied by the learning rate from the parameter W, that is, in the reverse gradient direction, update the parameter. in, is the direction of parameter gradient descent, that is, the partial derivative of loss(W), and ⁇ is the learning rate.
  • y h represents the true value of the sample
  • p h is the probability of being predicted to be the h-th class.
  • Step 9 Keep the EMG acquisition device in the same position as it was placed in Step 6.
  • Step 10 The collected EMG data is wirelessly transmitted to the computing device, and the computing device inputs the data into the neural network model we designed, recognizes the gesture to be done, and transmits the gesture command to the wearable device.
  • the robotic arm completes the action.
  • the overall flow chart is shown in Figure 3.
  • the index used in this embodiment is the accuracy rate, and the calculation method is as follows: For a binary classification problem, there are a total of n samples, which can be divided into positive examples T and negative examples F.
  • the present invention can realize intelligent control through EMG signals in daily life, and simplify some life operations, such as furniture control in smart home. Aiming at the handicapped, the present invention can help the handicapped to restore the movement of the palm through myoelectric signals, and is no longer just a prosthetic limb that plays an aesthetic role.
  • the enhancement of the human body can be achieved, and exoskeleton equipment can help soldiers have stronger strength, and at the same time, the operation can be made more flexible through electromyographic signals.
  • the position of the present invention is not fixed when collecting EMG signals, as long as the data collected by the training model and the data collected when using the model are kept at the same electrode placement position. This makes the product accessible to persons with disabilities in different conditions.
  • the acquisition equipment is very portable, and the number of channels for collecting EMG signals is small and easy to install.
  • the device is lightweight and has low energy consumption and long battery life.
  • the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
  • computer-usable storage media including, but not limited to, disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions
  • the apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Theoretical Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Psychiatry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physiology (AREA)
  • Fuzzy Systems (AREA)
  • Signal Processing (AREA)
  • Pathology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

An electromyographic signal-based intelligent gesture action generation method, comprising: acquiring electromyographic signals of N channels by means of an electromyographic signal acquisition device; training a gesture-related neural network model by means of the electromyographic signals acquired in real time; and generating a gesture for an unknown electromyographic signal by means of the trained neural network model, and then applying the generated gesture to various scenes, for example, intelligent prostheses assisting the disabled, gesture control in a smart home, and exoskeleton devices in a military. The beneficial effects of the method are to provide an electromyographic signal-based intelligent gesture action generation method with high practicability, high flexibility, high accuracy and a wide application range.

Description

一种基于肌电信号的智能手势动作生成方法A method for generating intelligent gesture actions based on EMG signals 技术领域technical field
本发明涉及人工智能技术领域，尤其涉及一种基于肌电信号的智能手势动作生成方法。The invention relates to the technical field of artificial intelligence, in particular to a method for generating an intelligent gesture action based on an electromyography signal.
背景技术Background technique
随着人工智能技术与生物电采集技术的发展，人们对于智能化的辅助设备的需求日益强烈。在残疾人的生活当中，义肢的需求不再仅仅局限于美观与一些简单的辅助，更多的是对智能化义肢的渴求。在日常生活当中，智能家居的发展我们也希望通过一些手势来控制一些智能设备。在军事方面，可以实现人体的增强，通过外骨骼设备帮助军人拥有更强的力量，同时通过肌电信号可以让操作变得更加灵活。With the development of artificial intelligence technology and bioelectricity acquisition technology, people's demand for intelligent auxiliary equipment is increasingly strong. In the life of the disabled, the demand for prosthetics is no longer limited to aesthetics and some simple assistance, but more of a desire for intelligent prosthetics. In daily life, in the development of smart home, we also hope to control some smart devices through some gestures. In the military aspect, the enhancement of the human body can be achieved, and exoskeleton equipment can help soldiers have stronger strength, and at the same time, the operation can be made more flexible through electromyographic signals.
发明内容SUMMARY OF THE INVENTION
本发明的目的是提供一种实用性高、灵活性高、准确率高、应用范围广的基于肌电信号的智能手势动作生成方法。The purpose of the present invention is to provide a method for generating intelligent gesture actions based on myoelectric signals with high practicability, high flexibility, high accuracy and wide application range.
本发明为实现上述发明目的采用如下技术方案：The present invention adopts following technical scheme for realizing above-mentioned purpose of invention:
本发明提供了一种基于肌电信号的智能手势动作生成方法，包括：The present invention provides a method for generating intelligent gesture actions based on myoelectric signals, including:
采用肌电信号采集设备获取N个通道的肌电信号；The EMG signal of N channels is acquired by the EMG signal acquisition equipment;
通过实时获取的肌电信号训练出手势相关的神经网络模型；The gesture-related neural network model is trained through the real-time EMG signals;
通过训练好的神经网络模型对未知的肌电信号进行手势生成。Gestures are generated from unknown EMG signals through a trained neural network model.
进一步地，通过实时获取的肌电信号训练出手势相关的神经网络模型的方法具体包括：Further, the method for training a gesture-related neural network model through the electromyographic signal acquired in real time specifically includes:
构建神经网络模型；Build a neural network model;
通过肌电采集设备采集的肌电信号与对应的手势来对神经网络模型进行训练。The neural network model is trained through the EMG signals collected by the EMG acquisition device and the corresponding gestures.
进一步地，构建神经网络模型的方法具体包括：Further, the method for constructing the neural network model specifically includes:
首先构建输入层，用来获取输入的肌电信号。First, the input layer is constructed to obtain the input EMG signal.
接着构建卷积层，使用多个一维卷积核对输入层获取的肌电信号进行卷积运算，对输入层进行边界补0操作得到新的序列，在卷积层中每个卷积核从新序列首端做内积运算一直到序列末端，得到一个新的特征层；Then build a convolution layer, use multiple one-dimensional convolution kernels to perform convolution operation on the EMG signals obtained by the input layer, and perform boundary padding 0 operation on the input layer to obtain a new sequence. In the convolution layer, each convolution kernel is regenerated The inner product operation is performed at the beginning of the sequence until the end of the sequence to obtain a new feature layer;
然后构建激活层，将新的特征层的结果通过激活函数进行激活，得到激活层的输出；Then build the activation layer, activate the result of the new feature layer through the activation function, and get the output of the activation layer;
接着构建最大池化层，对激活层输出的值进行最大池化，得到最大池化的结果；Then build a maximum pooling layer, and perform maximum pooling on the output value of the activation layer to obtain the result of maximum pooling;
接着构建全连接层，将最大池化的结果输入到全连接层中进行分类，把分布式特征映射到样本标记空间；得到全连接层的输出；Then build a fully connected layer, input the result of maximum pooling into the fully connected layer for classification, map the distributed features to the sample label space; get the output of the fully connected layer;
最后构建SoftMax层，将全连接层输出的值输入到SoftMax中，得到每个类别的概率。Finally, the SoftMax layer is constructed, and the output value of the fully connected layer is input into SoftMax to obtain the probability of each category.
进一步地，将特征层输出的结果通过Relu激活函数激活并得到激活层，Relu激活函数的公式如下：Further, the result of the feature layer output is activated through the Relu activation function to obtain the activation layer. The formula of the Relu activation function is as follows:
Ac＝max(0,f)Ac=max(0,f)
其中，max表示所有输入的数值当中最大的数值；f表示输入的数据；Ac表示激活函数的输出结果。Among them, max represents the largest value among all input values; f represents the input data; Ac represents the output result of the activation function.
进一步地，对激活层输出的Ac进行最大池化，得到最大池化的结果，公式如下：Further, the maximum pooling is performed on the Ac output from the activation layer to obtain the result of the maximum pooling. The formula is as follows:
Po i＝max({Ac i,Ac i+1...Ac i+ma-2,Ac i+ma-1}) Po i =max({Ac i ,Ac i+1 ...Ac i+ma-2 ,Ac i+ma-1 })
其中，ma为最大池化的长度；Po i为最大池化的结果；i表示Ac矩阵中第i个元素。 Among them, ma is the length of the maximum pooling; Po i is the result of the maximum pooling; i represents the ith element in the Ac matrix.
进一步地，全连接层由池化层得到的Po i连成一个一维向量PO，再将一维向量M中的所有值连接到n个神经元并构成输出，n的值为手势生成的类别总数，表达式如下： Further, the fully connected layer connects Po i obtained by the pooling layer into a one-dimensional vector PO, and then connects all the values in the one-dimensional vector M to n neurons to form the output, and the value of n is the category of gesture generation. The total, the expression is as follows:
(fc 1,fc 2,…,fc n)＝(∑PO·w 1,∑PO·w 2,…,∑PO·w n) (fc 1 , fc 2 ,…,fc n )=(∑PO·w 1 ,∑PO·w 2 ,…,∑PO·w n )
其中，w 1,w 2,…,w n为随机的权重，将w 1,w 2,…,w n与Po i进行对位相乘并求和得到多个值，fc 1,fc 2,…,fc n为全连接层的输出结果。 Among them, w 1 ,w 2 ,…,w n are random weights, and w 1 ,w 2 ,…,w n and Po i are multiplied by bits and summed to obtain multiple values, fc 1 ,fc 2 , ...,fc n is the output result of the fully connected layer.
进一步地，所述SoftMax计算方法公式如下：Further, the SoftMax calculation method formula is as follows:
Figure PCTCN2020120831-appb-000001
Figure PCTCN2020120831-appb-000001
p h为每个fc h经由SoftMax的输出，得到的p h为当前目标为第h类手势的概率。 ph is the output of each fch via SoftMax, and the obtained ph is the probability that the current target is the h -th type of gesture.
进一步地，通过肌电采集设备采集的肌电信号与对应的手势来对神经网络模型进行训练的方法包括：Further, the method for training the neural network model by using the EMG signals collected by the EMG acquisition device and the corresponding gestures includes:
通过肌电采集设备实时采集肌电信号，记录肌电采集设备的电极的放置位置与对应的手势标签，并使用肌电信号与手势标签来对神经网络模型进行训练，使用自适应矩估计优化器来优化网络参数。使用梯度下降策略训练构建的神经网络模型。Collect the EMG signal in real time through the EMG acquisition device, record the electrode placement position and the corresponding gesture label of the EMG acquisition device, and use the EMG signal and gesture label to train the neural network model, and use the adaptive moment estimation optimizer to optimize the network parameters. A neural network model built using gradient descent strategy training.
对于给定的迭代次数，首先基于在整个数据集上求出的罚函数loss(W)对模型的参数向量W计算梯度向量。然后对参数W进行更新：对参数W减去梯度值乘学习率的值， 也就是在反梯度方向，更新参数。For a given number of iterations, the gradient vector is first calculated for the parameter vector W of the model based on the penalty function loss(W) computed over the entire dataset. Then update the parameter W: subtract the value of the gradient value multiplied by the learning rate from the parameter W, that is, in the reverse gradient direction, update the parameter.
Figure PCTCN2020120831-appb-000002
Figure PCTCN2020120831-appb-000002
Figure PCTCN2020120831-appb-000003
Figure PCTCN2020120831-appb-000003
其中，
Figure PCTCN2020120831-appb-000004
为参数梯度下降方向，即loss(W)的偏导数，η为学习率。其中y h表示样本的真实值，p h为预测为第h类的概率。当完成迭代时，实现W的更新与模型的建立。 in,
Figure PCTCN2020120831-appb-000004
is the parameter gradient descent direction, that is, the partial derivative of loss(W), and η is the learning rate. where y h represents the true value of the sample, and p h is the probability of being predicted to be the h-th class. When the iteration is completed, the update of W and the establishment of the model are implemented.
进一步地，通过训练好的神经网络模型对未知的肌电信号进行手势生成具体包括：Further, generating gestures for unknown EMG signals through the trained neural network model specifically includes:
将肌电信号采集设备的电极放置在记录的电极位置，通过肌电采集设备来获取肌电信号，使用训练好的神经网络模型将肌电信号生成对应的手势。Place the electrodes of the EMG signal acquisition device at the recorded electrode position, acquire the EMG signal through the EMG acquisition device, and use the trained neural network model to generate the corresponding gesture from the EMG signal.
进一步地，将获取的肌电信号通过无线的方式传输到计算设备上，计算设备将肌电信号输入到训练好的神经网络模型中，并生成出相应的手势。Further, the acquired EMG signal is wirelessly transmitted to the computing device, and the computing device inputs the EMG signal into the trained neural network model, and generates corresponding gestures.
本发明的有益效果如下：The beneficial effects of the present invention are as follows:
本发明提供一种实用性高、灵活性高、准确率高、应用范围广的基于肌电信号的智能手势动作生成方法。The invention provides an electromyographic signal-based intelligent gesture action generation method with high practicability, high flexibility, high accuracy and wide application range.
实用性高：本发明在日常生活当中可以通过肌电信号来实现智能控制，简化一些生活的操作，例如智能家居中的家具控制。针对残疾人本发明可以帮助残疾人通过肌电信号恢复手掌的动作，不再仅仅是起到美观作用的义肢。在军事方面，可以实现人体的增强，通过外骨骼设备帮助军人拥有更强的力量，同时通过肌电信号可以让操作变得更加灵活。High practicability: the present invention can realize intelligent control through EMG signals in daily life, and simplify some life operations, such as furniture control in smart home. Aiming at the handicapped, the present invention can help the handicapped to restore the movement of the palm through myoelectric signals, and is no longer just a prosthetic limb that plays an aesthetic role. In the military aspect, the enhancement of the human body can be achieved, and exoskeleton equipment can help soldiers have stronger strength, and at the same time, the operation can be made more flexible through electromyographic signals.
灵活性高：本发明在采集肌电信号时位置不固定，只要保持训练模型采集的数据与使用模型时采集的数据的电极放置位置相同即可。这使得不同状况的残疾人都可以使用该产品。High flexibility: the position of the present invention is not fixed when collecting EMG signals, as long as the data collected by the training model and the data collected when using the model are kept at the same electrode placement position. This makes the product accessible to persons with disabilities in different conditions.
准确率高：目前我们针对六种手势的识别进行了实验，最终我们设计的模型对肌电信号识别手势的准确率达到了98.3％。High accuracy: At present, we have conducted experiments on the recognition of six kinds of gestures, and finally the model we designed has an accuracy of 98.3% for EMG signal recognition of gestures.
设备简单：采集设备十分便携，采集肌电信号的通道数较少易于安装。设备轻量化能耗较低，续航持久。Simple equipment: The acquisition equipment is very portable, and the number of channels for collecting EMG signals is small and easy to install. The device is lightweight and has low energy consumption and long battery life.
附图说明Description of drawings
图1为根据本发明实施例提供的神经网络模型结构图；1 is a structural diagram of a neural network model provided according to an embodiment of the present invention;
图2为根据本发明实施例提供的肌电采集示意图；2 is a schematic diagram of electromyography acquisition provided according to an embodiment of the present invention;
图3为根据本发明实施例提供的智能义肢整体流程图。FIG. 3 is an overall flowchart of an intelligent prosthesis provided according to an embodiment of the present invention.
具体实施方式detailed description
本实施例的主要思想为：构建神经网络模型。通过肌电采集设备采集的肌电信号与对应的手势来对神经网络模型进行训练，使用自适应矩估计优化器来优化网络参数。本案例训练了六种手势动作，分别为：打响指，握拳，张开，向右挥手，向左挥手，静止等动作。通过8通道的肌电采集设备来获取数据，通过训练好的模型生成手势动作，并传输到机械手臂实现手势生成。The main idea of this embodiment is to construct a neural network model. The neural network model is trained through the EMG signals collected by the EMG acquisition equipment and the corresponding gestures, and the adaptive moment estimation optimizer is used to optimize the network parameters. This case trains six gestures, namely: snapping fingers, making a fist, opening, waving to the right, waving to the left, and resting. The data is acquired through an 8-channel electromyography acquisition device, and gestures are generated through the trained model, which is then transmitted to the robotic arm for gesture generation.
实施例：本发明已经构建了原型***，通过8个通道的轻量化肌电采集设备采集小臂处的肌电信号生成了6种状态的手势，分别为打响指，握拳，张开，向右挥手，向左挥手，静止。经过大量的实验验证准确率可以达到98.3％。Example: The present invention has built a prototype system. The 8-channel light-weight EMG acquisition device collects EMG signals at the forearm to generate gestures in 6 states, namely, snapping fingers, clenching fist, opening, and right. Waving, waving to the left, still. After a large number of experiments, the accuracy rate can reach 98.3%.
本案例使用8个通道OpenBCI的生物电采集设备与单层卷积神经网络模型的情况下保证了检测的精度，减少了计算开销。包括以下步骤：In this case, the 8-channel OpenBCI bioelectricity acquisition device and the single-layer convolutional neural network model are used to ensure the detection accuracy and reduce the computational overhead. Include the following steps:
步骤1：本案例使用python实现代码，所述神经网络模型通过Tensorflow框架搭建，使用32个一维卷积核进行卷积运算，在卷积层中每个卷积核与输入层中边界补0的输入序列从序列首端做内积运算一直到序列末端，得到新的特征层；Step 1: This case uses python to implement the code. The neural network model is built through the Tensorflow framework, using 32 one-dimensional convolution kernels to perform convolution operations, and each convolution kernel in the convolution layer and the boundary in the input layer are filled with 0 The input sequence of , performs the inner product operation from the beginning of the sequence to the end of the sequence to obtain a new feature layer;
步骤2：将输出层的结果通过Relu激活函数激活，所述Relu激活函数如式(1)所示：Step 2: Activate the result of the output layer through the Relu activation function, and the Relu activation function is shown in formula (1):
Ac＝max(0,f)    (1)Ac=max(0,f) (1)
其中max函数表示输出所有输入的数值当中最大的数值，其中x为输入的数据，Y为激活层的输出数据。The max function represents the largest value among all input values, where x is the input data, and Y is the output data of the activation layer.
步骤3：对激活层输出的Ac进行最大池化，得到最大池化的结果，公式如下：Step 3: Perform maximum pooling on the Ac output from the activation layer to obtain the result of maximum pooling. The formula is as follows:
Po i＝max({Ac i,Ac i+1...Ac i+ma-2,Ac i+ma-1})    (2) Po i =max({Ac i ,Ac i+1 ...Ac i+ma-2 ,Ac i+ma-1 }) (2)
步骤4：将池化的结果输入到全连接层中进行分类，把分布式特征映射到样本标记空间；池化层得到的Po i连成一个一维向量PO，最后连接6个神经元输出构成； Step 4: Input the result of pooling into the fully connected layer for classification, and map the distributed features to the sample label space; the Po i obtained by the pooling layer is connected into a one-dimensional vector PO, and finally connected to the output of 6 neurons. ;
(fc 1,fc 2,…,fc 6)＝(∑PO·w 1,∑PO·w 2,…,∑PO·w 6)   (3) (fc 1 ,fc 2 ,…,fc 6 )=(∑PO·w 1 ,∑PO·w 2 ,…,∑PO·w 6 ) (3)
其中，w 1,w 2,…,w 6为随机的权重，将w 1,w 2,…,w 6与Po i进行对位相乘并求和得到多个值，fc 1,fc 2,…,fc 6为全连接层的输出结果。 Among them, w 1 ,w 2 ,...,w 6 are random weights, and w 1 ,w 2 ,...,w 6 and Po i are multiplied and summed to obtain multiple values, fc 1 ,fc 2 , ..., fc 6 is the output result of the fully connected layer.
步骤5：将全连接层输出的值输入到SoftMax层中，得到每个手势类别的概率，计算方法如式(4)：Step 5: Input the output value of the fully connected layer into the SoftMax layer to obtain the probability of each gesture category. The calculation method is as follows:
Figure PCTCN2020120831-appb-000005
Figure PCTCN2020120831-appb-000005
p h为每个fc h经由SoftMax的输出，得到的p h为当前目标为第h类手势的概率。 ph is the output of each fch via SoftMax, and the obtained ph is the probability that the current target is the h -th type of gesture.
步骤1到步骤5本案例通过python语言与tensorflow框架完成了神经网络模型的构建，具体模型结构图由图1所示。 Step 1 to Step 5 In this case, the construction of the neural network model is completed through the python language and the tensorflow framework. The specific model structure diagram is shown in Figure 1.
步骤6：本案例使用OpenBci的8通道生物电信号采集设备作为便携式肌电采集设备与小臂相连接，并记录位置电极，根据六种指令做出相应的动作：打响指，握拳，张开，向右挥手，向左挥手，静止等动作，完成肌电信号的30秒的数据采集。将采集的数据进行长度为1秒、步长为0.5秒的分割(3秒的EEG样本会被分为：0-1秒,0.5-1.5秒,1-2秒,1.5-2.5秒,2-3秒)，用来训练构建的神经网络模型。图2为肌电采集的示意图。Step 6: In this case, OpenBci's 8-channel bioelectric signal acquisition device is used as a portable EMG acquisition device to connect to the forearm, record the position electrode, and make corresponding actions according to six instructions: snap your fingers, make a fist, open, Wave your hand to the right, wave your hand to the left, stand still and other actions to complete the 30-second data collection of the EMG signal. Divide the collected data into 1 second length and 0.5 second step (3-second EEG samples will be divided into: 0-1 seconds, 0.5-1.5 seconds, 1-2 seconds, 1.5-2.5 seconds, 2- 3 seconds), used to train the constructed neural network model. Figure 2 is a schematic diagram of EMG acquisition.
步骤7：假设肌电信号数据X的每个通道的采样频率为1000赫兹，则第i秒第n个通道的采样的数据形式为{x n,i,1,x n,i,2,x n,i,3,x n,i,3,...,x n,i,v-2,x n,i,v-1,x n,i,v}；第i秒第n个通道的分析样本为X i,n＝{x n,i-1,v/2,x n,i-1,v/2+1,...,x n,i+1,v-2,x n,i+1,v-1,x n,i+1,v}；第i个样本为X i＝{X i,1,X i,2,X i,3,...,X i,n-1,X i,n}为一个2v·N的矩阵； Step 7: Assuming that the sampling frequency of each channel of the EMG signal data X is 1000 Hz, the data format of the sampling of the n-th channel in the ith second is {x n,i,1 ,x n,i,2 ,x n,i,3 ,x n,i,3 ,...,x n,i,v-2 ,x n,i,v-1 ,x n,i,v }; the nth channel in the ith second The analysis samples for X i,n ={x n,i-1,v/2 ,x n,i-1,v/2+1 ,...,x n,i+1,v-2 ,x n,i+1,v-1 ,x n,i+1,v }; the ith sample is X i ={X i,1 ,X i,2 ,X i,3 ,...,X i ,n-1 ,X i,n } is a 2v·N matrix;
每一个样本有一个数字化的标签，假设有n个动作，那么第i个动作的标签为y i＝(label 1，label 2，…,label i,…,label n-1，label n)。其中label i＝1，其它值均为0。 Each sample has a digital label, assuming there are n actions, then the label of the i-th action is yi = (label 1 , label 2 , ..., label i , ..., label n-1 , label n ). where label i = 1, and other values are 0.
步骤8：将X输入到M(x)中，得到M(x)的输出(p 1,p 2,…,p n)，其中p n表示是第n个类别的手势的概率。 Step 8: Input X into M(x), and get the output of M( x ) (p 1 , p 2 , .
本实施例采用有标签数据作为训练样本数据，使用梯度下降策略训练构建的神经网络模型。对于给定的迭代次数，首先基于在整个数据集上求出的罚函数loss(W)对模型的参数向量W计算梯度向量。然后对参数W进行更新：对参数W减去梯度值乘学习率的值，也就是在反梯度方向，更新参数。其中，
Figure PCTCN2020120831-appb-000006
为参数梯度下降方向，即loss(W)的偏导数，η为学习率。其中y h表示样本的真实值，p h为预测为第h类的概率。当完成迭代时，实现W的更新与模型的建立。 In this embodiment, the labeled data is used as the training sample data, and the neural network model constructed by the gradient descent strategy is trained. For a given number of iterations, the gradient vector is first calculated for the parameter vector W of the model based on the penalty function loss(W) calculated over the entire dataset. Then update the parameter W: subtract the value of the gradient value multiplied by the learning rate from the parameter W, that is, in the reverse gradient direction, update the parameter. in,
Figure PCTCN2020120831-appb-000006
is the direction of parameter gradient descent, that is, the partial derivative of loss(W), and η is the learning rate. where y h represents the true value of the sample, and p h is the probability of being predicted to be the h-th class. When the iteration is completed, the update of W and the establishment of the model are realized.
Figure PCTCN2020120831-appb-000007
Figure PCTCN2020120831-appb-000007
Figure PCTCN2020120831-appb-000008
Figure PCTCN2020120831-appb-000008
步骤9：将肌电采集设备保持与步骤6中放置的位置保持一致。Step 9: Keep the EMG acquisition device in the same position as it was placed in Step 6.
步骤10：将采集的肌电数据通过无线的方式传输到计算设备上，计算设备将数据输入到我们设计的神经网络模型当中，并识别处想要做的手势，将手势指令传输到可穿戴的机械手臂完成动作。整体流程图如图3所示。Step 10: The collected EMG data is wirelessly transmitted to the computing device, and the computing device inputs the data into the neural network model we designed, recognizes the gesture to be done, and transmits the gesture command to the wearable device. The robotic arm completes the action. The overall flow chart is shown in Figure 3.
本实施例所用指标为准确率，计算方法如下：对于一个二分类问题，共有n个样本，可以将其分为正例T与负例F。The index used in this embodiment is the accuracy rate, and the calculation method is as follows: For a binary classification problem, there are a total of n samples, which can be divided into positive examples T and negative examples F.
TableTable
   预测为Tpredicted to be T 预测为FPredicted to be F
样本为Tsample is T True Positive(TP)True Positive(TP) False Negative(FN)False Negative(FN)
样本为Fsample is F False Positive(FP)False Positive(FP) True Negative(TN)True Negative(TN)
准确率的计算公式为：The formula for calculating the accuracy rate is:
ACC＝(TP+TN)/(TP+FP+FN+TN)   (7)ACC=(TP+TN)/(TP+FP+FN+TN) (7)
最终对六种手势：打响指，握拳，张开，向右挥手，向左挥手，静止识别的准确率达到了98.3％。Finally, for six gestures: snap fingers, make a fist, open, wave to the right, wave to the left, the accuracy of static recognition reached 98.3%.
实用性高：本发明在日常生活当中可以通过肌电信号来实现智能控制，简化一些生活的操作，例如智能家居中的家具控制。针对残疾人本发明可以帮助残疾人通过肌电信号恢复手掌的动作，不再仅仅是起到美观作用的义肢。在军事方面，可以实现人体的增强，通过外骨骼设备帮助军人拥有更强的力量，同时通过肌电信号可以让操作变得更加灵活。High practicability: the present invention can realize intelligent control through EMG signals in daily life, and simplify some life operations, such as furniture control in smart home. Aiming at the handicapped, the present invention can help the handicapped to restore the movement of the palm through myoelectric signals, and is no longer just a prosthetic limb that plays an aesthetic role. In the military aspect, the enhancement of the human body can be achieved, and exoskeleton equipment can help soldiers have stronger strength, and at the same time, the operation can be made more flexible through electromyographic signals.
灵活性高：本发明在采集肌电信号时位置不固定，只要保持训练模型采集的数据与使用模型时采集的数据的电极放置位置相同即可。这使得不同状况的残疾人都可以使用该产品。High flexibility: the position of the present invention is not fixed when collecting EMG signals, as long as the data collected by the training model and the data collected when using the model are kept at the same electrode placement position. This makes the product accessible to persons with disabilities in different conditions.
准确率高：目前我们针对六种手势的识别进行了实验，最终我们设计的模型对肌电信号识别手势的准确率达到了98.3％。High accuracy: At present, we have conducted experiments on the recognition of six kinds of gestures, and finally the model we designed has an accuracy of 98.3% for EMG signal recognition of gestures.
设备简单：采集设备十分便携，采集肌电信号的通道数较少易于安装。设备轻量化能耗较低，续航持久。Simple equipment: The acquisition equipment is very portable, and the number of channels for collecting EMG signals is small and easy to install. The device is lightweight and has low energy consumption and long battery life.
本领域内的技术人员应明白，本申请的实施例可提供为方法、***、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
本申请是参照根据本申请实施例的方法、设备(***)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指 令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

Claims (10)

  1. 一种基于肌电信号的智能手势动作生成方法，其特征在于，包括：A method for generating intelligent gesture actions based on myoelectric signals, comprising:
    采用肌电信号采集设备获取N个通道的肌电信号；The EMG signal of N channels is acquired by the EMG signal acquisition equipment;
    通过实时获取的肌电信号训练出手势相关的神经网络模型；The gesture-related neural network model is trained through the real-time EMG signals;
    通过训练好的神经网络模型对未知的肌电信号进行手势生成。Gestures are generated from unknown EMG signals through a trained neural network model.
  2. 根据权利要求1所述的一种基于肌电信号的智能手势动作生成方法，其特征在于，通过实时获取的肌电信号训练出手势相关的神经网络模型的方法具体包括：The method for generating an intelligent gesture action based on myoelectric signals according to claim 1, wherein the method for training a gesture-related neural network model through the myoelectric signals obtained in real time specifically includes:
    构建神经网络模型；Build a neural network model;
    通过肌电采集设备采集的肌电信号与对应的手势来对神经网络模型进行训练。The neural network model is trained through the EMG signals collected by the EMG acquisition device and the corresponding gestures.
  3. 根据权利要求2所述的一种基于肌电信号的智能手势动作生成方法，其特征在于，构建神经网络模型的方法具体包括：The method for generating an intelligent gesture action based on myoelectric signals according to claim 2, wherein the method for constructing a neural network model specifically comprises:
    首先构建输入层，用来获取输入的肌电信号。First, the input layer is constructed to obtain the input EMG signal.
    接着构建卷积层，使用多个一维卷积核对输入层获取的肌电信号进行卷积运算，对输入层进行边界补0操作得到新的序列，在卷积层中每个卷积核从新序列首端做内积运算一直到序列末端，得到一个新的特征层；Then build a convolution layer, use multiple one-dimensional convolution kernels to perform convolution operation on the EMG signal obtained by the input layer, and perform a boundary padding 0 operation on the input layer to obtain a new sequence. In the convolution layer, each convolution kernel is regenerated The inner product operation is performed at the beginning of the sequence until the end of the sequence to obtain a new feature layer;
    然后构建激活层，将新的特征层的结果通过激活函数进行激活，得到激活层的输出；Then build the activation layer, activate the result of the new feature layer through the activation function, and get the output of the activation layer;
    接着构建最大池化层，对激活层输出的值进行最大池化，得到最大池化的结果；Then build a maximum pooling layer, and perform maximum pooling on the output value of the activation layer to obtain the result of maximum pooling;
    接着构建全连接层，将最大池化的结果输入到全连接层中进行分类，把分布式特征映射到样本标记空间；得到全连接层的输出；Then build a fully connected layer, input the result of maximum pooling into the fully connected layer for classification, map the distributed features to the sample label space; get the output of the fully connected layer;
    最后构建SoftMax层，将全连接层输出的值输入到SoftMax层中，得到每个类别的概率。Finally, the SoftMax layer is constructed, and the value output by the fully connected layer is input into the SoftMax layer to obtain the probability of each category.
  4. 根据权利要求3所述的一种基于肌电信号的智能手势动作生成方法，其特征在于，将特征层输出的结果通过Relu激活函数激活并得到激活层，Relu激活函数的公式如下：A kind of intelligent gesture action generation method based on myoelectric signal according to claim 3, is characterized in that, the result of feature layer output is activated by Relu activation function and obtains activation layer, and the formula of Relu activation function is as follows:
    Ac＝max(0,f)Ac=max(0,f)
    其中，max表示所有输入的数值当中最大的数值；f表示输入的数据；Ac表示激活函数的输出结果。Among them, max represents the largest value among all input values; f represents the input data; Ac represents the output result of the activation function.
  5. 根据权利要求4所述的一种基于肌电信号的智能手势动作生成方法，其特征在于，对激活层输出的Ac进行最大池化，得到最大池化的结果，公式如下：The method for generating an intelligent gesture action based on myoelectric signals according to claim 4, wherein the maximum pooling is performed on the Ac output from the activation layer to obtain the result of the maximum pooling, and the formula is as follows:
    Po i＝max({Ac i,Ac i+1...Ac i+ma-2,Ac i+ma-1}) Po i =max({Ac i ,Ac i+1 ...Ac i+ma-2 ,Ac i+ma-1 })
    其中，ma为最大池化的长度；Po i为最大池化的结果；Ac i表示Ac矩阵中第i个元素。 Among them, ma is the length of the maximum pooling; Po i is the result of the maximum pooling; Ac i represents the i-th element in the Ac matrix.
  6. 根据权利要求5所述的一种基于肌电信号的智能手势动作生成方法，其特征在于，所述全连接层由池化层得到的Po i连成一个一维向量PO，再将一维向量M中的所有值连接到n个神经元并构成输出，表达式如下： The method for generating intelligent gesture actions based on myoelectric signals according to claim 5, wherein the fully connected layer is connected by the Po i obtained by the pooling layer into a one-dimensional vector PO, and then the one-dimensional vector All values in M are connected to n neurons and form the output, which is expressed as:
    (fc 1,fc 2,…,fc n)＝(ΣPO·w 1,ΣPO·w 2,…,ΣPO·w n) (fc 1 ,fc 2 ,…,fc n )=(ΣPO·w 1 ,ΣPO·w 2 ,…,ΣPO·w n )
    其中，w 1,w 2,…,w n为随机的权重，将w 1,w 2,…,w n与Po i进行对位相乘并求和得到多个值，fc 1,fc 2,…,fc n为全连接层的输出，n的值为手势生成的类别总数。 Among them, w 1 ,w 2 ,…,w n are random weights, and w 1 ,w 2 ,…,w n and Po i are multiplied by bits and summed to obtain multiple values, fc 1 ,fc 2 , ...,fc n is the output of the fully connected layer, and the value of n is the total number of categories generated by gestures.
  7. 根据权利要求6所述的一种基于肌电信号的智能手势动作生成方法，其特征在于，所述SoftMax计算方法公式如下：A kind of intelligent gesture action generation method based on myoelectric signal according to claim 6, is characterized in that, described SoftMax calculation method formula is as follows:
    Figure PCTCN2020120831-appb-100001
    Figure PCTCN2020120831-appb-100001
    p h为每个fc h经由SoftMax的输出，得到的p h为当前目标为第h类手势的概率。 ph is the output of each fch via SoftMax, and the obtained ph is the probability that the current target is the h -th type of gesture.
  8. 根据权利要求2所述的一种基于肌电信号的智能手势动作生成方法，其特征在于，通过肌电采集设备采集的肌电信号与对应的手势来对神经网络模型进行训练的方法包括：The method for generating an intelligent gesture action based on an EMG signal according to claim 2, wherein the method for training the neural network model through the EMG signal collected by the EMG acquisition device and the corresponding gesture includes:
    通过肌电采集设备实时采集肌电信号，记录肌电采集设备的电极的放置位置与对应的手势标签，并使用肌电信号与手势标签来对神经网络模型进行训练，使用自适应矩估计优化器来优化网络参数。Collect the EMG signal in real time through the EMG acquisition device, record the electrode placement position and the corresponding gesture label of the EMG acquisition device, and use the EMG signal and gesture label to train the neural network model, and use the adaptive moment estimation optimizer to optimize the network parameters.
  9. 根据权利要求8所述的一种基于肌电信号的智能手势动作生成方法，其特征在于， 通过训练好的神经网络模型对未知的肌电信号进行手势生成具体包括：The method for generating an intelligent gesture action based on myoelectric signal according to claim 8, wherein the gesture generation on the unknown myoelectric signal by the trained neural network model specifically includes:
    将肌电信号采集设备的电极放置在记录的电极位置，通过肌电采集设备来获取肌电信号，使用训练好的神经网络模型将获取的肌电信号生成对应的手势。Place the electrodes of the EMG signal acquisition device at the recorded electrode position, acquire the EMG signal through the EMG acquisition device, and use the trained neural network model to generate the corresponding gesture from the acquired EMG signal.
  10. 根据权利要求9所述的一种基于肌电信号的智能手势动作生成方法，其特征在于，将获取的肌电信号通过无线的方式传输到计算设备上，计算设备将肌电信号输入到训练好的神经网络模型中，并生成出相应的手势。The method for generating an intelligent gesture action based on myoelectric signals according to claim 9, wherein the obtained myoelectric signals are wirelessly transmitted to a computing device, and the computing device inputs the myoelectric signals into the training equipment. in the neural network model and generate corresponding gestures.
PCT/CN2020/120831 2020-08-03 2020-10-14 Electromyographic signal-based intelligent gesture action generation method WO2022027822A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010765905.3 2020-08-03
CN202010765905.3A CN111870242A (en) 2020-08-03 2020-08-03 Intelligent gesture action generation method based on electromyographic signals

Publications (1)

Publication Number Publication Date
WO2022027822A1 true WO2022027822A1 (en) 2022-02-10

Family

ID=73205458

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/120831 WO2022027822A1 (en) 2020-08-03 2020-10-14 Electromyographic signal-based intelligent gesture action generation method

Country Status (2)

Country Link
CN (1) CN111870242A (en)
WO (1) WO2022027822A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114569142A (en) * 2022-02-28 2022-06-03 浙江柔灵科技有限公司 Gesture recognition method and system based on brain-like calculation and gesture recognition device
CN114848315A (en) * 2022-05-05 2022-08-05 广东工业大学 Intelligent wheelchair man-machine cooperative control system based on surface electromyogram signals
CN114863912A (en) * 2022-05-05 2022-08-05 中国科学技术大学 Silent voice decoding method based on surface electromyogram signals
CN114931389A (en) * 2022-04-27 2022-08-23 福州大学 Electromyographic signal identification method based on residual error network and graph convolution network
CN116214511A (en) * 2023-02-07 2023-06-06 南方科技大学 Outer limb control method, device, electronic equipment and readable storage medium
WO2023185887A1 (en) * 2022-03-29 2023-10-05 深圳市应和脑科学有限公司 Model acquisition system, gesture recognition method and apparatus, device, and storage medium

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114707562B (en) * 2022-06-01 2022-09-02 深圳市心流科技有限公司 Electromyographic signal sampling frequency control method and device and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100106044A1 (en) * 2008-10-27 2010-04-29 Michael Linderman EMG measured during controlled hand movement for biometric analysis, medical diagnosis and related analysis
CN105608432A (en) * 2015-12-21 2016-05-25 浙江大学 Instantaneous myoelectricity image based gesture identification method
CN107688773A (en) * 2017-07-07 2018-02-13 北京联合大学 A kind of gesture identification method based on deep learning
CN110084201A (en) * 2019-04-29 2019-08-02 福州大学 A kind of human motion recognition method of convolutional neural networks based on specific objective tracking under monitoring scene
CN110367967A (en) * 2019-07-19 2019-10-25 南京邮电大学 A kind of pocket lightweight human brain condition detection method based on data fusion
CN111209885A (en) * 2020-01-13 2020-05-29 腾讯科技（深圳）有限公司 Gesture information processing method and device, electronic equipment and storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106293057A (en) * 2016-07-20 2017-01-04 西安中科比奇创新科技有限责任公司 Gesture identification method based on BP neutral net
CN107861628A (en) * 2017-12-19 2018-03-30 许昌学院 A kind of hand gestures identifying system based on human body surface myoelectric signal

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100106044A1 (en) * 2008-10-27 2010-04-29 Michael Linderman EMG measured during controlled hand movement for biometric analysis, medical diagnosis and related analysis
CN105608432A (en) * 2015-12-21 2016-05-25 浙江大学 Instantaneous myoelectricity image based gesture identification method
CN107688773A (en) * 2017-07-07 2018-02-13 北京联合大学 A kind of gesture identification method based on deep learning
CN110084201A (en) * 2019-04-29 2019-08-02 福州大学 A kind of human motion recognition method of convolutional neural networks based on specific objective tracking under monitoring scene
CN110367967A (en) * 2019-07-19 2019-10-25 南京邮电大学 A kind of pocket lightweight human brain condition detection method based on data fusion
CN111209885A (en) * 2020-01-13 2020-05-29 腾讯科技（深圳）有限公司 Gesture information processing method and device, electronic equipment and storage medium

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114569142A (en) * 2022-02-28 2022-06-03 浙江柔灵科技有限公司 Gesture recognition method and system based on brain-like calculation and gesture recognition device
WO2023185887A1 (en) * 2022-03-29 2023-10-05 深圳市应和脑科学有限公司 Model acquisition system, gesture recognition method and apparatus, device, and storage medium
CN114931389A (en) * 2022-04-27 2022-08-23 福州大学 Electromyographic signal identification method based on residual error network and graph convolution network
CN114848315A (en) * 2022-05-05 2022-08-05 广东工业大学 Intelligent wheelchair man-machine cooperative control system based on surface electromyogram signals
CN114863912A (en) * 2022-05-05 2022-08-05 中国科学技术大学 Silent voice decoding method based on surface electromyogram signals
CN114848315B (en) * 2022-05-05 2022-12-13 广东工业大学 Intelligent wheelchair man-machine cooperative control system based on surface electromyogram signals
CN114863912B (en) * 2022-05-05 2024-05-10 中国科学技术大学 Silent voice decoding method based on surface electromyographic signals
CN116214511A (en) * 2023-02-07 2023-06-06 南方科技大学 Outer limb control method, device, electronic equipment and readable storage medium
CN116214511B (en) * 2023-02-07 2024-04-16 南方科技大学 Outer limb control method, device, electronic equipment and readable storage medium

Also Published As

Publication number Publication date
CN111870242A (en) 2020-11-03

Similar Documents

Publication Publication Date Title
WO2022027822A1 (en) Electromyographic signal-based intelligent gesture action generation method
Tuncer et al. Surface EMG signal classification using ternary pattern and discrete wavelet transform based feature extraction for hand movement recognition
Mukhopadhyay et al. An experimental study on upper limb position invariant EMG signal classification based on deep neural network
CN109262618B (en) Muscle cooperation-based upper limb multi-joint synchronous proportional myoelectric control method and system
CN111544856B (en) Brain-myoelectricity intelligent full limb rehabilitation method based on novel transfer learning model
Alwasiti et al. Motor imagery classification for brain computer interface using deep metric learning
Hwaidi et al. Classification of motor imagery EEG signals based on deep autoencoder and convolutional neural network approach
Ye et al. Optimal feature selection for EMG-based finger force estimation using LightGBM model
Shin et al. Korean sign language recognition using EMG and IMU sensors based on group-dependent NN models
Fang et al. Improve inter-day hand gesture recognition via convolutional neural network-based feature fusion
Mahendran EMG signal based control of an intelligent wheelchair
Ma et al. A novel and efficient feature extraction method for deep learning based continuous estimation
MP Idendifying eye movements using neural networks for human computer interaction
Tyacke et al. Hand gesture recognition via transient sEMG using transfer learning of dilated efficient CapsNet: towards generalization for neurorobotics
Lin et al. A normalisation approach improves the performance of inter-subject sEMG-based hand gesture recognition with a ConvNet
Rahimian et al. Trustworthy adaptation with few-shot learning for hand gesture recognition
Das et al. Hierarchical approach for fusion of electroencephalography and electromyography for predicting finger movements and kinematics using deep learning
Xie et al. Gesture recognition from bio-signals using hybrid deep neural networks
Kim et al. Sequential transfer learning via segment after cue enhances the motor imagery-based brain-computer interface
Pancholi et al. DLPR: Deep learning-based enhanced pattern recognition frame-work for improved myoelectric prosthesis control
CN112998725A (en) Rehabilitation method and system of brain-computer interface technology based on motion observation
Zanghieri sEMG-based Hand gesture recognition with deep learning
JP3816762B2 (en) Neural network, neural network system, and neural network processing program
Taghizadeh et al. Classification of Electromyography Signals Using Neural Networks and Features From Various Domains
Song et al. Lower limb movement intent recognition based on grid search random forest algorithm

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20948769

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20948769

Country of ref document: EP

Kind code of ref document: A1