CN115268629A - Human-robot cooperative control method based on human skill learning and simulation - Google Patents

Human-robot cooperative control method based on human skill learning and simulation Download PDF

Info

Publication number
CN115268629A
CN115268629A CN202210666670.1A CN202210666670A CN115268629A CN 115268629 A CN115268629 A CN 115268629A CN 202210666670 A CN202210666670 A CN 202210666670A CN 115268629 A CN115268629 A CN 115268629A
Authority
CN
China
Prior art keywords
human
information
robot
arm strength
force
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210666670.1A
Other languages
Chinese (zh)
Inventor
张铁
孙韩磊
邹焱飚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN202210666670.1A priority Critical patent/CN115268629A/en
Publication of CN115268629A publication Critical patent/CN115268629A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/0081—Programme-controlled manipulators with master teach-in means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Mechanical Engineering (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Robotics (AREA)
  • Human Computer Interaction (AREA)
  • Manipulator (AREA)

Abstract

The invention provides a human-robot cooperative control method based on human skill learning and simulation, which comprises the following steps: the human instructor drags the robot to move freely, and myoelectric signals, joint corner information and real arm strength of arm muscle groups of the human instructor in the process are collected to be used as training samples of the three-dimensional arm strength estimation model; extracting amplitude information and tremor information in the electromyographic signals; carrying out data fusion on the amplitude information and the joint corner information to obtain force related information in the three-dimensional motion direction; constructing a three-dimensional arm strength estimation model by means of a parallel long-time memory neural network added with an electromyographic signal correction unit and an input-output control unit; establishing a regression relationship of the arm strength as input and the movement speed as output to complete the construction of the human cooperation skill simulation model; the three-dimensional arm strength is converted into the speed adjustment of the robot, so that the robot is controlled to complete a cooperation task in cooperation with a human, and a cooperation system has more flexible interaction performance.

Description

Human-robot cooperative control method based on human skill learning and simulation
Technical Field
The invention belongs to the technical field of human-computer interaction and artificial intelligence, and particularly relates to a human-robot cooperative control method based on human skill learning and simulation.
Background
In the past decades, robotics has played an increasingly important role in industrial development. But with the demand of users for personalized and flexible manufacturing, human-robot cooperative operation is receiving wide attention. In some cooperative scenes, such as cooperative assembly, the features of small gaps and non-structuring are often included, so that the machine vision technology cannot be fully utilized. And the interaction strength information generated by the human body in the cooperation process has bidirectional feedback capability, and is very suitable for being a medium for interaction between the human body and the robot. At present, a force sensor is usually arranged at the tail end of a robot to measure the arm strength of a human body. However, this method is not completely suitable for a human-robot cooperation scenario, for example, in a human-robot cooperation process, the value of the force sensor changes due to the gravity of a moving object or the contact force between the assembly parts, so that the real arm strength information of a human body is difficult to distinguish.
Researchers find it convenient and efficient to acquire arm strength by acquiring muscle activity levels through muscle telecommunication sensors, and the method can ensure safety and fluency of human-robot interaction. But how to accurately estimate muscle strength is always the core issue of this study. With the development of computer technology, a large number of non-parametric models based on machine learning algorithms are applied to research in this field, such as FOS models, PCI models, long Short Term Memory (LSTM) models, convolutional Neural Network (CNN) models, and the like. However, currently such models focus mostly on the estimation of one-dimensional arm strength. Limited by the arm force model, the current human-robot cooperative system based on electromyographic signals is only suitable for performing some simple tasks, such as sawing, screwing and the like. Therefore, the estimation of the arm strength is expanded to a three-dimensional motion scene, the universality of the human-robot cooperation system is improved, and the method has important research value.
Clinical trials have found that human output is inaccurate and unstable, and that the output always carries some degree of involuntary tremor. And the tremor effect muscle of the muscle can be shown on the amplitude information of the myoelectric signal. The human-robot interaction information is stable and valuable, and the non-stationarity of the electromyographic signals is not beneficial to the human-robot cooperation to complete precise tasks. Therefore, for improving the accuracy of the arm force estimation, it is necessary to eliminate the fluctuation of the amplitude information of the electric signal, especially the signal fluctuation generated by the tremor effect.
In addition, with the development of human-robot cooperation technology, scientific researchers realize that rich motion information and operation skills are carried in human strength information, and the human skills are transferred to the robot, so that the efficiency of the human-robot cooperation system is improved. Peternel et al propose a multi-modal robot Teaching framework to learn the motion amplitude and motion frequency of a human instructor during a sawing process, and let the robot autonomously complete a collaborative sawing task with humans (l.peternel, t.petric, e.oztop, j.babic, teaching robots to operate with humans in a dynamic manipulation tasks of multiple-modal man-in-the-loop procedure, auton.robot.36 (2014) 123-136.). Dong et al adopt dynamic primitive language to transmit the coupling information of the collected expected track and Stiffness to the robot by way of teaching demonstration by human instructor, so as to realize the transfer and reproduction of human-robot impedance Adaptive technology (J.L.Dong, W.Y.Si, C.G.Yang, A DMP-based on Adaptive Stiff less Adjustment Method, in: IE2021-47th annular Conference of the IEEE Industrial Electronics society,2021, pp.1-6, https:/doi.org/10.1109/IECON48115.2021.9589707.). However, this kind of method only focuses on simulating the operation skill of human, and does not fully consider the direct human-robot interaction, and lacks the flexible autonomy of robot control. The skill simulation, combined with direct interaction, helps to tightly connect the operator and the robot, allowing the robot to better understand and execute human motor intent.
Disclosure of Invention
The invention aims to provide a human-robot cooperative control method based on human skill learning and simulation, and aims to transfer human skills of adjusting following speed by sensing external force change to a robot, so that the robot has more flexible cooperative capability.
The invention is realized by at least one of the following technical schemes.
The human-robot cooperative control method based on human skill learning and simulation comprises the following steps of signal acquisition and processing, three-dimensional arm strength estimation model construction, human cooperation skill simulation model construction and human-robot cooperation, wherein the steps are as follows:
step 1, a human instructor drags a robot to move freely by applying arm strength with different sizes and directions, and arm movement signals (including myoelectric signals and joint corner information of arm muscle groups) and real arm strength of the human instructor in the process are collected to be used as training samples of a three-dimensional arm strength estimation model; extracting amplitude information in the electromyographic signals by adopting a root-mean-square filter, and extracting tremor information in the electromyographic signals by adopting fast Fourier transform so as to finish signal acquisition and processing;
step 2, performing data fusion on the obtained amplitude information and joint corner information by adopting a Fast Orthogonal Search (FOS) method to obtain force related information in a three-dimensional motion direction; the acquired amplitude information, tremor information and force related information are used as input, real arm strength is output, and a three-dimensional arm strength estimation model for estimating and obtaining three-dimensional arm strength is constructed by means of a parallel long-time memory (LSTM) neural network added with an electromyographic signal correction unit and an input-output control unit;
step 3, obtaining arm strength information and speed information of a human cooperation demonstration process by means of a three-dimensional arm strength estimation model and an angle sensor, and establishing a regression relation with the arm strength as input and the movement speed as output so as to complete construction of a human cooperation skill simulation model;
and 4, directly converting the estimated three-dimensional arm strength into the speed adjustment quantity of the robot according to the constructed human cooperation skill simulation model, so as to control the robot to complete a cooperation task in cooperation with the human.
Preferably, in step 1, the electromyographic signals of the arm muscle groups include electromyographic signals of a deltoid front end, a deltoid rear end, a biceps brachii, a triceps brachii, a pectoralis major, and a infraspinatus. Wherein, the front end of the deltoid and the rear end of the deltoid are a pair of antagonistic muscles which are responsible for estimating the arm strength in the X-axis direction; the biceps brachii and the triceps brachii are a pair of antagonistic muscles and are responsible for estimating the arm strength in the Y-axis direction; the pectoralis major and infraspinatus muscles are a pair of antagonistic muscles responsible for the Z-axis arm strength estimation. In the experiment, a human instructor faces to a YZ plane of a robot base coordinate system, and the division of arm strength and motion direction is based on the robot base coordinate system.
Preferably, in step 1, the joint rotation angle information includes elbow joint rotation angle information and shoulder joint rotation angle information.
Preferably, in step 1, the tremor information is an average amplitude of the electromyographic signal of 4-12Hz, and the expression is as follows:
Figure BDA0003693162810000041
in the formula, EFFTFor tremor information, FErawThe bilateral power density spectrum of the time domain signal is obtained by fast Fourier transform; n is the sample data size; n is a radical of an alkyl radicalminAnd nmaxRepresenting the corresponding bilateral power density spectrum serial numbers at frequencies of 4Hz and 12Hz respectively.
Preferably, in step 2, the FOS method extracts the force-related information of the electromyographic signals without the influence of joint rotation by using the correlation coefficient of the extracted signals and the actual output arm force as an iteration standard, wherein the expression of the iteration standard is,
Figure BDA0003693162810000042
wherein, cov (F)y,Ey) As force-related information EyAnd measuring arm force FyThe covariance of (a); var [ F ]y]For arm strength FyThe variance of (a); var [ E ]y]As force-related information EyThe variance of (c).
Preferably, in step 2, the electromyographic signal modifying unit adopts a neural network structure based on a low-pass discrete filter principle,
EES[n]＝EES[n-1]+σ(EFFTWE+bE)(ERMS[n]-EES[n-1])；
in the formula, EES[n]The corrected electromyographic signal of the nth data; wEAnd bEThe weight and the bias of the electromyographic signal correction unit; σ (-) is a Sigmoid function; eRMS[n]And the information is the nth electromyographic signal amplitude information subjected to the root mean square filtering processing.
Preferably, the input/output control unit adopts a naive Bayes algorithm to obtain the direction of the human movement intention, and the expression is,
Figure BDA0003693162810000051
in the formula, ERMS1,ERMS2，……，ERMS6The method comprises the steps that for samples to be classified, six muscle groups are subjected to root mean square filtering, and electromyographic signal amplitude information is obtained; y iskThe human body movement intention direction is divided into four characteristic attributes which are respectively no output force, force in the X-axis direction, force in the Y-axis direction and force in the Z-axis direction, wherein when the forces in the three directions are all smaller than 4N, the human body movement intention direction is regarded as no output force; p (E)RMSi|yk) The probability of each classification sample under the condition of occurrence of the characteristic attribute is obtained; p (y)k) The conditional probability of occurrence of each characteristic attribute; p (E)RMSi) Is the probability of occurrence of the classified sample; y istThe direction of human movement intention at the current moment.
Preferably, in step 2, the parallel LSTM neural network estimates the three-dimensional arm strength by using three sub-LSTM neural networks of parallel structures.
Preferably, in step 3, the human cooperation skill simulation model adopts a K-means clustering-based multi-model gaussian process regression algorithm to obtain a regression relationship between the arm strength and the speed adjustment amount (i.e., the arm strength and the movement speed are automatically classified by a K-means clustering method, and the regression relationship between the arm strength and the speed adjustment amount is obtained by fitting a plurality of gaussian process regression models according to the classification result).
Preferably, in step 3, the human cooperation skill simulation model is a speed adjustment skill when the human faces the strength information change.
Compared with the prior art, the invention has the beneficial effects that:
the invention takes the human arm strength acquired by human motion information such as electromyographic signals and the like as a medium for human-robot interaction, and controls the robot to complete a cooperative task by learning the speed regulation skill when the human faces the external strength information change. The three-dimensional arm strength is estimated by adopting a parallel LSTM neural network added with an electromyographic signal correction unit based on tremor information and an input/output control unit based on naive Bayes, so that the influence of electromyographic signal fluctuation on the accuracy and stability of interactive information is relieved. And capturing the cooperative skills of different scenes from inaccurate human demonstration samples in a probability estimation mode by adopting a multi-model Gaussian process regression algorithm, so that the cooperative system has more flexible interactive performance.
The invention can directly convert the electromyographic signals of the human body into the speed adjustment quantity of the robot by means of the human skill simulation model. The method effectively avoids the difficulty of control parameter selection in the traditional control model, and shows good coordination capability between accurate tracking and comfortable cooperation.
Drawings
Fig. 1 is a framework of a human-robot cooperative control method based on human skill learning and simulation according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of a signal acquisition platform according to an embodiment of the present invention.
Fig. 3 is a schematic structural diagram of a three-dimensional arm force estimation model according to an embodiment of the present invention.
FIG. 4 is a schematic diagram of a human collaboration demonstration of an embodiment of the invention.
Fig. 5 is a control flowchart of a human-robot cooperative control method based on human skill learning and simulation according to an embodiment of the present invention.
Detailed Description
The human beings have super-strong perception ability and decision-making ability, and in human cooperation, the follower can adjust the speed and the position of following through perceiving external strength change. Transferring such human cooperation skills to a robot will help the development of human-robot cooperation technology towards intellectualization. However, limited by the arm force estimation model and the human-robot cooperative control method, the current human-robot cooperative system based on the electromyographic signals is only suitable for performing some simple tasks, such as sawing, screwing and the like.
The invention adds a tremor information-based electromyographic signal correction unit and a naive Bayes-based input-output control unit in the original long-and-short time memory neural network, and estimates the three-dimensional arm strength by adopting a parallel network structure. And capturing the cooperative skills of different scenes from inaccurate human demonstration samples in a probability estimation mode by adopting a multi-model Gaussian process regression algorithm, so that the human-robot cooperative control has both accuracy and rapidity. The human-robot cooperative control method provided by the invention is an artificial intelligent control mode, can directly convert arm motion information of a human instructor into the speed adjustment quantity of the robot, avoids the difficulty of control model selection in the traditional control, and realizes accurate estimation of three-dimensional arm strength, and can realize more human-robot cooperative tasks by means of the force estimation model.
For a better understanding of the present invention, reference is made to the following further description taken in conjunction with the accompanying drawings.
Example 1
As shown in fig. 1, the human-robot cooperative control method based on human skill learning and simulation includes the following steps:
1) Signal acquisition and processing
The human instructor drags the robot to move freely by applying arm strength with different sizes and directions, and arm movement signals (including myoelectric signals and joint corner information of arm muscle groups) and real arm strength of the human instructor in the process are collected to be used as training samples of the three-dimensional arm strength estimation model.
1-1) as shown in fig. 2, the goniometer is arranged outside the elbow joint and is responsible for collecting one-dimensional elbow joint rotation angle information; an Inertia Measurement Unit (IMU) is arranged at the lower end of the inner side of the large arm and is responsible for acquiring and measuring three-dimensional shoulder joint corner information; the six electromyographic sensors are respectively arranged at the front end of the deltoid, the rear end of the deltoid, the biceps brachii, the triceps brachii, the pectoralis major and the infraspinatus and collect electromyographic signals of each muscle group. And a six-dimensional force sensor is adopted to acquire real arm force. In the experiment, a human instructor faces to a YZ plane of a robot base coordinate system, and the division of arm strength and motion direction is based on the robot base coordinate system. The front end of the deltoid and the rear end of the deltoid are a pair of antagonistic muscles which are responsible for estimating the arm strength in the X-axis direction; the biceps brachii and triceps brachii are a pair of antagonistic muscles and are responsible for estimating the arm strength in the Y-axis direction; the pectoralis major and infraspinatus muscles are a pair of antagonistic muscles responsible for the Z-axis arm strength estimation.
1-2) the raw electromyogram signal cannot be directly used for the estimation of the arm strength because the electromyogram signal is time-varying, nonlinear, and it is necessary to extract the features related to the arm strength in the time-varying electromyogram signal. Extracting the amplitude information of the electromyographic signals by adopting a filter with root mean square, wherein the expression is as follows,
Figure BDA0003693162810000081
in the formula, Eraw[m]The m-th original electromyographic signal; eRMS[i]The amplitude information of the ith electromyographic signal subjected to root mean square filtering processing; n is a radical of1Is the sliding window length.
1-3) the tremor of the electromyographic signals is related to the frequency thereof, in particular to the power spectrum of 4-12Hz, therefore, the power spectrum of the electromyographic signals in the original electromyographic signals is extracted by adopting fast Fourier transform, and the average amplitude of the electromyographic signals of 4-12Hz is calculated.
The Parseval theorem indicates that the energy in the time domain and the frequency domain of the signal is equal, and the relationship between the two is as follows,
Figure BDA0003693162810000082
in the formula, FEraw[k]The kth numerical value of the bilateral power density spectrum of the time domain signal is obtained by fast Fourier transform; and N is the sample data size.
From equation (2), an average amplitude of 4-12Hz, i.e., tremor information E, can be derivedFFTIn order to realize the purpose,
Figure BDA0003693162810000091
in the formula, nminAnd nmaxRespectively represent the corresponding double-edge power density spectrum serial numbers when the frequency in the power density spectrum is 4Hz and 12 Hz.
2) Estimation of three-dimensional arm strength
In the three-dimensional arm strength analysis process, the electromyographic signals of all muscle groups are complicated, if the strength and the direction are completely predicted by depending on a neural network, the complexity of a network model is undoubtedly increased, and meanwhile, the required training data amount is huge. Therefore, the invention firstly judges the human intention direction according to the characteristic information of the electromyographic signals, acquires the corresponding intention direction and then estimates the strength value according to the electromyographic signals in the direction related to the intention direction.
2-1) naive Bayes based human intent direction prediction
Taking the electromyographic signal amplitude information of the six muscle groups subjected to root mean square filtering as a sample to be classified, wherein the sample set to be classified is E = { E = }RMS1，ERMS2，…,ERMS6}. The main output force direction (the direction of the main output force which is the direction of the maximum output force, and in the present invention, the direction of the maximum output force is regarded as the direction of human intention) is divided into four characteristic attributes, which are respectively a no-output force, an X-axis direction force, a Y-axis direction force, and a Z-axis direction force. And when the forces in the three directions are all smaller than the minimum resolution threshold of the three-dimensional arm force estimation model, determining that no output force exists.
As the electromyographic signals of each muscle group are independently collected and do not interfere with each other, the Bayesian theorem is satisfied, at the moment, the human intention direction prediction model is,
Figure BDA0003693162810000101
in the formula, ERMS1,ERMS2，……，ERMS6Respectively taking the six muscle groups of the electromyographic signal amplitude information after root mean square filtering as samples to be classified; y isiThe human body movement intention direction is divided into four characteristic attributes which are respectively a non-output force, an X-axis direction force, a Y-axis direction force and a Z-axis direction force; p (E)RMSi|yi) The probability of each classified sample under the condition of the occurrence of the characteristic attribute; p (y)i) The conditional probability of occurrence of each characteristic attribute; p (E)RMSi) Is the probability of occurrence of the classified sample; y istIs the predicted human movement intention direction at the current moment.
2-2) force-related information model
And obtaining a force related information model in each direction by adopting an FOS algorithm. The method has the advantages that the influence of the joint rotation angle on the arm strength estimation can be removed, and the fluctuation of the electromyographic signals can be reduced through a deep learning algorithm.
In this case, taking the arm strength in the Y direction as an example, the information model related to the force in the Y direction based on the FOS algorithm is,
Ey(ER[n],θ[n])＝(ER1[n]-ER2[n])-Ec(ERMS[n],θ[n])； (5)
in the formula, Ey(ER[n],θ[n]) Is a force-related information model; ec(ER[n],θ[n]) A joint rotation compensation model; eR[n]The n group of electromyographic signal amplitude information after root mean square filtering, the muscle group related to the Y direction comprises pectoralis major and infraspinatus; eR1[n]And ER2[n]The amplitude information of the nth electromyographic signals of the pectoralis major and the infraspinatus after root mean square filtering; theta [ n ]]And the nth group of joint angle values comprise a shoulder joint angle and an elbow joint angle.
Performing orthogonalization treatment on the joint rotation compensation model by adopting Gram-Schmidt,
Figure BDA0003693162810000102
in the formula, gmFor orthogonal basis functions qmThe coefficient of (n); m is the polynomial term number.
Using the correlation coefficient of the extracted force correlation information and the actual output arm force as a standard, and searching an orthogonal basis function q by adopting a quasi-Newton methodm(n) and calculating an optimum coefficient gmAnd the construction of the force-related information model is completed. The iteration criterion can be expressed as,
Figure BDA0003693162810000111
wherein, cov (F)y,Ey) As force-related information EyAnd measuring arm force FyThe covariance of (a); var [ F ]y]For arm strength FyThe variance of (a); var [ E ]y]As force-related information EyThe variance of (c).
2-3) construction of three-dimensional arm strength estimation model based on parallel LSTM neural network
(1) Myoelectric signal correction unit
The average amplitude of the electromyographic signal of 4-12Hz is closely related to the fluctuation of the electromyographic signal, and tremor information E is adoptedFFTAs a regulating parameter for reducing the fluctuation of the electromyographic signals, a discrete low-pass filter with a neural network unit structure is provided, and the amplitude E of the electromyographic signals after root mean square filtering is subjected toRMS[n]And (6) correcting. The electromyographic signal correction unit has the following structural form,
EES[n]＝EES[n-1]+σ(EFFTWE+bE)(ERMS[n]-EES[n-1])； (8)
in the formula, EES[n]The corrected electromyographic signal of the nth data; w is a group ofEAnd bECorrecting the weight and the offset of the unit for the data; σ (-) is a Sigmoid function; eRMS[n]For the nth root mean square filteringProcessed electromyographic signal amplitude information.
(2) Input/output control unit
As shown in fig. 3, the input/output control unit is responsible for controlling the input/output of data activating the main movement direction and selecting the training model. The activation direction is the direction of human intention predicted by naive Bayes.
(3) sub-LSTM neural network
And 3 sub LSTM neural networks connected in parallel are adopted to train the data model respectively, the model structure is simplified, and the operational stability is improved. In the improved neural network, except that an electromyographic signal correction unit and an input-output control unit are added, the sub LSTM neural network structure still adopts the traditional LSTM neural network structure.
3) Construction of human collaborative skill simulation model
The construction method of the human cooperation skill simulation model is introduced by taking the assembly of a human-robot cooperation shaft hole as an example. As shown in fig. 4, in the human cooperative assembly process, the leader drives the collaborators to complete the shaft hole assembly, the follower moves only along the direction with the largest sensed stress, and the arm strength information and the movement speed information of the human leader in the cooperative process are collected.
2 local optimal models (a fast model and a slow model) are adopted to fit human cooperation skills, and two performances of 'fast' and 'accurate' are considered at the same time. Dividing the cooperation force information into 2 clusters by adopting a K-means clustering algorithm, and searching the central point of each cluster in an iterative learning mode.
And respectively storing the clustered cooperation force information and the clustered speed adjustment information in different sample spaces according to the obtained clustering result, and constructing a multi-model Gaussian process regression model. Let a certain clustering space model training sample be
Figure BDA0003693162810000121
Training sample pair by adopting Gaussian process regression algorithm
Figure BDA0003693162810000122
Fitting is carried out to obtainObtaining the speed regulation skill F (F) of human facing the change of strength informationi,dFi) In which F isiTo the arm's strength, dFiIs the differential of the arm's strength. Since the sample data after clustering process obeys multidimensional Gaussian distribution, the speed adjustment skill model can be expressed as,
f(Ff,dFf)～GP(μ(Ff,dFf),k([Ff,dFf]i,[Ff,dFf]λ))； (9)
in the formula,. Mu.ffdF) is a mean function; k ([ F ]i,dFi],[Fj,dFj]) Is a sample point [ Fi,dFi],[Fj,dFj]Of the measured data.
According to the nature of the Gaussian process, for the predicted sample point [ F ]t+1,dFt+1]There is a joint gaussian distribution of, for example,
Figure BDA0003693162810000123
in the formula, v1:tSet of velocities of human motion { v } representing observations1,v2,…,vt}；ft+1Is the predicted human body velocity value. K represents the covariance matrix of the known samples; k represents the predicted sample point [ F ]t+1,dFt+1]A matrix of covariances from the remaining samples.
At this time, ft+1The a-posteriori probability of (a) is,
Figure BDA0003693162810000131
μ(Ft+1,dFt+1) I.e. the amount of robot adjustment speed obtained from the human mentor's strength information.
4) Human-robot collaboration
As shown in fig. 5, in the speed adjustment mode, the robot may convert the acquired three-dimensional arm strength into a robot speed adjustment amount according to the constructed human cooperation skill simulation model, and control the robot to complete a cooperation task in cooperation with the human.
Example 2
The present embodiment is different from embodiment 1 in that the input/output control unit recognizes the direction of the intention of movement of the human mentor using a naive bayes algorithm with a threshold. By adding a probability threshold pbReducing the prediction error caused by the noise of the sample to be classified when the maximum posterior probability is larger than the probability threshold value pbThe predicted human movement intention direction is updated only when it is. The expression of the naive bayes algorithm with the threshold is as follows,
Figure BDA0003693162810000132
in the formula, pbIs a probability threshold; y istThe predicted human movement intention direction at the current moment; y ist-1The direction of the human movement intention at the previous moment.
Example 3
Compared with embodiment 1, the difference between this embodiment and embodiment 1 is that the input/output control unit adopts an RBF neural network to identify the movement intention direction of the human instructor, and the structure of the RBF neural network is as follows:
Figure BDA0003693162810000141
wherein, is E = { E = { (E)RMS1，ERMS2，…,ERMS6Is the sample set to be classified, ERMSiRoot mean square filtered electromyographic signal amplitude information of each muscle group; eciClustering center vectors of electromyographic signals of all muscle groups; sigmaiWidth vector for hidden layer neurons; ri(E) A base function that is a hidden layer node; wikAnd dkThe output layer weight and the threshold value; m is the number of arm muscle groups; p (y)k) Is the probability of each direction of motion.
Example 4
Compared with embodiment 1, the difference of this embodiment is that a relative euclidean distance improvement is adopted for the K-means clustering, so as to improve the processing capability of the K-means on the aggregations of the boundary discrete points. The ratio of the standard euclidean distance to the maximum euclidean distance within the cluster is defined as the relative euclidean distance R. The improved k-means clustering expression is,
Figure BDA0003693162810000142
where K is the category type to which it belongs, K is the category to be selected, and xfIs that the sample to be classified is measured by the arm strength FfAnd differential thereof
Figure BDA0003693162810000143
Composition of CkIs a fast and slow sample space. dkIs the maximum euclidean distance in the kth cluster.
The sample point with the maximum Euclidean distance is searched by adopting a cross distance measurement mode, the calculation mode is as follows,
Figure BDA0003693162810000151
in the formula i1Sample point number of 2 maximum Euclidean distance from the clustering center, where i2The sample point number is the maximum euclidean distance from the cluster center 1.
At this time, the maximum euclidean distances of the sample points to the respective cluster centers can be approximately expressed as,
Figure BDA0003693162810000152
in the formula, xi1And xi2Sample point of maximum cross Euclidean distance, d1Maximum Euclidean distance of cluster center 1, d2The maximum euclidean distance of the cluster center 2.
And (4) searching the clustering center which is closest to the cooperation force information by taking the minimum sum of squares of errors of Euclidean distances from the sample points to the clustering center as an optimization target. The optimization goal may be expressed as,
Figure BDA0003693162810000153
and (4) iteratively updating the cluster centers according to the formula (14-17) until the cluster centers are not changed.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.

Claims (10)

1. The human-robot cooperative control method based on human skill learning and simulation is characterized by comprising the following steps of signal acquisition and processing, three-dimensional arm strength estimation model construction, human cooperation skill simulation model construction and human-robot cooperation, wherein the steps are as follows:
step 1, a human guide drags the robot to move freely by applying arm strength with different sizes and directions, and myoelectric signals, joint corner information and real arm strength of arm muscle groups of the human guide in the process are collected to serve as training samples of a three-dimensional arm strength estimation model; extracting amplitude information in the electromyographic signals by adopting a root-mean-square filter, and extracting tremor information in the electromyographic signals by adopting fast Fourier transform so as to finish signal acquisition and processing;
step 2, performing data fusion on the obtained amplitude information and joint corner information by adopting a fast orthogonal search method to obtain force related information in a three-dimensional motion direction; the acquired amplitude information, tremor information and force related information are used as input, real arm strength is output, and a three-dimensional arm strength estimation model for estimating and obtaining three-dimensional arm strength is constructed by means of a parallel long-time memory neural network added with an electromyographic signal correction unit and an input-output control unit;
step 3, obtaining arm strength information and speed information of a human cooperation demonstration process by means of a three-dimensional arm strength estimation model and an angle sensor, and establishing a regression relation with the arm strength as input and the movement speed as output so as to complete construction of a human cooperation skill simulation model;
and 4, directly converting the estimated three-dimensional arm strength into the speed adjustment quantity of the robot according to the constructed human cooperation skill simulation model, thereby controlling the robot to complete the cooperation task in cooperation with the human.
2. The human-robot cooperative control method based on human skill learning and simulation as claimed in claim 1, wherein the myoelectric signals of the arm muscle groups include myoelectric signals of a deltoid front end, a deltoid rear end, a biceps brachii, a triceps brachii, a pectoralis major, and a infraspinatus.
3. The human-robot cooperative control method based on human skill learning and simulation as recited in claim 1, wherein the joint rotation angle information includes elbow joint rotation angle information and shoulder joint rotation angle information.
4. The human-robot cooperative control method based on human skill learning and simulation as claimed in claim 1, wherein the tremor information is a 4-12Hz electromyographic signal mean amplitude expressed by:
Figure FDA0003693162800000021
in the formula, EFFTFor tremor information, FEraw[k]The kth numerical value of the bilateral power density spectrum of the time domain signal is obtained by fast Fourier transform; n is the sample data size; n isminAnd nmaxRepresenting the corresponding bilateral power density spectrum serial number at the frequency of 4Hz and 12Hz respectively.
5. The human-robot cooperative control method based on human skill learning and simulation as claimed in claim 2, wherein the fast orthogonal search method adopts a correlation coefficient of an extracted signal and an actual output arm force as an iteration standard, extracts force-related electromyographic signal amplitude information without joint rotation influence, and the expression of the iteration standard is,
Figure FDA0003693162800000022
wherein, cov (F)y,Ey) As force-related information EyAnd measuring arm force FyThe covariance of (a); var [ F ]y]For arm strength FyThe variance of (a); var [ E ]y]As force-related information EyThe variance of (c).
6. The human-robot cooperative control method based on human skill learning and imitation as claimed in claim 5, wherein the EMG signal correcting unit adopts a neural network structure based on a low-pass discrete filter principle,
EES[n]＝EES[n-1]+σ(EFFTWE+bE)(ERMS[n]-EES[n-1])；
in the formula, EES[n]The corrected electromyographic signal of the nth data; wEAnd bEThe weight and the offset of the electromyographic signal correction unit are obtained; σ (-) is a Sigmoid function; eRMS[n]The nth electromyographic signal amplitude information after the root mean square filtering processing.
7. The human-robot cooperative control method based on human skill learning and simulation as claimed in claim 6, wherein the input and output control unit adopts naive Bayes algorithm to obtain the direction of human motor intention, and the expression is,
Figure FDA0003693162800000031
in the formula, ERMS1,ERMS2，……，ERMS6The method comprises the steps that for samples to be classified, six muscle groups are subjected to root mean square filtering, and electromyographic signal amplitude information is obtained; y iskThe human body movement intention direction is divided into four characteristic attributes which are respectively no output force, force in the X-axis direction, force in the Y-axis direction and force in the Z-axis direction, wherein when the forces in the three directions are all smaller than 4N, the human body movement intention direction is regarded as no output force; p (E)RMSi|yk) The probability of each classified sample under the condition of the occurrence of the characteristic attribute; p (y)k) The conditional probability of occurrence of each characteristic attribute; p (E)RMSi) Is the probability of occurrence of the classified sample; y istThe direction of human movement intention at the current moment.
8. The human-robot cooperative control method based on human skill learning and simulation as claimed in claim 1, wherein the parallel long-time memory neural network adopts three sub LSTM neural networks in parallel structure to estimate the three-dimensional arm strength respectively.
9. The human-robot cooperative control method based on human skill learning and simulation of claim 1, wherein the human cooperative skill simulation model adopts a K-means clustering-based multi-model gaussian process regression algorithm to obtain a regression relationship between arm strength and speed adjustment.
10. The human-robot cooperative control method based on human skill learning and simulation as claimed in claim 1, wherein the human cooperative skill simulation model is a speed adjustment skill of human facing the change of force information.
CN202210666670.1A 2022-06-14 2022-06-14 Human-robot cooperative control method based on human skill learning and simulation Pending CN115268629A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210666670.1A CN115268629A (en) 2022-06-14 2022-06-14 Human-robot cooperative control method based on human skill learning and simulation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210666670.1A CN115268629A (en) 2022-06-14 2022-06-14 Human-robot cooperative control method based on human skill learning and simulation

Publications (1)

Publication Number Publication Date
CN115268629A true CN115268629A (en) 2022-11-01

Family

ID=83759698

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210666670.1A Pending CN115268629A (en) 2022-06-14 2022-06-14 Human-robot cooperative control method based on human skill learning and simulation

Country Status (1)

Country Link
CN (1) CN115268629A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115592671A (en) * 2022-11-03 2023-01-13 哈尔滨工业大学（Cn） Robot tail end contact positioning method based on deep learning

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115592671A (en) * 2022-11-03 2023-01-13 哈尔滨工业大学（Cn） Robot tail end contact positioning method based on deep learning

Similar Documents

Publication Publication Date Title
CN111281743B (en) Self-adaptive flexible control method for exoskeleton robot for upper limb rehabilitation
Qi et al. Intelligent human-computer interaction based on surface EMG gesture recognition
Su et al. Deep neural network approach in human-like redundancy optimization for anthropomorphic manipulators
CN108284442B (en) Mechanical arm flexible joint control method based on fuzzy neural network
CN108115681A (en) Learning by imitation method, apparatus, robot and the storage medium of robot
CN113059570B (en) Human-robot cooperative control method based on human body dynamic arm strength estimation model
CN111531537B (en) Mechanical arm control method based on multiple sensors
CN107618018B (en) Manipulator action speed proportional control method based on myoelectricity
CN112276944A (en) Man-machine cooperation system control method based on intention recognition
Zeng et al. Encoding multiple sensor data for robotic learning skills from multimodal demonstration
WO2020118730A1 (en) Compliance control method and apparatus for robot, device, and storage medium
Zhou et al. A novel muscle-computer interface for hand gesture recognition using depth vision
CN115268629A (en) Human-robot cooperative control method based on human skill learning and simulation
Jetchev et al. Task Space Retrieval Using Inverse Feedback Control.
Yang et al. Development of a robotic teaching interface for human to human skill transfer
Zhang et al. An electromyography signals-based human-robot collaboration method for human skill learning and imitation
CN117621051A (en) Active human-computer cooperation method based on human body multi-mode information
Wang et al. Continuous motion estimation of lower limbs based on deep belief networks and random forest
CN115778372A (en) Knee joint angle estimation method based on surface electromyogram signals
Wei et al. Research on robotic arm movement grasping system based on MYO
CN115091467A (en) Intent prediction and disambiguation method and system based on fuzzy Petri net
Xia et al. Echo state networks for multidimensional data: Exploiting noncircularity and widely linear models
CN112232484B (en) Space debris identification and capture method and system based on brain-like neural network
CN114952791A (en) Control method and device for musculoskeletal robot
Zhang et al. A robust electromyography signals-based interaction interface for human-robot collaboration in 3D operation scenarios

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination