CN108664129A - It is a kind of based on gesture identification gloves can learning data acquisition system and implementation method - Google Patents
It is a kind of based on gesture identification gloves can learning data acquisition system and implementation method Download PDFInfo
- Publication number
- CN108664129A CN108664129A CN201810721658.XA CN201810721658A CN108664129A CN 108664129 A CN108664129 A CN 108664129A CN 201810721658 A CN201810721658 A CN 201810721658A CN 108664129 A CN108664129 A CN 108664129A
- Authority
- CN
- China
- Prior art keywords
- pins
- data
- connect
- microcontroller
- resistance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
Abstract
It is a kind of based on gesture identification gloves can learning data acquisition system and implementation method be related to wearable smart machine field.The precision of optical profile type motion capture at present is high, but of high cost, and is limited by place；The single acceleration transducer data of inertia-type motion capture can not ensure high-precision.The present invention solves problem above.System includes nine axis attitude transducers, curvature sensor, peripheral analog to digital conversion circuit, microcontroller, Bluetooth chip and sign language database, the curvature sensor is electrically connected with the peripheral analog to digital conversion circuit, the periphery analog to digital conversion circuit is connect with the microcontroller film, the nine axis attitude transducer is electrically connected with the microcontroller, the microcontroller is electrically connected with the Bluetooth chip, and the Bluetooth chip is connected with sign language database data transmission.Feature database, input feature vector data are established in systems, and BP network identifies with trained BP networks and understand the required gesture meaning identified.
Description
Technical field
The present invention relates to wearable smart machines, and in particular to it is a kind of based on gesture identification gloves can learning data acquisition
System and implementation method.The present invention relates to a kind of intelligent controllers.The present invention relates to a kind of intelligent body-sensing game stations.
Background technology
The sign language tutoring system of mainstream is to capture user's gesture state by equipment such as video cameras at present, by identifying hand
The high bright spot opponent gesture of contouring or user's hand label is identified.This method is generally more demanding to ambient light, and
It needs to be equipped with certain picture pick-up device, using relatively not convenient enough.Due to people, the complicated flexible movements of hand are for gesture identification
Difficulty is higher, and sign language action has ambiguity, makes a breakthrough so the Sign Language Recognition equipment based on body-sensing is more difficult.
The somatic sensation television game control device of mainstream is by hand-held sensor and to coordinate button and realize simple game at present
Control, is such as waved, and the actions such as strike are mainly used in VR game.This method can only detect the mobile status of entire sensor,
Can not actual crawl and identification of the completion for player's hand motion, can not more really allow player to pass through the dynamic of itself
Make to complete to capture in game, the specific and complicated action of various gestures.
Mechanical arm is that recent decades grow up a kind of high-tech automation equipment.The control mode of Current mechanical arm
It is mostly controlled by operating lever or button, or mechanical arm is programmed and is automatically controlled with realizing, but traditional manipulator
Arm control mode efficiency is low, and inconvenience is had when some specific areas carry out more careful operation.Body-sensing technology is as one
The efficient man-machine interaction mode of kind is different from traditional interactive mode, the body-sensing technology such as button and touch and improves the straight of operation
The property seen, accuracy, flexibility.Current body-sensing technology is broadly divided into two class of optical profile type and inertia-type motion capture, and optical profile type is dynamic
Make to capture precision height, but of high cost, and is limited by place；The inertia-type motion capture of early stage is mainly adopted using acceleration transducer
Collect human action data, but single acceleration transducer data can not ensure it is high-precision.With more micromations, low power dissipation electron
The substantially development of device actively promotes research and its feasibility of wearable device.
Invention content
The present invention provides a kind of based on nine axis attitude transducer modules and curvature sensor to solve the above-mentioned problems
Wearable sign language tutoring system, at low cost and data are high-precision.
The present invention is achieved through the following technical solutions above-mentioned purpose：
It is a kind of based on gesture identification gloves can learning data acquisition system, it is characterised in that：It is sensed including nine axis postures
Device, curvature sensor, peripheral analog to digital conversion circuit, microcontroller, Bluetooth chip and host computer, curvature sensor be it is multiple,
Multiple bending sensors are set on gloves on five fingers；
The curvature sensor is electrically connected with the peripheral analog to digital conversion circuit, the periphery analog to digital conversion circuit with
The microcontroller is electrically connected, and the nine axis attitude transducer is electrically connected with the microcontroller, the microcontroller and the indigo plant
Tooth chip is electrically connected, and the Bluetooth chip is connect with host computer.
Further, including a STM32F103 microcontroller, three LM358 chips, a HC-05 bluetooth module, one
Nine axis attitude transducer modules of JY-901, five bending sensors and five resistance are constituted；One end of first bending sensor
Connect Vin1, one end of 3 pins (anode) and divider resistance r1 of another termination LM358-1；Another termination LM358-1 of resistance r1
4 pins (GND)；1 pin (Vout1) of LM358-1 connects 2 pins (cathode) of LM358-1, and connects STM32F103 microcontrollers
PA1 pins；One termination Vin2 of second bending sensor；8 pins (anode) and divider resistance of another termination LM358-1
One end of r2；4 pins (GND) of another termination LM358-1 of resistance r2；6 pins (Vout2) of LM358-1 connect LM358-1's
7 pins (cathode), and connect the PA2 pins of STM32F103 microcontrollers；One termination Vin3 of third bending sensor, the other end
Connect one end of 3 pins (anode) and divider resistance r3 of LM358-2；4 pins (GND) of another termination LM358-2 of resistance r3；
1 pin (Vout3) of LM358-2 connects 2 pins (cathode) of LM358-2, and connects the PA3 pins of STM32F103 microcontrollers；4th
One termination Vin4 of a bending sensor, one end of 3 pins (anode) and divider resistance r4 of another termination LM358-2；Resistance
4 pins (GND) of another termination LM358-2 of r4；6 pins (Vout4) of LM358-2 connect 7 pins (cathode) of LM358-2,
And connect the PA4 pins of STM32F103 microcontrollers；The one of 5th bending sensor terminates Vin5, another to terminate the 3 of LM358-3
One end of pin (anode) and divider resistance r5；4 pins (GND) of another termination LM358-3 of resistance r5；1 pipe of LM358-3
Foot (Vout5) connects 2 pins (cathode) of LM358-3, and connects the PA5 pins of STM32F103 microcontrollers；HC-05 bluetooth modules
TX pins connect the PA9 pins of STM32F103 microcontrollers；The RX pins of HC-05 bluetooth modules meet the PA10 of STM32F103 microcontrollers
Pin；The SCL pins of nine axis attitude transducer modules of JY-901 connect the PB10 pins of STM32F103 microcontrollers；Nine axis of JY-901
The SDA pins of attitude transducer module connect the PB11 pins of STM32F103 microcontrollers；By STM32F103 microcontrollers, three
LM358 chips, HC-05 bluetooth modules, the VCC pins of nine axis attitude transducer modules of JY-901, Vin1-Vin5 all connect 5V's
DC power supply；By STM32F103 microcontrollers, three LM358 chips, HC-05 bluetooth modules, nine axis attitude transducer moulds of JY-901
Block, five bending sensors GND pins all connect holding altogether.
Bending sensor and nine axis attitude transducers can provide 17 characteristics altogether；Wherein 5 item datas are hand data,
The actual physical meaning of this five groups of data is the bending degree of finger, and expression-form is the reflected voltage of electric resistance partial pressure institute
Value, comes from 5 bending sensors；The position and direction data of 12 position hands, this 12 groups of data are respectively 3 groups of Eulerian angles numbers
According to, three groups of acceleration informations, three groups of magnetic fields and three groups of gyroscopes come from nine axis attitude transducers；Establish the spy of output data
Library is levied, thereby determines which kind of gesture is the data of output be；Gesture is divided into two classes, one kind is the gesture for focusing on hand, another
Class is the gesture in the direction and position that focus on hand, during training and identification, is first divided into 17 data of input
Two parts, a part are HS data, and both shap hand-types data, referred to the digital flexion representated by bending sensor input resistance data
Degree, another part are HP and HO data, i.e. bearing data position and position data orientation, are referred to by nine divisions of China in remote antiquity appearance
The read 12 groups of related datas of state sensor；17 all data are trained in the training process, establish HP＆HO power
Value；During identification, the HS data that HS weights will be used first to identify input obtain output data, in feature after identification
It is compared in library, if finding, it is the gesture for focusing on hand, will directly export, if being not belonging to, HP＆HO be recycled to weigh
Value is identified.
Wherein training process is the learning training process of BP neural network, is divided into following several stages：
(1) hidden layer is set；Connection weight between input layer and hidden layer is set as Wij, will be between hidden layer and output layer
Connection weight be set as Wjk, each weights W is given at randomijAnd WjkAssigning an initial weight, it is desirable that each weights are not mutually equal,
Value between (- 1,1)；
(2) to each sample in sample set, the real output value O of network is determinedp；
(3) reality output O is calculatedpWith corresponding ideal output YpBetween difference；
(4) it presses minimum mode error and adjusts weights；
(5) judge whether maximum iteration is more than a previously given number, do not reach, go to (2)；
Every group of sample has m data, for pth group sample, if output valve is ypj, real output value OpjTake its sample
Error
The biasing b of iteration each time is calculated by sample errorp
α is learning rate in each iteration, it should is gradually reduced during iteration；Study is obtained by following formula
Rate
L is current iterations；
Input layer data structure is set as 17 dimensional vector X [n], n=17；Each data in vector be normalization with
Afterwards as a result, codomain be [0,1]；Hidden node number is set, considers best hidden unit reference formula
K is sample number, n1For hidden unit number, n is input unit number, if i>n1,
M is output neuron number, the constant that a is 1-10；
n1=lbn
N is input unit number；
Lb is the logarithm bottom of for 2.
Further, the Bluetooth chip is the bluetooth transceiving chip of model HC-05.
Further, the nine axis attitude transducer is the nine axis attitude transducers of model JY-901.
Further, affiliated microcontroller is model STM32F1036 microcontrollers.
The multiple bending sensor is set to the dorsal side of five fingers on gloves.
The nine axis attitude transducer is set at the back of the hand.
The microcontroller is set at the back of the hand.
The periphery analog to digital conversion circuit is set at the back of the hand.
The bluetooth module is set at the back of the hand.
The curvature sensor is for measuring wearer's digital flexion posture information.
The nine axis attitude transducer is for measuring wearer's hand and arm critical movements posture information.
The periphery analog to digital conversion circuit, microcontroller, Bluetooth chip, for data processing and transmission.
The sign language database knows transmission data using neural network.
Description of the drawings
Fig. 1 is the structural schematic diagram of a specific embodiment of the invention.
In figure：1. --- it is 5. five curvature sensors, wherein it is 1. shorter, it is 5.80 centimetres, the variation range of resistance value
It is 9k~22k ohm；2. --- it is 5. longer, it is 10.50 centimetres, the variation range of resistance value is 9k~22k ohm.6. being outer boxing
Number conversion circuit.7. being STM32F103C8T6 microcontrollers.8. being nine axis attitude transducers (JY-901).9. being Bluetooth transmission mould
Block (HC-05).
Fig. 2 is the circuit diagram of STM32F103C8T6 microcontrollers.
Fig. 3 is analog to digital conversion circuit.
Fig. 4 is hardware effort flow.
Fig. 5 is total system work-based logic figure.
Fig. 6 is whole hardware connecting circuit figure.
Specific implementation mode
Below in conjunction with drawings and examples, the technical solution of the present invention is further explained, but content below is not
For limiting protection scope of the present invention.
As shown in Figure 1, the present embodiment provides a kind of wearable sign language tutoring system based on gesture identification gloves, including hand
1. gesture identifies gloves module --- it is 5. five curvature sensors, wherein it is 1. shorter, it is 5.80 centimetres, the variation range of resistance value
It is 9k~22k ohm；2. --- it is 5. longer, it is 10.50 centimetres, the variation range of resistance value is 9k~22k ohm.6. being outer boxing
Number conversion circuit.7. being STM32F103 microcontrollers.8. being nine axis attitude transducers (JY-901).9. being Bluetooth communication modules
(HC-05)；Wherein：1. the gesture identification gloves module is equipped with five curvature sensors --- and 5. and nine axis postures sense
Device (JY-901) is 8. used to acquire wearer's gesture posture and motion track information；9. the periphery analog to digital conversion circuit is used to locate
Manage five curvature sensors 1. --- 5. output voltage values, and 7. carry out data transmission with one-chip computer module；The microcontroller
7. the information for handling the multiple sensor acquisition；7. the bluetooth module (HC-05) 9. carries out data biography with microcontroller
It is defeated, and send the data to host computer.
In the present embodiment, database uses the study and identification that BP networks carry out gesture.Bending sensor and nine axis appearances
State sensor can provide 17 characteristics altogether.We establish feature database in systems, input feature vector data, BP network,
The required gesture meaning identified is identified and understood with trained BP networks.
BP networks are a kind of multilayer feedforward neural networks, and network is made of input layer, hidden layer and output node layer, in structure
Each neuron indicates that hidden layer can be one layer with a node, can also be multilayer, and front layer is to passing through company between rear node layer
Weights are connect to be linked.When BP neural network learns, input signal is transmitted to output layer (forward-propagating) from input layer by hidden layer, if
Output layer obtains desired output, then learning algorithm terminates；Otherwise, backpropagation is gone to.Backpropagation is exactly by error signal
(sample exports and the difference of network output) is adjusted the power of each layer neuron by gradient descent method by former connecting path backwards calculation
Value, makes error signal reduce.
The transmission functions of BP networks require must can be micro-, commonly used has Sigmoid type logarithms.It has as follows
Form：
Its derivative is：
Due to transmission function be everywhere can be micro-, so for BP networks, on the one hand, the region divided is no longer
One linear partition, but the region being made of a non-linear hyperplane, it is smoother curved surface, thus its classification
More more acurrate than linear partition, fault-tolerance is also more preferable than linear partition；On the other hand, network can strictly use gradient descent method into
Row study, the analytic expression of modified weight are also very clear.
In this project, bending sensor and nine axis attitude transducers can provide 17 characteristics altogether.Wherein 5 item datas
For hand (Hand Shape, HS) data, come from 5 bending sensors；12 position hands position (Hand Position,
HP) and direction (Hand Orientation, HO) data, nine axis attitude transducers are come from.We will establish output in systems
The feature database of data thereby determines which kind of gesture is the data of output be.Gesture is divided into two classes by us, and one kind is to focus on hand
Gesture, such as number 1-10, this kind of gesture be most widely used in real life, it is only necessary to which hand is the direction it is understood that hand
Belong to invalid information with position.Another kind of is the gesture in the direction and position that focus on hand, such as some complicated sign languages.Consider
To two class gestures are almost without overlapping in practical applications, during training and identification, first by 17 data of input
It is divided into two parts, a part is HS data, and another part is HP and HO data, and HS numbers need to be only directed to for the gesture of hand emphatically
According to being trained, HS weights are established, the gesture in direction and position for hand emphatically needs in the training process to all 17
A data are trained, and establish HP＆HO weights.During identification, the HS data that HS weights will be used first to identify input are known
Output data is obtained after not, is compared in feature database, it is the gesture for focusing on hand if finding, will be directly defeated
Go out, if being not belonging to, HP＆HO weights is recycled to be identified, with this to simplify identification process, reduces and identify difficulty, quickening is known
Other rate.
The learning training process of BP neural network is divided into following several stages.
(6) hidden layer is set in this project.Connection weight between input layer and hidden layer is set as Wij, by hidden layer and defeated
The connection weight gone out between layer is set as Wjk, each weights W is given at randomijAnd WjkAssign an initial weight, it is desirable that each weights are mutually not
It is equal, and all be a smaller non-zero number, can between (- 1,1) value.
(7) to each sample in sample set, the real output value O of network is determinedp。
(8) reality output O is calculatedpWith corresponding ideal output YpBetween difference.
(9) it presses minimum mode error and adjusts weights.
(10) judge whether maximum iteration is more than a previously given number, do not reach, go to (2).
Wherein (1) (2) are known as propagation stage forward, and (3) (4) are back-propagation stage, the work one in two stages
As should be controlled by required precision, if every group of sample has m data, for pth group sample, if output valve is ypl, practical defeated
It is O to go out valuepjIt can use its sample error
The biasing b of iteration each time can be calculated by sample errorp
α is learning rate in each iteration, it should is gradually reduced during iteration.In this project, against
The training of the gesture of weight hand because data volume is smaller, can be slower by the decline of learning rate, to reach more
Accurately output.The gesture in direction and position for hand emphatically, can be very fast by the decline of learning rate.It therefore can be with
Learning rate is obtained by following formula
I is current iterations.
Input layer data structure is set in this system as 17 dimensional vector X [n], n=17.Each data in vector are
Normalize later as a result, codomain is [0,1].Hidden node number is set, it may be considered that best hidden unit reference formula
K is sample number, n1For hidden unit number, n is input unit number, if i>n1,
M is output neuron number, and a is the constant of 1-10.
n1=lbn
N is input unit number.
Lb is the logarithm bottom of for 2.
In the present embodiment, data analysis uses MLLR algorithms, we can obtain from the data pick-up on gloves
To left hand hand (Left Hand Shape, LHS), left hand position (Left Hand Position, LHP), left hand direction
(Left Hand Orientation, LHO), right hand hand (Right Hand Shape, RHS), right-hand lay (Right
Hand Position, RHP), right-hand direction (Right Hand Orientation, RHO).One data set of transmission is determined
Justice is a sample, such as S1={ LHS1, LHP1, LHO1, S2={ RHS1, RHP1, RHO1}.Usual one section of sign language is by multiple dynamic
It forms, each action can be considered as one " root ".Therefore sign language vocabulary can be decomposed into " word one by one by we
Root " obtains the code-book data of sign language, can greatly reduce the quantity of identification sample in this way.The data of gloves transmission are most evidences
The clock signal of stream, correlation is smaller between different data streams, and correlation is stronger within same data flow, therefore can be by sign language
Data are divided into multiple data flows, and each data flow is the signal that some sensor continuously transmits, and is needed to each sign language
Root trains a continuous HMM, is then clustered to the state of these HMM, after the mean cluster of multiple data stream, no
The similar fragments of two same sign words samples can be divided into same class, can thus be obtained using part sign words data
The code-book data of whole sign words.
By the model library that can obtain sign language of above step, we are referred to as unspecified person model.But it is individual
Sign language data have differences, thus there is also difference to the collected sign language data of same vocabulary, we are referred to as particular person
Data.This when, we needed user to input certain self-adapting data to adjust existing model library.MLLR algorithms utilize
The sufficient unspecified person model of training and a small amount of specific personal data, adjust original model parameter with maximum-likelihood criterion
It is whole so that new model generates the maximum probability of these data.MLLR algorithms are reduced initial by calculating one group of transformation matrix
For model parameter with the difference between self-adapting data, the effect of transformation matrix is can be by the mean value of initial model blending constituent
It is converted so that the model after transformation can generate the probability bigger of self-adapting data.
If W represents the transformation matrix (n represents characteristic dimension) of a n × (n+1), it is mixed to represent initial model with ξ
The extension mean vector of synthesis point, then it is adaptive after mean μ be
μ=W ξ
Wherein ξ=(μ1, μ2, μ3……μn, w)T, ω is constant 1.
Circuit is by a STM32F103 microcontroller, three LM358 chips, a HC-05 bluetooth module, a JY-901
Nine axis attitude transducer modules, five bending sensors and five resistance are constituted.One termination Vin1 of first bending sensor,
One end of 3 pins (anode) and divider resistance r1 of another termination LM358-1；4 pins of another termination LM358-1 of resistance r1
(GND)；1 pin (Vout1) of LM358-1 connects 2 pins (cathode) of LM358-1, and connects the PA1 pipes of STM32F103 microcontrollers
Foot.One termination Vin2 of second bending sensor；8 pins (anode) of another termination LM358-1 and the one of divider resistance r2
End；4 pins (GND) of another termination LM358-1 of resistance r2；6 pins (Vout2) of LM358-1 connect 7 pins of LM358-1
(cathode), and connect the PA2 pins of STM32F103 microcontrollers.One termination Vin3 of third bending sensor, another termination
One end of 3 pins (anode) and divider resistance r3 of LM358-2；4 pins (GND) of another termination LM358-2 of resistance r3；
1 pin (Vout3) of LM358-2 connects 2 pins (cathode) of LM358-2, and connects the PA3 pins of STM32F103 microcontrollers.4th
One termination Vin4 of a bending sensor, one end of 3 pins (anode) and divider resistance r4 of another termination LM358-2；Resistance
4 pins (GND) of another termination LM358-2 of r4；6 pins (Vout4) of LM358-2 connect 7 pins (cathode) of LM358-2,
And connect the PA4 pins of STM32F103 microcontrollers.The one of 5th bending sensor terminates Vin5, another to terminate the 3 of LM358-3
One end of pin (anode) and divider resistance r5；4 pins (GND) of another termination LM358-3 of resistance r5；1 pipe of LM358-3
Foot (Vout5) connects 2 pins (cathode) of LM358-3, and connects the PA5 pins of STM32F103 microcontrollers.HC-05 bluetooth modules
TX pins connect the PA9 pins of STM32F103 microcontrollers；The RX pins of HC-05 bluetooth modules meet the PA10 of STM32F103 microcontrollers
Pin.The SCL pins of nine axis attitude transducer modules of JY-901 connect the PB10 pins of STM32F103 microcontrollers；Nine axis of JY-901
The SDA pins of attitude transducer module connect the PB11 pins of STM32F103 microcontrollers.By STM32F103 microcontrollers, three
LM358 chips, HC-05 bluetooth modules, the VCC pins of nine axis attitude transducer modules of JY-901, Vin1-Vin5 all connect 5V's
DC power supply.
We can obtain left hand hand (Left Hand Shape, LHS), left hand from the data pick-up on gloves
Position (Left Hand Position, LHP), left hand direction (Left Hand Orientation, LHO), right hand hand
(Right Hand Shape, RHS), right-hand lay (Right Hand Position, RHP), right-hand direction (Right Hand
Orientation,RHO).One data set of transmission is defined as a sample, it can be by right-hand man's in traditional data set
Same type data are placed in a sample, but the action gap of two hands is larger in sign language, therefore the data of right-hand man are divided
It not being positioned in two datasets so that this data set is more applicable for sign language interpreter and gesture identification in post-processing,
Such as S1={ LHS1, LHP1, LHO1, S2={ RHS1, RHP1, RHO1}.Usual one section of sign language is made of multiple actions, each
Action can be considered as one " root ".Therefore sign language vocabulary can be decomposed into " root " one by one by we, obtain sign language
Code-book data can greatly reduce the quantity of identification sample in this way.The data of gloves transmission are the clock signals of multiple data stream, no
Smaller with correlation between data flow, correlation is stronger within same data flow, therefore can be divided into sign language data multiple
Data flow, each data flow are the signal that some sensor continuously transmits, and need to train a company to each sign language root
Continuous HMM, then clusters the state of these HMM, after the mean cluster of multiple data stream, different two sign words
The similar fragments of sample can be divided into same class, thus can obtain the code of whole sign words using part sign words data
Notebook data.
By the model library that can obtain sign language of above step, we are referred to as unspecified person model.But it is individual
Sign language data have differences, the movement range when size of each human hand, arm length thickness, body height are fat or thin and talk,
There are difference for position, style, hand-type conversion frequency etc., thus to the collected sign language data of same vocabulary there is also difference, I
Be referred to as specific personal data.This when, we needed user to input certain self-adapting data to adjust existing model
Library.MLLR algorithms are using the sufficient unspecified person model of training and a small amount of specific personal data, with maximum-likelihood criterion to initial
Model parameter is adjusted so that new model generates the maximum probability of these data.MLLR algorithms are by calculating one group of transformation
Matrix reduces original model parameter with the difference between self-adapting data, and the effect of transformation matrix is can be by initial model
The mean value of blending constituent is converted so that the model after transformation can generate the probability bigger of self-adapting data.
If W represents the transformation matrix (n represents characteristic dimension) of a n × (n+1), it is mixed to represent initial model with ξ
The extension mean vector of synthesis point, then it is adaptive after mean μ be
μ=W ξ
Wherein ξ=(μ1, μ2, μ3... ..., μn, ω)T, ω is constant 1.
To obtain the Robust Estimation of transformation matrix, MLLR bundlees mean vector.Share the equal of same transformation matrix
Value set is known as a regression class, and similar regression class continues to bundle, and gradually forms one using regression class as the regression class of node
Tree.Determine which level in regression class tree carries out adaptively according to the quantity of self-adapting data when adaptive：It is less in data
When, it is carried out in the level close to root of regression tree adaptive；When data are more, in the level close to leaf node of regression tree
It carries out adaptive.For a specific transformation matrix Wm, by corresponding R blending constituent { m1, m2... mnShared, lead to
Crossing expectation-maximization algorithm can obtain
WhereinFor t moment mrThe likelihood value of ingredient.By solving this formula, transformation matrix W can be calculatedm, from
And parameter transformation is carried out to unspecified person model.
Claims (4)
1. it is a kind of based on gesture identification gloves can learning data acquisition system, it is characterised in that：Including nine axis attitude transducers,
Curvature sensor, peripheral analog to digital conversion circuit, microcontroller, Bluetooth chip and host computer, curvature sensor be it is multiple, it is multiple
Bending sensor is set on gloves on five fingers；
The curvature sensor is electrically connected with the peripheral analog to digital conversion circuit, the periphery analog to digital conversion circuit with it is described
Microcontroller is electrically connected, and the nine axis attitude transducer is electrically connected with the microcontroller, the microcontroller and the bluetooth core
Piece is electrically connected, and the Bluetooth chip is connect with host computer.
2. system according to claim 1, it is characterised in that：
Including a STM32F103 microcontroller, three LM358 chips, a HC-05 bluetooth module, a nine axis appearance of JY-901
State sensor assembly, five bending sensors and five resistance are constituted；One termination Vin1 of first bending sensor, the other end
Connect one end of 3 pins (anode) and divider resistance r1 of LM358-1；4 pins (GND) of another termination LM358-1 of resistance r1；
1 pin (Vout1) of LM358-1 connects 2 pins (cathode) of LM358-1, and connects the PA1 pins of STM32F103 microcontrollers；Second
One termination Vin2 of a bending sensor；One end of 8 pins (anode) and divider resistance r2 of another termination LM358-1；Resistance
4 pins (GND) of another termination LM358-1 of r2；6 pins (Vout2) of LM358-1 connect 7 pins (cathode) of LM358-1,
And connect the PA2 pins of STM32F103 microcontrollers；The one of third bending sensor terminates Vin3, another to terminate the 3 of LM358-2
One end of pin (anode) and divider resistance r3；4 pins (GND) of another termination LM358-2 of resistance r3；1 pipe of LM358-2
Foot (Vout3) connects 2 pins (cathode) of LM358-2, and connects the PA3 pins of STM32F103 microcontrollers；4th bending sensor
One termination Vin4, it is another termination LM358-2 3 pins (anode) and divider resistance r4 one end；Another termination of resistance r4
4 pins (GND) of LM358-2；6 pins (Vout4) of LM358-2 connect 7 pins (cathode) of LM358-2, and meet STM32F103
The PA4 pins of microcontroller；One termination Vin5 of the 5th bending sensor, 3 pins (anode) of another termination LM358-3 and divides
One end of piezoresistance r5；4 pins (GND) of another termination LM358-3 of resistance r5；1 pin (Vout5) of LM358-3 connects
2 pins (cathode) of LM358-3, and connect the PA5 pins of STM32F103 microcontrollers；The TX pins of HC-05 bluetooth modules connect
The PA9 pins of STM32F103 microcontrollers；The RX pins of HC-05 bluetooth modules connect the PA10 pins of STM32F103 microcontrollers；JY-
The SCL pins of 901 9 axis attitude transducer modules connect the PB10 pins of STM32F103 microcontrollers；Nine axis postures of JY-901 sense
The SDA pins of device module connect the PB11 pins of STM32F103 microcontrollers；By STM32F103 microcontrollers, three LM358 chips,
HC-05 bluetooth modules, the VCC pins of nine axis attitude transducer modules of JY-901, Vin1-Vin5 all connect the DC power supply of 5V；
By STM32F103 microcontrollers, three LM358 chips, HC-05 bluetooth modules, nine axis attitude transducer modules of JY-901, five it is curved
The GND pins of bent sensor all connect holding altogether.
3. the method for the application such as system of claims 1 or 2, it is characterised in that：
Bending sensor and nine axis attitude transducers can provide 17 characteristics altogether；Wherein 5 item datas are hand data, this five
The actual physical meaning of group data is the bending degree of finger, and expression-form is the reflected voltage value of electric resistance partial pressure institute,
Come from 5 bending sensors；The position and direction data of 12 position hands, this 12 groups of data are respectively 3 groups of Euler's angular datas, and three
Group acceleration information, three groups of magnetic fields and three groups of gyroscopes, come from nine axis attitude transducers；The feature database of output data is established,
Thereby determine which kind of gesture is the data of output be；Gesture is divided into two classes, one kind is the gesture for focusing on hand, another kind of to be
The direction of hand and the gesture of position are overweighted, during training and identification, 17 data of input are divided into two parts first,
A part is HS data, and both shap hand-types data, referred to the digital flexion degree representated by bending sensor input resistance data, separately
A part is HP and HO data, i.e. bearing data position and position data orientation, is referred to by nine divisions of China in remote antiquity attitude transducer
Read 12 groups of related datas；17 all data are trained in the training process, establish HP＆HO weights；Knowing
During other, the HS data that HS weights will be used first to identify input obtain output data after identification, are carried out in feature database
Comparison, if finding, it is the gesture for focusing on hand, will directly be exported, if being not belonging to, HP＆HO weights is recycled to be known
Not.
4. method according to claim 3, which is characterized in that wherein training process is the learning training of BP neural network
Process is divided into following several stages：
(1) hidden layer is set；Connection weight between input layer and hidden layer is set as Wij, by the company between hidden layer and output layer
It connects weights and is set as Wjk, each weights W is given at randomijAnd WjkAssigning an initial weight, it is desirable that each weights are not mutually equal, (- 1,
1) value between；
(2) to each sample in sample set, the real output value O of network is determinedp；
(3) reality output O is calculatedpWith corresponding ideal output YpBetween difference；
(4) it presses minimum mode error and adjusts weights；
(5) judge whether maximum iteration is more than a previously given number, do not reach, go to (2)；
Every group of sample has m data, for pth group sample, if output valve is ypj, real output value OpjTake its sample error
The biasing b of iteration each time is calculated by sample errorp
α is learning rate in each iteration, it should is gradually reduced during iteration；Learning rate is obtained by following formula
L is current iterations；
Input layer data structure is set as 17 dimensional vector X [n], n=17；Each data in vector are that normalization is later
As a result, codomain is [0,1]；Hidden node number is set, considers best hidden unit reference formula
K is sample number, n1For hidden unit number, n is input unit number, if i>n1,
M is output neuron number, the constant that a is 1-10；
n1=lbn
N is input unit number；
Lb is the logarithm bottom of for 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810721658.XA CN108664129A (en) | 2018-07-04 | 2018-07-04 | It is a kind of based on gesture identification gloves can learning data acquisition system and implementation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810721658.XA CN108664129A (en) | 2018-07-04 | 2018-07-04 | It is a kind of based on gesture identification gloves can learning data acquisition system and implementation method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108664129A true CN108664129A (en) | 2018-10-16 |
Family
ID=63772722
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810721658.XA Pending CN108664129A (en) | 2018-07-04 | 2018-07-04 | It is a kind of based on gesture identification gloves can learning data acquisition system and implementation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108664129A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110189590A (en) * | 2019-06-18 | 2019-08-30 | 合肥工业大学 | A kind of adaptively correcting formula sign language mutual translation system and method |
| CN111722723A (en) * | 2020-06-29 | 2020-09-29 | 北京化工大学 | Bidirectional bending flexible sensor, sign language recognition system and method |
| CN112380976A (en) * | 2020-11-12 | 2021-02-19 | 华东师范大学 | Gesture recognition system and method based on neural network visual touch sensor fusion |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102972905A (en) * | 2012-11-23 | 2013-03-20 | 南京工业大学 | Tumble alarming shoes |
| CN105759970A (en) * | 2016-03-02 | 2016-07-13 | 华南理工大学 | Gesture recognition device based on bending sensor and sign language interpretation method |
| CN208569551U (en) * | 2018-07-04 | 2019-03-01 | 北京工业大学 | It is a kind of based on gesture identification gloves can learning data acquisition system |
-
2018
- 2018-07-04 CN CN201810721658.XA patent/CN108664129A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102972905A (en) * | 2012-11-23 | 2013-03-20 | 南京工业大学 | Tumble alarming shoes |
| CN105759970A (en) * | 2016-03-02 | 2016-07-13 | 华南理工大学 | Gesture recognition device based on bending sensor and sign language interpretation method |
| CN208569551U (en) * | 2018-07-04 | 2019-03-01 | 北京工业大学 | It is a kind of based on gesture identification gloves can learning data acquisition system |
Non-Patent Citations (1)
| Title |
|---|
| 江立等: "基于神经网络的手势识别技术研究", 《北京交通大学学报》, pages 1 - 5 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110189590A (en) * | 2019-06-18 | 2019-08-30 | 合肥工业大学 | A kind of adaptively correcting formula sign language mutual translation system and method |
| CN111722723A (en) * | 2020-06-29 | 2020-09-29 | 北京化工大学 | Bidirectional bending flexible sensor, sign language recognition system and method |
| CN111722723B (en) * | 2020-06-29 | 2021-07-13 | 北京化工大学 | Bidirectional bending flexible sensor, sign language recognition system and method |
| CN112380976A (en) * | 2020-11-12 | 2021-02-19 | 华东师范大学 | Gesture recognition system and method based on neural network visual touch sensor fusion |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Kudrinko et al. | Wearable sensor-based sign language recognition: A comprehensive review | |
| Lim et al. | Isolated sign language recognition using convolutional neural network hand modelling and hand energy image | |
| Yuan et al. | Hand gesture recognition using deep feature fusion network based on wearable sensors | |
| Dong et al. | Dynamic hand gesture recognition based on signals from specialized data glove and deep learning algorithms | |
| Shin et al. | Dynamic hand gesture recognition for wearable devices with low complexity recurrent neural networks | |
| Hasan et al. | Human computer interaction for vision based hand gesture recognition: a survey | |
| CN110008839B (en) | Intelligent sign language interaction system and method for self-adaptive gesture recognition | |
| CN108664129A (en) | It is a kind of based on gesture identification gloves can learning data acquisition system and implementation method | |
| CN109079794B (en) | Robot control and teaching method based on human body posture following | |
| CN112148128B (en) | Real-time gesture recognition method and device and man-machine interaction system | |
| CN208569551U (en) | It is a kind of based on gesture identification gloves can learning data acquisition system | |
| Diliberti et al. | Real-time gesture recognition using 3D sensory data and a light convolutional neural network | |
| CN107346207B (en) | Dynamic gesture segmentation recognition method based on hidden Markov model | |
| Alam et al. | Trajectory-based air-writing character recognition using convolutional neural network | |
| Liu et al. | Real-time robotic mirrored behavior of facial expressions and head motions based on lightweight networks | |
| Sooai et al. | Comparison of recognition accuracy on dynamic hand gesture using feature selection | |
| CN109857322A (en) | A kind of paintbrush width control method and device based on Android | |
| TWI829944B (en) | Avatar facial expression generating system and method of avatar facial expression generation | |
| US11841920B1 (en) | Machine learning based gesture recognition | |
| WO2020195017A1 (en) | Path recognition method, path recognition device, path recognition program, and path recognition program recording medium | |
| CN111230872B (en) | Object delivery intention recognition system and method based on multiple sensors | |
| Yang et al. | Towards computer-aided sign language recognition technique: A directional review | |
| Bandyopadhyay | a study on Indian sign language recognition using deep learning approach | |
| Zeltzer et al. | A survey of glove-based input | |
| CN111626158B (en) | Face mark point tracking prototype design method based on adaptive descent regression |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |