CN110110674B - Gesture recognition method based on acceleration micro-electromechanical system and basic strokes - Google Patents
Gesture recognition method based on acceleration micro-electromechanical system and basic strokes Download PDFInfo
- Publication number
- CN110110674B CN110110674B CN201910391987.7A CN201910391987A CN110110674B CN 110110674 B CN110110674 B CN 110110674B CN 201910391987 A CN201910391987 A CN 201910391987A CN 110110674 B CN110110674 B CN 110110674B
- Authority
- CN
- China
- Prior art keywords
- gesture
- code
- time
- information
- standard
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000001133 acceleration Effects 0.000 title claims abstract description 40
- 230000008569 process Effects 0.000 claims abstract description 17
- 230000033001 locomotion Effects 0.000 claims abstract description 15
- 230000011218 segmentation Effects 0.000 claims abstract description 6
- 230000003068 static effect Effects 0.000 claims description 28
- 238000011156 evaluation Methods 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 15
- 238000006243 chemical reaction Methods 0.000 claims description 14
- 230000008859 change Effects 0.000 claims description 9
- 150000001875 compounds Chemical class 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 7
- 230000009467 reduction Effects 0.000 claims description 7
- 238000012549 training Methods 0.000 claims description 5
- 238000006073 displacement reaction Methods 0.000 claims description 3
- 230000035945 sensitivity Effects 0.000 claims description 3
- 238000012706 support-vector machine Methods 0.000 claims description 3
- 230000007704 transition Effects 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 6
- 238000013519 translation Methods 0.000 description 5
- 230000003993 interaction Effects 0.000 description 3
- 230000001276 controlling effect Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000010030 laminating Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/30—Writer recognition; Reading and verifying signatures
- G06V40/37—Writer recognition; Reading and verifying signatures based only on signature signals such as velocity or pressure, e.g. dynamic signature recognition
- G06V40/382—Preprocessing; Feature extraction
- G06V40/388—Sampling; Contour coding; Stroke extraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/30—Writer recognition; Reading and verifying signatures
- G06V40/37—Writer recognition; Reading and verifying signatures based only on signature signals such as velocity or pressure, e.g. dynamic signature recognition
- G06V40/394—Matching; Classification
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Health & Medical Sciences (AREA)
- Character Discrimination (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a gesture recognition method based on an acceleration micro-electromechanical system and basic strokes, which comprises the following steps: converting the gesture into a sequence formed by labels according to the motion track of the gesture, and establishing a code recognition model based on code characteristics; setting a standard target gesture and a corresponding element code thereof; acquiring a gesture signal, sending the gesture signal into a code recognition model to obtain a label sequence, and obtaining a basic stroke of a gesture by utilizing a multi-time decision algorithm; and pre-classifying the meaning of the gesture to obtain the similarity between the gesture and the element codes with the same corner code feature number, and taking the gesture corresponding to the element code with the maximum similarity as a gesture recognition result. The invention describes the relationship among the concepts of strokes, element codes, code characteristics and the like of gestures, forms a complete set of gesture recognition method based on the code characteristics, realizes automatic segmentation of the gestures in the gesture recognition process through corner code characteristics, and has simple algorithm and high recognition precision.
Description
Technical Field
The invention relates to a gesture recognition method, in particular to a gesture recognition method based on an acceleration micro-electromechanical system and basic strokes.
Background
The gesture recognition is a process for recognizing human body gestures, and has wide application prospects in the fields of medical monitoring, network teaching, electronic equipment and the like. Moreover, the gesture is a natural, visual and convenient human-computer interaction mode, the human-computer interaction process is greatly simplified, and meanwhile, the development of the gesture information collector is promoted due to the diversification of the human-computer interaction mode. At present, a gesture information collector for gesture recognition research is mainly divided into: 1) An image pickup apparatus; the gesture recognition system based on the camera shooting equipment is used for acquiring images of hands and the surrounding environment and recognizing gestures contained in the images. 2) Wearing equipment, it is through built-in or integrated mode, with collector laminating, fix at the hand, gather the motion or the biological information of hand, accomplish the discernment of gesture. 3) The novel information acquisition mode, the information of this type of collection include WIFI, radar etc. is present latest gesture recognition method.
The application context of gesture recognition directly determines the selection of the gesture collector. For example, the placement of the camera equipment can seriously affect the user experience, and meanwhile, the requirement of real-time tracking is difficult to meet; gesture recognition methods based on novel information acquisition modes are not mature, and recognized gestures are generally simple; when single-object tracking gesture recognition is carried out, the requirements cannot be met by the single-object tracking gesture recognition.
Disclosure of Invention
In order to solve the technical problems, the invention provides a gesture recognition method based on an acceleration micro-electromechanical system and basic strokes, which has the advantages of simple algorithm and high recognition precision.
The technical scheme for solving the problems is as follows: a gesture recognition method based on an acceleration micro-electro-mechanical system and basic strokes comprises the following steps:
the method comprises the following steps: training a recognition model of the standard code to realize one-to-one correspondence of the gesture track and the standard code; and converting the gesture into a standard code sequence;
step two: determining a sample gesture and a corresponding element code thereof;
step three: obtaining basic strokes of the gestures by utilizing a multi-time decision algorithm;
step four: firstly, pre-classifying gesture meanings according to the number of basic strokes; and then, calculating the similarity between the gesture to be detected and the sample element codes with the same basic stroke number, and taking the sample gesture corresponding to the element code with the maximum similarity as a gesture recognition result.
The gesture recognition method based on the acceleration micro-electromechanical system and the basic strokes comprises the following specific steps:
1-1) according to the actual track of the gesture, classifying the gesture into 4 types and labeling as: corner, static, arc, straight; the corner represents a gesture track transition period, the theoretical duration is 0 and is marked as 4, and the segmentation of the gesture and the starting and ending of the gesture are embodied; the static state represents that the gesture is in a static state, and the time period corresponding to the time period that the hand before and after the gesture is motionless is marked as 3, so that the gesture recognition system can conveniently recognize the start and the end of the gesture; the arc represents a time period of the gesture track in an arc shape, and is marked as 2, which is a basis for distinguishing the gesture by the gesture recognition system; the straight line represents a time period when the gesture track is straight, and is marked as 1, which is a basis for distinguishing the gesture by the gesture recognition system; the gestures are equally divided into the four types of labels at each moment, and each acquired gesture is converted into a sequence formed by the four types of labels; theoretically, labels of gestures at different moments are obtained according to motion tracks, so that the conversion from a gesture sequence to a label sequence is completed, the manner is one-to-one corresponding like encoding, therefore, classified labels are marked as standard codes, and correspondingly, the gesture information at each moment is classified and labeled, and is marked as standard code identification;
1-2) calling the original information of each moment as moment information, and calling the corresponding standard code information of each moment as standard code information; in the standard code identification process, in order to obtain standard code information, by referring to the corresponding relationship between the speed and the motion curve, the time information of each time and the next N times is subjected to multiple conversions, so as to obtain conversion information, wherein the conversion information includes an Angle (Angle) and a variance (Var), and the conversion information includes:
in the formula (I), the compound is shown in the specification,
representing angle information between the time t and the time of the interval kT, wherein k is an integer; t is the period of information acquisition, and T =0.02s; a. The t Is the acceleration at time t; a. The t+kT Is the acceleration at time t + kT; pi is a constant;
Angle t set of angle information representing time t, elements thereinA pixel represents angle information between time t and time spaced kT (k =1,2 … N);
in the formula (I), the compound is shown in the specification,
representing variance information between time t and the time of interval kT;
Var t a set of variance information representing time t, the elements of which represent variance information between time t and times spaced kT (k =1,2 … N);
when the device is in a standard static state at a certain moment, the device has no speed and acceleration, so that the values of angle and variance information are stable around 0, and when the device is in a standard corner state, the change of speed is large, so that the change of acceleration information is obvious, so that the mean value and variance value of the angle and variance information are large; in this theory, by setting a threshold, two types of code features, namely static code features and corner code features, are identified;
at time t, taking into account N time intervals, a set of information about feature F is obtained
For the feature F, the mean value and the variance of the set measure the acceleration and speed change at the time t, and considering that under the condition of different features, L, N and thresholds, L is a corner range, the feature with the highest evaluation value, L, N and the threshold are selected to construct a filter function:
the filter function of the "corner" standard code is:
VAR(F t )/E(F t )≥ε+λv
in the formula, E (F) t ) Is a set (F) t ) Mean value of, VAR (F) t ) Is a set (F) t ) In a squareThe difference, ε is a constant; lambda is the speed sensitivity coefficient; v is the gesture velocity;
the filter function for the "static" standard code is:
wherein, alpha and beta are constants;
in order to identify the standard codes of 'corner' and 'straight line' by using filter function, the standard codes are selected
And
selecting the feature with the highest evaluation value, L, N and a threshold as evaluation parameters;
for the time information which can not be identified by using the angle and the variance, performing secondary classification by using a multi-Support Vector Machine (SVM) to obtain the membership degree of the linear or arc code characteristics;
when the SVM is used for the binary classification, the key point is that a sample is mapped, an optimal hyperplane B is obtained through the training of a statistical learning method, the binary classification of data is completed, and when the optimal hyperplane is trained, constraint conditions and an objective function are as follows:
wherein w is a hyperplane normal vector; b is a hyperplane displacement term; c is a punishment factor for controlling the error of edge classification, the larger C is, the fewer the number of the misclassified sample points is, and delta j Is a non-negative relaxation variable, nu is the number of data points to be classified; when standard code recognition is performed using SVM, x j Time at j point, y i The value of (b) represents the standard code of the jth point, and the value is n, wherein n =1,2;
when using SVM for the binary classification, the set of elements having the same standard code n is denoted as L n (ii) a The element set with n standard codes obtained by classification of the SVM classifier is S n The classification is carried out by utilizing SVM, and the classification result is the reliability delta of the standard code n n :
δ n For measuring the reliability of SVM classification, it means that for the moment of standard code result of classification "n", delta exists n Has a probability standard code of n and 1-delta n Has a probability of 3-n.
In the above gesture recognition method based on the acceleration micro-electromechanical system and the basic stroke, in the second step, the standard target gesture is a number from 0 to 9, and the corresponding element code is: the element code corresponding to 0 is (3, 4, 1, 4, 3), the element code corresponding to 1 is (3, 4, 1, 4, 3), the element code corresponding to 2 is (3, 4, 2, 4, 1, 4, 3), the element code corresponding to 3 is (3, 4, 2, 4, 3), the element code corresponding to 4 is (3, 4, 1, 4, 3), the element code corresponding to 5 is (3, 4, 1, 4, 2, 4, 3), the element code corresponding to 6 is (3, 4, 2, 4, 3), the element code corresponding to 7 is (3, 4, 1, 4, 3), the element code corresponding to 8 is (3, 4, 2, 4, 1, 4, 3), and the element code corresponding to 9 is (3, 4, 2, 4, 1, 4, 3).
In the above gesture recognition method based on the acceleration micro-electro-mechanical system and the basic stroke, in the third step, the recognition of the basic stroke of the gesture includes the following steps:
i) deleting the moment when all code features are static in the label sequence;
II) assigning code characteristics of l moments on two sides of the corner code characteristics as corners to prevent a single corner time period from being identified as the corners of a plurality of segments;
III) reducing the characteristics of the corner codes, and replacing all continuous corners with one corner to obtain a reduction sequence;
IV) carrying out multi-time decision on the sequence between the adjacent corner code features: the reference template is shown as
Re denotes two corner code features and its middle length M standard code sequence segments, where,
when the expression is identified by the code characteristics, the membership degree of the moment t belonging to the straight line is obtained by utilizing the SVM, and the membership degree of the moment t belonging to the arc is
And obtaining the membership degree of the stroke class in the time period as a straight line through multi-time decision:
where i represents the sequence number of the stroke in the gesture sequence,
representing the membership of the ith stroke of the gesture belonging to a straight line,
and (3) representing the membership degree of the straight line at the moment t, wherein the value of t is 1-M.
In the fourth step, after multi-time decision, data in the gesture sequence consists of corners and straight line membership degrees, wherein the higher the straight line membership degree is, the higher the probability that the time period belongs to the straight line stroke is, otherwise, the higher the probability of the arc stroke is; in this case, the decision is made using the number of corner code features, to pre-classify the meaning of the gesture,
is a reduction of the gesture sequence in which,
denotes the ith 1 Each stroke belongs to the membership degree of the 'straight line' code characteristic;
a sample gesture meta-code is represented, wherein,
denotes the ith 2 The standard code of each stroke comprises the following values: 1 (straight line) and 2 (arc); i all right angle 1 、i 2 Is the number of strokes in the gesture if equation i is satisfied 1 ＝i 2 Then the similarity between the two sequences is as follows:
in the formula (I), the compound is shown in the specification,
representing the similarity between the gesture R and the element code G,
representing the membership degree of the ith stroke belonging to the straight line; forming a set taking into account a number of sample gesture element codes
If sample g satisfies the equation:
and then, the gesture corresponding to the sample g is the gesture recognition result of the gesture to be detected.
The invention has the beneficial effects that: firstly, according to the motion track of the gesture, the gesture is converted into a sequence formed by labels, and a code recognition model based on code characteristics is established; then setting a standard target gesture and a corresponding element code thereof; acquiring a gesture signal, sending the gesture signal into the coding recognition model in the first step to obtain a label sequence, and obtaining a basic stroke of the gesture by using a multi-time decision algorithm; and finally, presorting the meanings of the gestures to obtain the similarity between the gestures and the element codes with the same corner code feature number, and taking the gesture corresponding to the element code with the maximum similarity as a gesture recognition result. The whole method describes the relationship among the concepts of strokes, element codes, code characteristics and the like of gestures, forms a complete set of gesture recognition method based on the code characteristics, realizes automatic segmentation of the gestures in the gesture recognition process through corner code characteristics, and has the advantages of simple algorithm and high recognition precision.
Drawings
FIG. 1 is a flow chart of the present invention.
FIG. 2 is a flow chart of code feature recognition in the code recognition model of the present invention.
FIG. 3 is a flow chart of SVM classification of the present invention.
FIG. 4 is a decision diagram of the present invention utilizing the number of corner code features.
Fig. 5 is a block diagram of a gesture capture system according to an embodiment of the present invention.
FIG. 6 is a diagram illustrating a gesture trajectory according to an embodiment of the present invention.
FIG. 7 is a waveform of an original acceleration in an embodiment of the present invention.
FIG. 8 is a graph of normalized acceleration waveforms in an embodiment of the present invention.
FIG. 9 is a waveform diagram of a code signature according to an embodiment of the present invention.
Detailed Description
The invention is further described below with reference to the figures and examples.
As shown in FIG. 1, a gesture recognition method based on an acceleration micro-electro-mechanical system and a basic stroke comprises the following steps:
the method comprises the following steps: and according to the motion track of the gesture, converting the gesture into a sequence formed by labels, and establishing a code recognition model based on code characteristics. The method comprises the following specific steps:
1-1) infinitesimal quantity, is a concept in calculus, i.e. a variable bounded by a number 0 and infinitely close to 0. In the process of calculus, in order to obtain the area of the curve, the area is generally subjected to segmented summation, and the area of the curve is obtained as the final segment length approaches infinity. However, in the calculus calculation process, the result obtained by the integration has high precision as long as the segment length value of the variable is small to a certain degree. By taking the thought as reference, the speed is the area of a curve formed by an acceleration curve and a time axis, and in the gesture recognition process, the information acquisition frequency is ensured to be high enough, so that the acceleration at any moment can be utilized to directly reflect the moment speed information. Of course, the gesture recognition cannot be performed by the speed information at a single time, and the gesture at the time can be classified by combining the speed information at the time and the speed information at the next n (n > 1) time to obtain the type of the gesture at the time.
Therefore, based on the actual trajectory of the gesture, the gesture is classified into 4 categories and labeled: corner, static, arc, straight; the corner represents a gesture track transition period, the theoretical duration is 0 and is marked as 4, and the segmentation of the gesture and the starting and ending of the gesture are embodied; the static state represents that the gesture is in a static state, and the time periods corresponding to the static state of the hands before and after the gesture are recorded as 3, so that the gesture recognition system can conveniently recognize the start and the end of the gesture; the arc represents a time period of the gesture track in an arc shape, and is marked as 2, which is a basis for distinguishing the gesture by the gesture recognition system; the straight line represents a time period when the gesture track is straight, and is marked as 1, which is a basis for distinguishing the gesture by the gesture recognition system; the gestures are equally divided into the four types of labels at each moment, and each acquired gesture is converted into a sequence formed by the four types of labels; theoretically, labels of gestures at different moments are obtained according to motion tracks, so that the conversion from a gesture sequence to a label sequence is completed, the mode is one-to-one corresponding like encoding, therefore, classified labels are marked as code features, and correspondingly, the gesture information at each moment is classified and marked as code feature identification;
TABLE 1
1-2) is a point on the time axis, which means that at a certain moment there is neither a magnitude nor a direction. The purpose of code feature identification is to obtain the corresponding code feature at each moment. In the process of code feature identification, for convenience of description, the original information of each moment is called time information, and the corresponding code feature information of each moment is called code feature information. As shown in fig. 2, in the code feature identification process, in order to obtain code feature information, the time information of each time and N subsequent times is subjected to multiple conversions by using the correspondence between the speed and the motion curve, so as to obtain conversion information, where the conversion information includes an Angle (Angle) and a variance (Var), where:
in the formula (I), the compound is shown in the specification,
representing angle information between the time t and the time of the interval kT, wherein k is an integer; t is the period of information acquisition, and T =0.02s; a. The t Is the acceleration at time t; a. The t+kT Is the acceleration at time t + kT; pi is a constant;
Angle t a set of angle information representing time t, the elements of which represent angle information between time t and times of interval kT (k =1,2 … N);
in the formula (I), the compound is shown in the specification,
representing variance information between time t and the time of interval kT;
Var t a set of variance information representing time t, the elements of which represent variance information between time t and times of interval kT (k =1,2 … N).
Theoretically, when the device is in a standard static state at a certain moment, the device has no speed and acceleration, so that the values of angle and variance information are stable around 0, and when the device is in a standard corner state, the change of speed is large, so that the change of acceleration information is obvious, so that the mean value and the variance value of the angle and variance information are large; in this theory, both stationary and corner code features are identified by setting a threshold.
At time t, taking into account N time intervals, a set of information about feature F is obtained
For the feature F, the mean value and the variance of the set measure the acceleration and speed change at the time t, and considering that under the condition of different features, L, N and thresholds, L is a corner range, the feature with the highest evaluation value, L, N and the threshold are selected to construct a filter function:
the filter function of the "corner" standard code is:
VAR(F t )/E(F t )≥ε+λv
in the formula, E (F) t ) Is a set (F) t ) Mean value of, VAR (F) t ) Is a set (F) t ) Epsilon is a constant; lambda is a speed sensitivity coefficient; v is the gesture velocity;
the filter function for the "static" standard code is:
wherein α and β are constants.
In order to identify the standard codes of 'corner' and 'straight line' by using filter function, the standard codes are selected
And
selecting the feature with the highest evaluation value, L, N and a threshold as evaluation parameters;
For the time information which can not be identified by using the angle and the variance, performing secondary classification by using a multi-Support Vector Machine (SVM), and obtaining the membership degree of the linear or arc code characteristics as shown in figure 3;
when the SVM is used for the binary classification, the key point is that a sample is mapped, an optimal hyperplane B is obtained through the training of a statistical learning method, the binary classification of data is completed, and when the optimal hyperplane is trained, constraint conditions and an objective function are as follows:
wherein w is a hyperplane normal vector; b is a hyperplane displacement term; c is a punishment factor for controlling the error of edge classification, the larger C is, the fewer the number of the misclassified sample points is, and delta j Is a non-negative relaxation variable, nu is the number of data points to be classified; when standard code recognition is performed using SVM, x j Time at j point, y i The value of (d) represents the standard code of the j point, and the value is n, n =1,2;
when using SVM for the binary classification, the set of elements having the same standard code n is denoted as L n (ii) a The element set with n standard codes obtained by classification of the SVM classifier is S n Classifying by using SVM, the classification result is the reliability delta of the standard code n n :
δ n For measuring the reliability of SVM classification, it means that for the time when the standard code result of classification is "n", delta exists n Has a probability standard code of n and 1-delta n Has a probability of 3-n.
Step two: and setting a standard target gesture and a corresponding element code thereof.
The meta code of the gesture refers to a code feature sequence with the minimum length and the minimum simplification obtained according to the type of a track curve of a standard track of the gesture at each moment. Obviously, the element code of the gesture also accords with the cognition of people for the concept of 'stroke'. Therefore, the element code of the gesture, which can also be said to be the basic stroke of a standard gesture, does not follow the personal habits of the writer. The gesture meta-code is obtained through the standard track of the gesture, and the two processes of translation and simplification are included.
The translation of the gesture meta-code refers to a process of translating a standard gesture track into a code feature sequence according to the type of the gesture at each moment. The translation of gesture meta-codes is similar to conventional code feature recognition. However, the translation of the gesture element code does not require a specific algorithm or program to recognize the sequence, which is based on the knowledge of people about standard gesture trajectories.
The purpose of the reduction of the gesture meta-code is to eliminate the influence of the writing length of the gesture and pay attention to the change of the curve type of the gesture. For example, a gesture with a standard trajectory of a straight line has a gesture element code of a straight line no matter how long the gesture is, and no matter how many times the translated code features the straight line. In theory, for the same gesture, the writer can see different standard gesture trajectories under different writing habits of the writer. However, as long as the personal habits of the writers do not affect the correct recognition of the code features, i.e. straight line segments, arc segments and corner segments are not mixed up, they all have the same gesture meta-code. Meanwhile, the standard gesture track does not affect the result of the gesture element code after operations such as rotation, translation, zooming and the like. Therefore, the gesture recognition method based on the element code characteristics can effectively avoid the influence of personal writing habits on gesture recognition.
As shown in table 2, the standard target gesture is set from 0 to 9, and the corresponding element code is: the element code corresponding to 0 is (3, 4, 1, 4, 3), the element code corresponding to 1 is (3, 4, 1, 4, 3), the element code corresponding to 2 is (3, 4, 2, 4, 1, 4, 3), the element code corresponding to 3 is (3, 4, 2, 4, 3), the element code corresponding to 4 is (3, 4, 1, 4, 3), the element code corresponding to 5 is (3, 4, 1, 4, 2, 4, 3), the element code corresponding to 6 is (3, 4, 2, 4, 3), the element code corresponding to 7 is (3, 4, 1, 4, 3), the element code corresponding to 8 is (3, 4, 2, 4, 1, 4, 3), and the element code corresponding to 9 is (3, 4, 2, 4, 1, 4, 3). Different gestures may have the same element code, such as gestures 3 and 6; at this time, the meta-code durations are additionally considered, for example, gestures 3 and 6 both have two "arc" standard codes, the two arc durations of gesture 3 are approximately equal, and the duration of the previous arc of gesture 6 is longer than that of the next arc.
TABLE 2
| 0 | P、C、L、C、L、C、L、C、L、C、P |
| 1 | P、C、L、C、P |
| 2 | P、C、A、C、L、C、L、C、P |
| 3 | P、C、A、C、A、C、P |
| 4 | P、C、L、C、L、C、L、C、P |
| 5 | P、C、L、C、L、C、A、C、P |
| 6 | P、C、A、C、A、C、P |
| 7 | P、C、L、C、L、C、P |
| 8 | P、C、A、C、L、C、A、C、L、C、P |
| 9 | P、C、A、C、L、C、P |
Step three: acquiring a gesture signal, sending the gesture signal into the coding recognition model in the first step to obtain a label sequence, and obtaining a basic stroke of the gesture by using a multi-time decision algorithm.
The gesture meaning can be under certain background condition, and the identification of obtaining the basic stroke according to the stroke of the gesture comprises the following steps:
i) Deleting the moment when all code features are static in the label sequence; in the gesture recognition process, the static moment segment corresponds to a gesture useless information segment, and has no practical significance for gesture recognition; however, the start and end times of the gesture may be detected by static code features.
II) assigning code features of 1 moment on two sides of the corner code feature as a corner to prevent a single corner time period from being identified as the corner of a plurality of segments; the corner code characteristics correspond to the current moment, the gesture category is changing or the start and the end of a useful segment of the gesture are started and ended; the segmentation of the gesture is embodied, and the gesture sequence is divided into two different types, namely front and back. The value of L can influence the recognition precision of the overall code characteristics and the recognition rate of the corner.
III) reducing the characteristics of the corner codes, and replacing all continuous corners with one corner to obtain a reduction sequence;
IV) making a multi-time decision on the sequence between adjacent corner code features: the reference template sequence is represented as
Denoted F are two corner code features and their middle length M standard code sequence segments, where,
when the representation is identified by the code characteristics, the SVM is utilized to obtain the membership degree of the moment t belonging to the straight line, and the membership degree of the moment t belonging to the arc is
And obtaining the membership degree of the stroke class in the time period as a straight line through multi-time decision:
wherein i is the serial number of the stroke in the gesture sequence,
representing the degree of membership of the ith stroke belonging to a straight line,
the membership degree of the straight line at the moment t is represented, and the value of t is1-M。
Step four: and pre-classifying the meanings of the gestures to obtain the similarity between the gestures and the element codes with the same corner code feature number, and taking the gesture corresponding to the element code with the maximum similarity as a gesture recognition result.
After the multi-time decision, the data in the gesture sequence consists of corner and straight line membership, wherein the higher the straight line membership, the higher the probability that the time period belongs to a straight line stroke, otherwise, the higher the probability of an arc stroke; in this case, the number of corner code features is used for decision making to pre-classify the meaning of the gesture, and the decision diagram is shown in fig. 4.
Is a reduction of the gesture sequence in which,
denotes the ith 1 Each stroke belongs to the membership degree of the 'straight line' code characteristic;
a sample gesture meta-code is represented, wherein,
denotes the ith 2 The standard code of each stroke comprises the following values: 1 (straight line) and 2 (arc); i.e. i 1 、i 2 Is the number of strokes in the gesture if equation i is satisfied 1 ＝i 2 The similarity between the two sequences is as follows:
in the formula (I), the compound is shown in the specification,
representing the similarity between the gesture R and the element code G,
representing the membership degree of the ith stroke belonging to the straight line; forming a set taking into account a number of sample gesture element codes
If sample g satisfies the equation:
and then, the gesture corresponding to the sample g is the gesture recognition result of the gesture to be detected.
Examples
The information acquisition system used by the invention is divided into two types of wired information acquisition and wireless information acquisition, the two types of information acquisition fully meet the requirement on portability, and the hardware connection of the two types of information acquisition is shown in figure 5. In the experimental process, a wired information acquisition system is selected to verify the accuracy of the gesture recognition method.
The sensor used in the invention is a nine-axis gyroscope attitude sensor module JY901 researched and developed by Shenzhen Weite Intelligent science and technology Limited. When the gesture is collected, the sensor and the ring-shaped fixture are integrated and worn on the finger. The size of the device is 15.24mm X15.24mm X2mm, which effectively ensures the user experience. JY901 provides four different range accelerations +/-2 g, +/-4 g, +/-8 g and +/-16 g, the working voltage is 3.3V-5V, the acceleration stability is 0.01g, and the attitude measurement stability is 0.01 degrees. The output frequency adjusting range is 0.1 Hz-200 Hz, and the serial port speed class selects 2400,4800,9600,19200,38400,57600,115200,230400,460800,921600HZ.
In the invention, the collected three-dimensional acceleration data is output and transmitted to a computer. The output frequency is selected to be 50Hz, namely the acquisition period is 0.02s. The acceleration measurement was determined to be ± 2g, based on the measured acceleration maxima for our movements.
The meaning and label of the gesture are the key points of the traditional gesture recognition method. The gesture recognition method provided by the invention not only can obtain the meaning and the label of the gesture, but also can obtain the gesture curve information of the gesture at different moments. In order to simultaneously verify the accuracy of gesture curve information identification and gesture meaning, gesture trajectory motion is executed by the robot. And moreover, the movement track and the length of each gesture are regulated, so that the reference value of the gesture curve information is conveniently obtained. When the machine executes gesture movement, the movement speed is 8m/min, and the tracks of the gestures are all simplified into the most common straight lines and arcs, and the corresponding tracks and the parameters of the tracks of the gestures are shown in fig. 6.
In the gesture data acquisition process, the start and stop of acquisition of each gesture are controlled by a switch on the sensor attached software. Each gesture is associated with the following requirements: 1) In the gesture collection process, hand information is kept in a static state between the time when the collector is opened and the time when the collector is closed after the collector executes useful gestures, and the time is more than 4 seconds, so that the starting time and the ending time of the algorithm recognition gestures are ensured. 2) Only one useful gesture exists in each piece of collected gesture data.
In this experiment, the data set contained 10 gestures (0-9) and two chinese character gestures, each performed 50 times by the robot according to a fixed trajectory for a total of 600 gesture samples. The results of some of the experiments of the method on the above data sets are shown in fig. 7-9, fig. 7 is a graph of the original acceleration waveform, and the standard code in each time period is calibrated by human. Fig. 8 is a waveform diagram of the original acceleration after normalization, and it can be seen that there is a certain separability between different standard codes. Fig. 9 is a code signature graph, where the time at which the standard code value k =4 corresponds to the standard code being "corner"; κ =3 corresponds to the time when the standard code is "static"; a time at which the normalized code value κ < 1 indicates that the "straight line" at that time has a degree of membership of κ and the corresponding "arc" has a degree of membership of 1- κ. . Where table 3 is a confusion matrix of gesture recognition accuracy.
TABLE 3
| |
0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
| 0 | ||||||||||
| 1 | ||||||||||
| 2 | ||||||||||
| 3 | ||||||||||
| 4 | ||||||||||
| 5 | ||||||||||
| 6 | ||||||||||
| 7 | ||||||||||
| 8 | ||||||||||
| 9 |
In order to verify the reliability of the obtained gesture curve information when performing gesture recognition by using the text method, the present invention proposes the following data for evaluation. Theoretical Total gesture Duration (TTD), calculated Total gesture Duration (CTD), calculated code feature Duration (c d) for X, which is four types of code features (CDX, X may be L, A, C, P), duration of different curves in the trajectory, theoretical Duration for X (TDX). The closer the CTD/TTD value is to 1, the closer the total gesture duration obtained by the method is to the theoretical gesture duration, and the more reliable the method is. For straight and curved lines, the closer the CDL/TDL and CDA/TDA are to 1, indicating that the method is more reliable. For corner features, the closer the CDC is to 0, indicating that the method is more reliable. For static code features, they are not evaluated because they actually represent the gesture as being static, and because of the provision of the gesture requirements, static code features are theoretically in useless segments of the gesture.
Claims (5)
1. A gesture recognition method based on an acceleration micro-electro-mechanical system and basic strokes comprises the following steps:
the method comprises the following steps: training a recognition model of the standard code to realize one-to-one correspondence of the gesture track and the standard code; and converting the gesture into a standard code sequence;
step two: determining a sample gesture and a corresponding element code thereof;
step three: obtaining basic strokes of the gestures by utilizing a multi-time decision algorithm;
step four: firstly, presorting gesture meanings according to the number of basic strokes; then, calculating the similarity between the gesture to be detected and the sample element codes with the same basic stroke number, and taking the sample gesture corresponding to the element code with the maximum similarity as a gesture recognition result.
2. The acceleration-based microelectromechanical system and the method for gesture recognition of a base stroke as defined in claim 1, wherein the step one comprises the steps of:
1-1) according to the actual track of the gesture, classifying the gesture into 4 types and labeling as follows: corner, static, arc, straight; the corner represents a gesture track transition period, the theoretical duration is 0 and is marked as 4, and the segmentation of the gesture and the starting and ending of the gesture are embodied; the static state represents that the gesture is in a static state, and the time period corresponding to the time period that the hand before and after the gesture is motionless is marked as 3, so that the gesture recognition system can conveniently recognize the start and the end of the gesture; the arc represents a time period of the gesture track in an arc shape, and is marked as 2, which is a basis for distinguishing the gesture by the gesture recognition system; the straight line represents a time period when the gesture track is straight, and is marked as 1, which is a basis for distinguishing the gesture by the gesture recognition system; the gestures are equally divided into the four types of labels at each moment, and each acquired gesture is converted into a sequence formed by the four types of labels; theoretically, labels of gestures at different moments are obtained according to motion tracks, so that the conversion from a gesture sequence to a label sequence is completed, the manner is one-to-one corresponding like encoding, therefore, classified labels are marked as standard codes, and correspondingly, the gesture information at each moment is classified and labeled, and is marked as standard code identification;
1-2) calling the original information of each moment as moment information, and calling the corresponding standard code information of each moment as standard code information; in the standard code identification process, in order to obtain standard code information, by referring to the corresponding relationship between the speed and the motion curve, the time information of each time and the next N times is subjected to multiple conversions, so as to obtain conversion information, wherein the conversion information includes an Angle (Angle) and a variance (Var), and the conversion information includes:
in the formula (I), the compound is shown in the specification,
representing angle information between the time t and the time of the interval kT, wherein k is an integer; t is the period of information acquisition, and T =0.02s; a. The t Is the acceleration at time t; a. The t+kT Is the acceleration at time t + kT; pi is a constant;
Angle t a set of angle information representing time t, the elements of which represent angle information between time t and times of interval kT (k =1,2 … N);
in the formula (I), the compound is shown in the specification,
between the time t and the time of interval kTVariance information of (2);
Var t a set of variance information representing time t, the elements of which represent variance information between time t and times spaced kT (k =1,2 … N);
when the device is in a standard static state at a certain moment, the device has no speed and acceleration, so that the values of angle and variance information are stable around 0, and when the device is in a standard corner state, the change of speed is large, so that the change of acceleration information is obvious, so that the mean value and variance value of the angle and variance information are large; in this theory, by setting a threshold, two types of code features, namely static code features and corner code features, are identified;
at time t, taking into account N time intervals, a set of information F is obtained about the feature F t ＝(F t 1 ，F t 2 ，…，F t N ) For the feature F, the mean and variance measures the acceleration and speed changes at time t, and considering that L is a corner range under the condition of different features L, N and thresholds, the feature with the highest evaluation value, L, N and the threshold are selected to construct a filter function:
the filter function of the "corner" standard code is:
VAR(Ft)/E(Ft)≥ε+λv
in the formula, E (F) t ) Is a set (F) t ) Mean value of, VAR (F) t ) Is a set (F) t ) Epsilon is a constant; lambda is a speed sensitivity coefficient; v is the gesture velocity;
the filter function for the "static" standard code is:
wherein, alpha and beta are constants;
in order to identify the standard codes of 'corner' and 'straight line' by using filter function, the standard codes are selected
And
selecting the feature with the highest evaluation value, L, N and a threshold as evaluation parameters;
for the time information which can not be identified by using the angle and the variance, performing secondary classification by using a multi-Support Vector Machine (SVM) to obtain the membership degree of the linear or arc code characteristics;
when the SVM is used for the binary classification, the key point is that a sample is mapped, an optimal hyperplane B is obtained through the training of a statistical learning method, the binary classification of data is completed, and when the optimal hyperplane is trained, constraint conditions and an objective function are as follows:
wherein w is a hyperplane normal vector; b is a hyperplane displacement term; c is a punishment factor for controlling the error of edge classification, the larger C is, the fewer the number of the misclassified sample points is, and delta j Is a non-negative relaxation variable, nu is the number of data points to be classified; when standard code recognition is performed using SVM, x j Time at j point, y i The value of (d) represents the standard code of the j point, and the value is n, n =1,2;
when using SVM for the dichotomy, the set of elements with the same standard code n is denoted as L n (ii) a The element set with n standard codes obtained by classification of the SVM classifier is S n The classification is carried out by utilizing SVM, and the classification result is the reliability delta of the standard code n n :
δ n For measuring the reliability of SVM classification, it means that for the moment of standard code result of classification "n", delta exists n Has a probability standard code of n and 1-delta n Has a probability of 3-n.
3. The acceleration-based microelectromechanical system and the basic stroke gesture recognition method of claim 2, characterized in that in the second step, the standard target gesture is a number from 0 to 9, and the corresponding element code is: the element code corresponding to 0 is (3, 4, 1, 4, 3), the element code corresponding to 1 is (3, 4, 1, 4, 3), the element code corresponding to 2 is (3, 4, 2, 4, 1, 4, 3), the element code corresponding to 3 is (3, 4, 2, 4, 3), the element code corresponding to 4 is (3, 4, 1, 4, 3), the element code corresponding to 5 is (3, 4, 1, 4, 2, 4, 3), the element code corresponding to 6 is (3, 4, 2, 4, 3), the element code corresponding to 7 is (3, 4, 1, 4, 3), the element code corresponding to 8 is (3, 4, 2, 4, 1, 4, 3), and the element code corresponding to 9 is (3, 4, 2, 4, 1, 4, 3).
4. The method of claim 3, wherein the step three, the recognition of the basic stroke of the gesture comprises the steps of:
i) deleting the moment when all code features are static in the label sequence;
II) assigning code characteristics of l moments on two sides of the corner code characteristics as corners to prevent a single corner time period from being identified as the corners of a plurality of segments;
III) reducing the characteristics of the corner codes, and replacing all continuous corners with one corner to obtain a reduction sequence;
IV) carrying out multi-time decision on the sequence between the adjacent corner code features: the reference template is represented as
Re denotes two corner code features and its middle length M standard code sequence segments, where,
when the expression is identified by the code characteristics, the membership degree of the moment t belonging to the straight line is obtained by utilizing the SVM, and the membership degree of the moment t belonging to the arc is
And obtaining the membership degree of the stroke class in the time period as a straight line through multi-time decision:
where i represents the serial number of the stroke in the gesture sequence,
representing the membership of the ith stroke of the gesture belonging to a straight line,
and (3) representing the membership degree of the straight line at the moment t, wherein the value of t is 1-M.
5. The acceleration-based microelectromechanical system and base stroke gesture recognition method of claim 3, characterized in that in step four, after a multi-time decision, the data in the gesture sequence consists of corner and straight line membership, wherein the higher the straight line membership, the higher the probability that the time period belongs to a straight line stroke, otherwise, the higher the probability of an arc stroke; in this case, the decision is made using the number of corner code features, to pre-classify the meaning of the gesture,
is a reduction of the gesture sequence in which,
denotes the ith 1 Each stroke belongs to the membership degree of the characteristic of the straight line code;
a sample gesture meta-code is represented, wherein,
denotes the ith 2 The standard code of each stroke comprises the following values: 1 (straight line) and 2 (arc); i.e. i 1 、i 2 Is the number of strokes in the gesture if equation i is satisfied 1 ＝i 2 Then the similarity between the two sequences is as follows:
in the formula (I), the compound is shown in the specification,
representing the similarity between the gesture R and the element code G,
representing the membership degree of the ith stroke belonging to the straight line; forming a set taking into account a number of sample gesture element codes
If sample g satisfies the equation:
and then, the gesture corresponding to the sample g is the gesture recognition result of the gesture to be detected.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910391987.7A CN110110674B (en) | 2019-05-13 | 2019-05-13 | Gesture recognition method based on acceleration micro-electromechanical system and basic strokes |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910391987.7A CN110110674B (en) | 2019-05-13 | 2019-05-13 | Gesture recognition method based on acceleration micro-electromechanical system and basic strokes |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110110674A CN110110674A (en) | 2019-08-09 |
| CN110110674B true CN110110674B (en) | 2022-12-13 |
Family
ID=67489581
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910391987.7A Active CN110110674B (en) | 2019-05-13 | 2019-05-13 | Gesture recognition method based on acceleration micro-electromechanical system and basic strokes |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110110674B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103257711A (en) * | 2013-05-24 | 2013-08-21 | 河南科技大学 | Space gesture input method |
| CN103679213A (en) * | 2013-12-13 | 2014-03-26 | 电子科技大学 | 3D gesture recognition method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107633227B (en) * | 2017-09-15 | 2020-04-28 | 华中科技大学 | CSI-based fine-grained gesture recognition method and system |
-
2019
- 2019-05-13 CN CN201910391987.7A patent/CN110110674B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103257711A (en) * | 2013-05-24 | 2013-08-21 | 河南科技大学 | Space gesture input method |
| CN103679213A (en) * | 2013-12-13 | 2014-03-26 | 电子科技大学 | 3D gesture recognition method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110110674A (en) | 2019-08-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109325454B (en) | Static gesture real-time recognition method based on YOLOv3 | |
| Frolova et al. | Most probable longest common subsequence for recognition of gesture character input | |
| Várkonyi-Kóczy et al. | Human–computer interaction for smart environment applications using fuzzy hand posture and gesture models | |
| Wu et al. | View-independent recognition of hand postures | |
| CN110575663B (en) | Physical education auxiliary training method based on artificial intelligence | |
| CN116226691B (en) | Intelligent finger ring data processing method for gesture sensing | |
| CN108703824B (en) | Bionic hand control system and control method based on myoelectricity bracelet | |
| Wang et al. | American Sign Language Recognition Using Multi-dimensional Hidden Markov Models. | |
| CN112148128A (en) | Real-time gesture recognition method and device and man-machine interaction system | |
| CN111199202B (en) | Human body action recognition method and recognition device based on circulating attention network | |
| CN113435335B (en) | Microscopic expression recognition method and device, electronic equipment and storage medium | |
| Angona et al. | Automated Bangla sign language translation system for alphabets by means of MobileNet | |
| CN111368762A (en) | Robot gesture recognition method based on improved K-means clustering algorithm | |
| CN108830223A (en) | A kind of micro- expression recognition method based on batch mode Active Learning | |
| Debard et al. | Learning to recognize touch gestures: Recurrent vs. convolutional features and dynamic sampling | |
| CN113946218A (en) | Activity recognition on a device | |
| CN110008847B (en) | Swimming stroke identification method based on convolutional neural network | |
| CN114384999B (en) | User-independent myoelectric gesture recognition system based on self-adaptive learning | |
| CN116127065A (en) | Simple and easy-to-use incremental learning text classification method and system | |
| Madapana et al. | A semantical & analytical approach for zero shot gesture learning | |
| CN107346207A (en) | A kind of dynamic gesture cutting recognition methods based on HMM | |
| CN110110674B (en) | Gesture recognition method based on acceleration micro-electromechanical system and basic strokes | |
| CN116400831B (en) | Handwriting pressure induction compensation algorithm of passive electromagnetic touch screen | |
| CN105046193B (en) | A kind of human motion recognition method based on fusion rarefaction representation matrix | |
| Tormási et al. | Comparing the efficiency of a fuzzy single-stroke character recognizer with various parameter values |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20190809 Assignee: SUZHOU MEDCOIL HEALTHCARE Co.,Ltd. Assignor: XIANGTAN University Contract record no.: X2024980006092 Denomination of invention: A gesture recognition method based on acceleration microelectromechanical system and basic strokes Granted publication date: 20221213 License type: Common License Record date: 20240523 |