CN107450715A - A kind of man-machine interaction multifunctional wrist strap terminal based on gesture identification - Google Patents
A kind of man-machine interaction multifunctional wrist strap terminal based on gesture identification Download PDFInfo
- Publication number
- CN107450715A CN107450715A CN201610369667.8A CN201610369667A CN107450715A CN 107450715 A CN107450715 A CN 107450715A CN 201610369667 A CN201610369667 A CN 201610369667A CN 107450715 A CN107450715 A CN 107450715A
- Authority
- CN
- China
- Prior art keywords
- gesture
- man
- wrist strap
- matching
- strap terminal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 210000000707 wrist Anatomy 0.000 title claims abstract description 21
- 230000003993 interaction Effects 0.000 title claims abstract description 17
- 238000000034 method Methods 0.000 claims abstract description 26
- 230000008569 process Effects 0.000 claims abstract description 19
- 230000011218 segmentation Effects 0.000 claims abstract description 19
- 230000002452 interceptive effect Effects 0.000 claims abstract description 12
- 230000009466 transformation Effects 0.000 claims description 5
- 238000013145 classification model Methods 0.000 claims 3
- 238000004519 manufacturing process Methods 0.000 claims 1
- 238000012545 processing Methods 0.000 abstract description 4
- 230000007547 defect Effects 0.000 abstract 1
- 238000005516 engineering process Methods 0.000 description 6
- 238000011161 development Methods 0.000 description 3
- 230000018109 developmental process Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 238000012067 mathematical method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000026676 system process Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Social Psychology (AREA)
- Psychiatry (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The present invention relates to the field of human-computer interaction of wearable design, is based particularly on the man-machine interaction multifunctional wrist strap terminal of gesture identification, the wrist strap terminal is including dressing man-machine interactive system and gesture recognition process device.Wearing man-machine interactive system includes monocular cam and wearable operating system, and it is responsible for the collection of video flowing of the collection comprising gesture information and transmitting video-frequency flow data to gesture recognition processing module；Gesture recognition process device includes Hand Gesture Segmentation module and gesture classification and matching module, and it carries out Hand Gesture Segmentation, gesture analysis, gesture classification and matching to the gesture information in video flowing, finally obtains recognition result, returns to system and instruct accordingly.The implementation of the invention efficiently utilizes existing shooting collecting device in wrist strap terminal, without extra hardware spending, using efficient Gesture Recognition Algorithm, improves Consumer's Experience, the defects of overcoming the interaction design inconvenience such as wearable device screen is small.
Description
Technical field
The present invention relates to wearable technology field, particularly a kind of multifunctional wrist strap terminal based on gesture identification.
Background technology
Wearable device is the development trend of Intelligent hardware of future generation, and wearable device has certain computing function, can
The existing Intelligent mobile equipment such as mobile phone and flat board is connected, the product of main flow includes the bracelet using wrist as support and wrist-watch system
Row, and the eyes series using head as support, wearable device just occurs in the form of more enriching, but development is most at present
The equipment of main flow be still wrist strap series product, according to the statistics of current major hardware vendor, wrist product be it is wearable go out
Goods amount highest.The research contents of wearable device includes, I/O Interface, the calculating of continuous sensing data, Activity recognition, environment
Context-aware etc., or even the research also comprising material and energy technology.Wearable device relate in substantial amounts of correlative technology field
Hold, be the content that multidisciplinary field intersects.
With the development of wearable device, wearable wrist product has endurance poor, it is impossible to independent use or work(
Can not entirely, the shortcomings of interactive screen is small.The problems such as screen of wearable wrist, has had a strong impact on the usage experience of user, but
Interaction aspect, if screen removed, Consumer's Experience can be very poor, it is impossible to means that product function can not be complete with product direct interaction
Entirety is existing, has so had a strong impact on that the data exchange between user and wearable device produces obstacle, then wearable device is just
User can not be served, if a wearable device does not have software service, smart machine cannot be referred to as.
In order to solve the problems, such as above section, it is necessary to using a kind of more reasonable and advanced man-machine interaction mode, based on wearing
The human-computer interaction technology of vision is worn, also known as wearable vision interacts.It is achieved in that on wearable device outfit shooting is first-class
Imaging device, capture static state in interaction scenarios, multidate information at any time by visually-perceptible technology, fully understanding that context believes
On the basis of breath, the intention of user is fully understood, so as to complete nature, efficient, reliable interactive task.Using monocular vision
Gesture identification method carry out operation and control to wearable device.
The content of the invention
The present invention is directed to the drawbacks described above of prior art, it is proposed that a kind of multi-functional wrist of man-machine interaction based on gesture identification
Tape terminal.The system includes wearing man-machine interactive system and gesture recognition process device；Wherein, described wearing man-machine interaction system
System, including monocular cam and wearable operating system, it is responsible for the video for including gesture video of terminal monocular cam collection
Stream, and give video stream to gesture recognition process device, the analysis and identification of gesture are carried out, finally sends the finger of specific function
Order；The gesture recognition process device, including Hand Gesture Segmentation module and gesture classification and matching module, according to described by transmission
Video flowing is handled by specific algorithm, by the processing of each processing module in gesture recognition process device, finally
The specific function representated by the gesture of user is identified, sending function command to wrist terminal carries out specific action response.
Wherein, the wearing man-machine interactive system and the gesture recognition process device in wrist strap terminal by being mounted in wrist strap terminal
On operating system carry out data interaction.
Further, it is described wearing man-machine interactive system include wrist strap terminal on monocular cam and wrist strap terminal in itself
The wearable operating system carried；Wherein, monocular cam is responsible for the collection of video, is by operating by the video data of collection
System passes to gesture recognition process module and carries out the operation such as segmentation, analysis, matching of gesture.Wherein, the segmentation of the gesture is adopted
It is the algorithm that Adaptive Gauss model is combined with MLP, the process directly influences the accuracy and precision of identification.The hand
Gesture recognition process unit includes the geometrical normalization of images of gestures, removes arm interference module, also gesture matching module.
Further, in described gesture recognition process module, the Shape context based on interior distance has been used
Gesture matching algorithm, calculate the characteristic vector in gesture profile point first in the algorithm, then found according to the characteristics of hand
Optimum sampling point is matched, and finds the principal direction of palm center, then carries out corresponding polar coordinate transform, by polar
Direction rotated to and coincided with arm regions principal direction beginning, is classified using MLP algorithms, is finally reached identification rapidly and efficiently
Gesture.
Brief description of the drawings
The present invention is further detailed explanation with reference to the accompanying drawings and detailed description.
Fig. 1 is the entire block diagram of the present invention.
In figure：Including wearing man-machine interactive system, gesture recognition process device.
Fig. 2 is the system process chart of the present invention.
In figure：Including Hand Gesture Segmentation module and gesture classification and matching module.
Fig. 3 is the flow chart of Hand Gesture Segmentation module.
Fig. 4 is MLP iterative process figures.
In figure：Using the EM algorithms of classics.
Fig. 5 is Gesture Recognition Algorithm flow chart.
In figure：Contain Euclidean transformation, the matching algorithm based on interior distance.
Embodiment
As shown in figure 1, the present invention is the man-machine interaction multifunctional wrist strap terminal based on gesture identification, wearing man-machine interaction system
System includes monocular cam and wearable operating system, wherein wearing man-machine interactive system is responsible for collection regarding comprising gesture information
Frequency flow data, and pass to gesture recognition process device progress gesture identification, gesture recognition process dress by dressing operating system
Put including Hand Gesture Segmentation module and gesture classification and matching module, be responsible for the identifying processing of gesture.
Specific system handling process is as shown in Fig. 2 wherein the video comprising gesture is imaged by the monocular in interactive system
Head collection.After camera collects video flowing, Hand Gesture Segmentation, Hand Gesture Segmentation such as Fig. 3 are carried out for each two field picture of video
It is shown.The video of input is split according to fixed threshold value first, single Gauss model then is established to each cut zone,
Then variance is reduced with iterative method, obtains the likelihood figure of regional output, the segmentation of threshold value is fixed again, with reference to fixation
The model that Threshold segmentation, single Gauss model are combined with gauss hybrid models, and the Gauss model with adaptivity, are solved
Can not linear partition the problem of.For hand region, using geometry plavini and connected region is removed；For background area
Domain, influence of the background to recognition accuracy is weakened using geometry etch.
When the change of background is less than the threshold value set, iteration convergence, Hand Gesture Segmentation terminates；Conversely, carry out MLP instructions
Practice, until result restrains.Hand Gesture Segmentation it is not convergent in the case of, carry out MLP repetitive exercise such as Fig. 4, for background change
Data, estimate the initial value of Gauss model parameter first, variance reduced according to EM algorithms, while also reduces the region of segmentation,
Meet the threshold value that originally set until variance is sufficiently small, otherwise just iteration always.
As shown in figure 5, after splitting by image, after the profile for obtaining image, by Euclidean transformation, wherein
Range formula in Euclidean transformation is：
ED=min (Di)
Try to achieve centre of the palm position.Palm is assumed for ellipse from human morphology, and major and minor axis is known and then calculates oval
Parameter, its formula are as follows：
Spalm=max (EDxy)
Lpaim=1.5max (EDxy)
Palm and arm are removed by calculating elliptic parameter, then use the algorithmic match gesture based on interior distance, it is interior
The algorithm of distance mainly has two steps：Initially set up the connection figure of sampled point.For configuration sampling point set P={ pi, i=1,2,
3...n }, if pi、pjConnecting line falls in profile, then connecting line is drawn in connection figure.Pay attention to, sample connection figure here and may be used also
To show out the hole information in profile；Then the algorithm of shortest path planning is used sampled point connection figure, finally obtains knowledge
Other result.Influence of the mode of interior distance metric point distance for non-rigid object swivel of hand and part-structure deformation is smaller, more
Suitable for gesture identification.If accelerating the process of matching using finger fingertip as sampled point can in addition, answering for algorithm is reduced
Miscellaneous to spend the problem of high, finger fingertip can be obtained by following mathematical method, with OcCentered on calculate hand profile and OcAway from
From curve f (x) is obtained, f (x) can be converted into by least square method by continuous function f (x)=a0+a1x+...anxn's
Form, then tries to achieve the position of extreme point, and then identifies the position of finger tip.
Wherein ε=0.5Lpalm。
Although the foregoing describing the embodiment of the present invention, those skilled in the art should be appreciated that this
Be merely illustrative of, various changes or modifications can be made to present embodiment, without departing from the present invention principle and essence,
Protection scope of the present invention is only limited by the claims that follow.
Claims (5)
- A kind of 1. man-machine interaction multifunctional wrist strap terminal based on gesture identification, it is characterised in that：The human-computer interaction terminal includes Wearing man-machine interactive system and gesture recognition process device.
- 2. according to the wearing man-machine interactive system of the multifunctional wrist strap terminal based on gesture identification described in claim 1, it is special Sign is：Including monocular cam and wearable operating system, the video flowing containing gesture information is carried out using monocular cam, selected Selecting most popular equipment reduces the manufacturing cost of hardware, while on the premise of basic acquisition hardware determines, carries specific Wearable operating system, it is responsible for sending the video information of collection to gesture recognition process device.
- 3. according to the gesture recognition process device of the multifunctional wrist strap terminal based on gesture identification described in claim 1, it is special Sign is：Including Hand Gesture Segmentation module and gesture classification and matching module；Wherein Hand Gesture Segmentation module uses the side based on region segmentation Method and MLP training methods, and by establishing single Gauss model to regional and then reducing error by iterative method；Wherein, hand Gesture classification includes the gesture classification model of the context based on shape with matching module and the matching algorithm based on interior distance is adopted The matching of gesture is carried out with the matching algorithm based on interior distance, finally gives matching result.
- 4. MLP training methods in Hand Gesture Segmentation module according to claim 3, it is characterised in that：By the parameter of EM algorithm iterations progressively improved model, the likelihood probability of parameter and training sample is set gradually to increase, finally A maximal point is terminated at, EM algorithms is employed and reduces variance, improve the accuracy of identification.
- 5. the gesture classification model of the context according to claim 3 based on shape, it is characterised in that：The gesture classification model of context employs oval palm model and carries out image modeling, including Euclidean transformation, Ask for centre of the palm position and calculate three steps of elliptic parameter, palm is reduced to ellipse, is obtained and slapped by Euclidean transformation Heart position, then palm area is positioned by calculating oval major and minor axis, then remove the matching and knowledge that arm regions carry out gesture Not.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610369667.8A CN107450715A (en) | 2016-05-31 | 2016-05-31 | A kind of man-machine interaction multifunctional wrist strap terminal based on gesture identification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610369667.8A CN107450715A (en) | 2016-05-31 | 2016-05-31 | A kind of man-machine interaction multifunctional wrist strap terminal based on gesture identification |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107450715A true CN107450715A (en) | 2017-12-08 |
Family
ID=60485646
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610369667.8A Pending CN107450715A (en) | 2016-05-31 | 2016-05-31 | A kind of man-machine interaction multifunctional wrist strap terminal based on gesture identification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107450715A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108814572A (en) * | 2018-05-28 | 2018-11-16 | Oppo广东移动通信有限公司 | Wearing state detection method and relevant device |
| CN111338461A (en) * | 2018-12-18 | 2020-06-26 | 鸿合科技股份有限公司 | Gesture operation method and device and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104268514A (en) * | 2014-09-17 | 2015-01-07 | 西安交通大学 | Gesture detection method based on multi-feature fusion |
| CN104298354A (en) * | 2014-10-11 | 2015-01-21 | 河海大学 | Man-machine interaction gesture recognition method |
| CN204191757U (en) * | 2014-09-24 | 2015-03-11 | 南京航空航天大学 | A kind of gesture catches bracelet |
| CN104503582A (en) * | 2014-12-29 | 2015-04-08 | 北京智谷睿拓技术服务有限公司 | Wearable-equipment-based interaction method and interaction device, and wearable equipment |
| CN105354551A (en) * | 2015-11-03 | 2016-02-24 | 北京英梅吉科技有限公司 | Gesture recognition method based on monocular camera |
| CN105487673A (en) * | 2016-01-04 | 2016-04-13 | 京东方科技集团股份有限公司 | Man-machine interactive system, method and device |
-
2016
- 2016-05-31 CN CN201610369667.8A patent/CN107450715A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104268514A (en) * | 2014-09-17 | 2015-01-07 | 西安交通大学 | Gesture detection method based on multi-feature fusion |
| CN204191757U (en) * | 2014-09-24 | 2015-03-11 | 南京航空航天大学 | A kind of gesture catches bracelet |
| CN104298354A (en) * | 2014-10-11 | 2015-01-21 | 河海大学 | Man-machine interaction gesture recognition method |
| CN104503582A (en) * | 2014-12-29 | 2015-04-08 | 北京智谷睿拓技术服务有限公司 | Wearable-equipment-based interaction method and interaction device, and wearable equipment |
| CN105354551A (en) * | 2015-11-03 | 2016-02-24 | 北京英梅吉科技有限公司 | Gesture recognition method based on monocular camera |
| CN105487673A (en) * | 2016-01-04 | 2016-04-13 | 京东方科技集团股份有限公司 | Man-machine interactive system, method and device |
Non-Patent Citations (1)
| Title |
|---|
| 陈泽宁: "基于穿戴视觉的人机交互技术", 《中国知网硕士论文库》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108814572A (en) * | 2018-05-28 | 2018-11-16 | Oppo广东移动通信有限公司 | Wearing state detection method and relevant device |
| CN111338461A (en) * | 2018-12-18 | 2020-06-26 | 鸿合科技股份有限公司 | Gesture operation method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Joint activity recognition and indoor localization with WiFi fingerprints | |
| CN107765855A (en) | A kind of method and system based on gesture identification control machine people motion | |
| CN103984928B (en) | Finger gesture recognition methods based on depth image | |
| CN107908288A (en) | A kind of quick human motion recognition method towards human-computer interaction | |
| Chen et al. | Rapid recognition of dynamic hand gestures using leap motion | |
| CN111985265A (en) | Image processing method and device | |
| Qi et al. | Computer vision-based hand gesture recognition for human-robot interaction: a review | |
| CN110471526A (en) | A kind of human body attitude estimates the unmanned aerial vehicle (UAV) control method in conjunction with gesture identification | |
| CN104407694A (en) | Man-machine interaction method and device combining human face and gesture control | |
| CN104777775A (en) | Two-wheeled self-balancing robot control method based on Kinect device | |
| CN109800676A (en) | Gesture identification method and system based on depth information | |
| US20220262093A1 (en) | Object detection method and system, and non-transitory computer-readable medium | |
| CN109032337A (en) | A kind of KEM Gesture Recognition Algorithm based on data glove | |
| CN111209812A (en) | Target face picture extraction method and device and terminal equipment | |
| WO2023061037A1 (en) | Air gesture recognition method and related device thereof | |
| CN106814853A (en) | A kind of eye control tracking based on machine learning | |
| CN112036261A (en) | Gesture recognition method and device, storage medium and electronic device | |
| CN110807391A (en) | Human body posture instruction identification method for human-unmanned aerial vehicle interaction based on vision | |
| CN110569775A (en) | Method, system, storage medium and electronic device for recognizing human body posture | |
| Xu et al. | Robust hand gesture recognition based on RGB-D Data for natural human–computer interaction | |
| CN110866468A (en) | Gesture recognition system and method based on passive RFID | |
| Hu et al. | Trajectory image based dynamic gesture recognition with convolutional neural networks | |
| CN107450715A (en) | A kind of man-machine interaction multifunctional wrist strap terminal based on gesture identification | |
| CN111695408A (en) | Intelligent gesture information recognition system and method and information data processing terminal | |
| Li et al. | Hand gesture tracking and recognition based human-computer interaction system and its applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171208 |
|
| RJ01 | Rejection of invention patent application after publication |