CN107015636A - The aobvious equipment gestural control method of virtual reality - Google Patents
The aobvious equipment gestural control method of virtual reality Download PDFInfo
- Publication number
- CN107015636A CN107015636A CN201610958045.9A CN201610958045A CN107015636A CN 107015636 A CN107015636 A CN 107015636A CN 201610958045 A CN201610958045 A CN 201610958045A CN 107015636 A CN107015636 A CN 107015636A
- Authority
- CN
- China
- Prior art keywords
- hand
- mouse
- control method
- image
- gestural control
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Multimedia (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The present invention relates to a kind of aobvious equipment gestural control method of virtual reality, including：Step S11, the identification of hand is carried out by video acquisition device, and judge whether into mouse following state according to the static time in pickup area；Step S12, into after mouse following state, hand exercise is gathered by video acquisition device frame by frame, and corresponding movement is done according to mouse following hand in the positional information control display system of hand in continuous each two field picture；Step S13, in mouse following state, is identified to judge the change of hand gesture by the hand collected video acquisition device in image, realizes the operation clicked on and confirmed.The present invention realizes the action of mouse in the aobvious equipment of virtual reality by hand motion, has broken away from hardware auxiliary input device.
Description
Technical field
The present invention relates to field of human-computer interaction, and in particular to a kind of aobvious equipment gestural control method of virtual reality.
Background technology
The aobvious equipment of virtual reality is the head mounted display of a new generation, and such as virtual implementing helmet is heavy there is provided one kind
The man-machine interaction mode of immersion virtual reality, it generates the virtual of three dimensions using computer in head mounted display
Simulation of the world there is provided user on sense organs such as vision, the sense of hearing, tactiles, allows user as being personally on the scene, immerses user
Into virtual environment, the things in three dimensions is observed in time, without limitation, is a kind of man-machine friendship for Consumer's Experience of more fitting
Mutual mode.
At present, the man-machine interaction of the aobvious equipment of existing virtual reality is typically normal using mouse, Bluetooth handle etc.
Advise input mode to interact with equipment, cost is high and is inconvenient to carry.
The content of the invention
In order to solve above mentioned problem of the prior art, the present invention proposes a kind of aobvious equipment gesture control side of virtual reality
Method, the action of mouse in the aobvious equipment of virtual reality is realized by hand motion, hardware auxiliary input device has been broken away from.
The aobvious equipment gestural control method of a kind of virtual reality proposed by the present invention, comprises the following steps：
Step S11, the identification of hand is carried out by video acquisition device, and is sentenced according to the static time in pickup area
It is disconnected whether to enter mouse following state；
Step S12, into after mouse following state, hand exercise is gathered by video acquisition device, according to continuous frame by frame
Mouse following hand does corresponding movement in the positional information control display system of hand in each two field picture；
Step S13, in mouse following state, is identified by collecting the hand in image to video acquisition device
To judge the change of hand gesture, the operation clicked on and confirmed is realized.
It is preferred that, the strong classifier that the identification to hand in acquired image is built using AdaBoost algorithms is carried out.
It is preferred that, use AdaBoost algorithms build strong classifier method for：
Step S21, builds the training sample (x identified with positive and negative samples1,y1)、(x2,y2)……(xn,yn), wherein
yi=0 is negative sample, yi=1 is positive sample, and n is the total number of training sample；
Step S22, initializes the weight w of training samplet,i, t is cycle-index；
Step S23, the T that takes t=1 ..., T for training maximum cycle；
Step S231, normalizes weights：
Step S232, to each feature f, trains a grader h (xi, f, p, θ), that is, equation is estimated on qt's
Residual error：
εf=∑iqi|h(xi,f,p,θ)-yi|
Wherein, p is the extreme value direction of inequality, and θ is predetermined threshold value；
Step S233, the minimum grader h of selection residual errort
minF, p, θ∑iqi|h(xi, f, p, θ) and-yi|=∑iqi|h(xi, ft, pt, θt)-yi|
ht(x)=h (x, ft, pt, θ t)
Step S234, updates weights
Wherein, εtFor the grader h selected in step S233tCorresponding residual error.
If sample xiCorrectly classified then ei=0, otherwise ei=1；
Step S24, is finally combined into strong classifier：
Wherein：
It is preferred that, the hand gesture recognized in this method includes clenching fist and non-two kinds of gestures of clenching fist, non-gesture use of clenching fist
Mouse following hand does corresponding movement in control display system, and gesture of clenching fist is used to realize that the click of mouse in display system is true
Recognize operation.
It is preferred that, gesture of clenching fist is used to realize that the click of mouse in display system to confirm that the method for operation is：Hand is by non-
State change of clenching fist is state of clenching fist, and is moved forward, and mobile distance is less than the distance threshold of setting, translational speed and is less than
The threshold speed of setting, the then click for performing mouse in display system confirms behaviour.
It is preferred that, the identification to hand in acquired image needs to carry out image the pretreatment of image segmentation, will scheme
Hand as in is opened with background segment.
It is preferred that, the method for the pretreatment of described image segmentation is to be entered based on the difference between skin color and background colour
Row region recognition.
It is preferred that, the method for the pretreatment of described image segmentation is the image partition method based on profile.
Hand is identified by video acquisition device by the present invention, the positional information according to hand in continuous each two field picture
Mouse following hand does corresponding movement in control display system, and by judging that the change of hand gesture carries out the behaviour of click confirmation
Make, realize the control to the aobvious equipment of virtual reality by hand, broken away from hardware auxiliary input device, greatly facilitated use
The use at family, improves operating experience.
Brief description of the drawings
Fig. 1 is the schematic flow sheet of the aobvious equipment gestural control method of virtual reality of the present invention.
Embodiment
The preferred embodiment of the present invention described with reference to the accompanying drawings.It will be apparent to a skilled person that this
A little embodiments are used only for explaining the technical principle of the present invention, it is not intended that limit the scope of the invention.
The present invention is designed for the aobvious equipment of virtual reality at this stage, it is adaptable to video acquisition device (such as
Mobile phone camera or the aobvious equipment external camera of virtual reality or respective sensor) the aobvious equipment of virtual reality, such as VR
Mirror or virtual implementing helmet.When needing to be operated in the scene in the aobvious device display screen of virtual reality, pass through camera
Intelligent Recognition hand simultaneously simulates a virtual mouse arrow (or similar virtual signage) in VR virtual scenes, passes through identification
Go out the movement locus of hand, the operation of selection confirmation is carried out in VR virtual scenes.Choose and confirmation operation, move now with mobile
The dynamic movement locus chosen by recognizing hand, confirms that behaviour, to click on forward, by calculating the difference of far and near distance, is determined as
Confirm operation.
The aobvious equipment gestural control method of a kind of virtual reality proposed by the present invention, as shown in figure 1, comprising the following steps：
Step S11, the identification of hand is carried out by video acquisition device, and is sentenced according to the static time in pickup area
It is disconnected whether to enter mouse following state；
Step S12, into after mouse following state, hand exercise is gathered by video acquisition device, according to continuous frame by frame
Mouse following hand does corresponding movement in the positional information control display system of hand in each two field picture；
Step S13, in mouse following state, is identified by collecting the hand in image to video acquisition device
To judge the change of hand gesture, the operation clicked on and confirmed is realized.
The hand gesture recognized in the present embodiment includes clenching fist and non-two kinds of gestures of clenching fist, and non-gesture of clenching fist is used to control
Mouse following hand does corresponding movement in display system, and gesture of clenching fist is used to realize that the click of mouse in display system to confirm behaviour
Make.Gesture of clenching fist is used to realize that the click of mouse in display system to confirm that the method for operation is：Hand is by non-state change of clenching fist
For the state of clenching fist, and move forward, and mobile distance is less than the speed threshold that the distance threshold of setting, translational speed are less than setting
Value, the then click for performing mouse in display system confirms behaviour.Non- state of clenching fist is in open configuration at least one finger.Hold
Fist and the discrimination of non-state of clenching fist are very high, are conducive to fast and accurately controlling to judge.
The identification of hand in acquired image is needed to carry out image the pretreatment of image segmentation in the present embodiment, will
Hand in image is opened with background segment.The method of the pretreatment of image segmentation is to be based between skin color and background colour
Difference carry out region recognition, it would however also be possible to employ the image partition method based on profile.
The strong classifier that the identification of hand in acquired image is built using AdaBoost algorithms is entered in the present embodiment
OK.Use AdaBoost algorithms build strong classifier method for：
Step S21, builds the training sample (x identified with positive and negative samples1,y1)、(x2,y2)……(xn,yn), wherein
yi=0 is negative sample, yi=1 is positive sample, and n is the total number of training sample；
Step S22, initializes the weight w of training sampleT, i；T is cycle-index；
Step S23, the T that takes t=1 ..., T for training maximum cycle；
Step S231, normalizes weights, shown in such as formula (1)：
Step S232, to each feature f, trains a grader h (xi, f, p, θ), that is, equation is estimated on qt's
Shown in residual error, such as formula (2)：
εf=∑iqi|h(xi, f, p, θ) and-yi| (2)
Wherein, p is the extreme value direction of inequality, and θ is predetermined threshold value；
Step S233, the minimum grader ht of selection residual error, shown in such as formula (3), (4)
minF, p, θ∑iqi|h(xi,f,p,θ)-yi|=∑iqi|h(xi,ft,pt,θt)-yi| (3)
ht(x)=h (x, ft,pt,θt) (4)
Step S234, updates weights, shown in such as formula (5), (6)
Wherein, εtFor the grader h selected in step S233t(x) corresponding residual error.
If sample xiCorrectly classified then ei=0, otherwise ei=1；
Step S24, is finally combined into strong classifier, shown in such as formula (7), (8)
Wherein：
Those skilled in the art should be able to recognize that, the side of each example described with reference to the embodiments described herein
Method step, can be realized with electronic hardware, computer software or the combination of the two, in order to clearly demonstrate electronic hardware and
The interchangeability of software, generally describes the composition and step of each example according to function in the above description.These
Function is performed with electronic hardware or software mode actually, depending on the application-specific and design constraint of technical scheme.
Those skilled in the art can realize described function to each specific application using distinct methods, but this reality
Now it is not considered that beyond the scope of this invention.
So far, combined preferred embodiment shown in the drawings describes technical scheme, still, this area
Technical staff is it is easily understood that protection scope of the present invention is expressly not limited to these embodiments.Without departing from this
On the premise of the principle of invention, those skilled in the art can make equivalent change or replacement to correlation technique feature, these
Technical scheme after changing or replacing it is fallen within protection scope of the present invention.
Claims (8)
1. a kind of aobvious equipment gestural control method of virtual reality, it is characterised in that comprise the following steps：
Step S11, the identification of hand is carried out by video acquisition device, and be according to time judgement static in pickup area
No entrance mouse following state；
Step S12, into after mouse following state, hand exercise is gathered by video acquisition device frame by frame, according to continuous each frame
Mouse following hand does corresponding movement in the positional information control display system of hand in image；
Step S13, in mouse following state, is identified to sentence by the hand collected video acquisition device in image
The change of disconnected hand gesture, realizes the operation clicked on and confirmed.
2. gestural control method according to claim 1, it is characterised in that the identification to hand in acquired image is adopted
The strong classifier built with AdaBoost algorithms is carried out.
3. gestural control method according to claim 2, it is characterised in that the strong classification built using AdaBoost algorithms
The method of device is：
Step S21, builds the training sample (x identified with positive and negative samples1,y1)、(x2,y2)……(xn,yn), wherein yi=0
For negative sample, yi=1 is positive sample, and n is the total number of training sample；
Step S22, initializes the weight w of training samplet,i；T is cycle-index；
Step S23, the T that takes t=1 ..., T for training maximum cycle；
Step S231, normalizes weights：
Step S232, to each feature f, trains a grader h (xi, f, p, θ), that is, equation is estimated on qtResidual error：
εf=∑iqi|h(xi,f,p,θ)-yi|
Wherein, p is the extreme value direction of inequality, and θ is predetermined threshold value；
Step S233, the minimum grader h of selection residual errort
minf,p,θΣiqi|h(xi,f,p,θ)-yi|=∑iqi|h(xi,ft,pt,θt)-yi|
ht(x)=h (x, ft,pt,θt)
Step S234, updates weights
Wherein, εtFor the grader h selected in step S233t(x) corresponding residual error；
If sample xiCorrectly classified then ei=0, otherwise ei=1；
Step S24, is finally combined into strong classifier：
Wherein：
4. according to gestural control method according to any one of claims 1 to 3, it is characterised in that recognized in this method
Hand gesture includes clenching fist and non-two kinds of gestures of clenching fist, and non-gesture of clenching fist is used to control mouse following hand in display system to do phase
It should move, gesture of clenching fist is used to realize that the click of mouse in display system to confirm operation.
5. gestural control method according to claim 4, it is characterised in that gesture of clenching fist is used to realize mouse in display system
Target clicks on the method for confirming to operate：Hand clenches fist state change to clench fist state by non-, and moves forward, and it is mobile away from
From the threshold speed that the distance threshold less than setting, translational speed are less than setting, then the click for performing mouse in display system is true
Recognize behaviour.
6. according to gestural control method according to any one of claims 1 to 3, it is characterised in that in acquired image
The identification of hand is needed to carry out image the pretreatment of image segmentation, and the hand in image and background segment are opened.
7. gestural control method according to claim 6, it is characterised in that the method for the pretreatment of described image segmentation
To carry out region recognition based on the difference between skin color and background colour.
8. gestural control method according to claim 6, it is characterised in that the method for the pretreatment of described image segmentation is
Image partition method based on profile.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610958045.9A CN107015636A (en) | 2016-10-27 | 2016-10-27 | The aobvious equipment gestural control method of virtual reality |
| PCT/CN2017/095028 WO2018076848A1 (en) | 2016-10-27 | 2017-07-28 | Gesture control method for virtual reality head-mounted display device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610958045.9A CN107015636A (en) | 2016-10-27 | 2016-10-27 | The aobvious equipment gestural control method of virtual reality |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107015636A true CN107015636A (en) | 2017-08-04 |
Family
ID=59439481
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610958045.9A Pending CN107015636A (en) | 2016-10-27 | 2016-10-27 | The aobvious equipment gestural control method of virtual reality |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107015636A (en) |
| WO (1) | WO2018076848A1 (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101344816A (en) * | 2008-08-15 | 2009-01-14 | 华南理工大学 | Human-machine interaction method and device based on sight tracing and gesture discriminating |
| CN101763515A (en) * | 2009-09-23 | 2010-06-30 | 中国科学院自动化研究所 | Real-time gesture interaction method based on computer vision |
| CN103376895A (en) * | 2012-04-24 | 2013-10-30 | 纬创资通股份有限公司 | Gesture control method and gesture control device |
| CN103530613A (en) * | 2013-10-15 | 2014-01-22 | 无锡易视腾科技有限公司 | Target person hand gesture interaction method based on monocular video sequence |
| CN104182132A (en) * | 2014-08-07 | 2014-12-03 | 天津三星电子有限公司 | Gesture control method for intelligent terminal and intelligent terminal |
-
2016
- 2016-10-27 CN CN201610958045.9A patent/CN107015636A/en active Pending
-
2017
- 2017-07-28 WO PCT/CN2017/095028 patent/WO2018076848A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101344816A (en) * | 2008-08-15 | 2009-01-14 | 华南理工大学 | Human-machine interaction method and device based on sight tracing and gesture discriminating |
| CN101763515A (en) * | 2009-09-23 | 2010-06-30 | 中国科学院自动化研究所 | Real-time gesture interaction method based on computer vision |
| CN103376895A (en) * | 2012-04-24 | 2013-10-30 | 纬创资通股份有限公司 | Gesture control method and gesture control device |
| CN103530613A (en) * | 2013-10-15 | 2014-01-22 | 无锡易视腾科技有限公司 | Target person hand gesture interaction method based on monocular video sequence |
| CN104182132A (en) * | 2014-08-07 | 2014-12-03 | 天津三星电子有限公司 | Gesture control method for intelligent terminal and intelligent terminal |
Non-Patent Citations (1)
| Title |
|---|
| 李长风: "基于AdaBoost算法的人脸检测研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2018076848A1 (en) | 2018-05-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Kumar et al. | Independent Bayesian classifier combination based sign language recognition using facial expression | |
| CN109325454B (en) | Static gesture real-time recognition method based on YOLOv3 | |
| Mohandes et al. | Arabic sign language recognition using the leap motion controller | |
| EP2877909B1 (en) | Multimodal interaction with near-to-eye display | |
| CN107578023A (en) | Man-machine interaction gesture identification method, apparatus and system | |
| Agrawal et al. | A survey on manual and non-manual sign language recognition for isolated and continuous sign | |
| Jain et al. | Deep neural learning techniques with long short-term memory for gesture recognition | |
| Angona et al. | Automated Bangla sign language translation system for alphabets by means of MobileNet | |
| CN103793056A (en) | Mid-air gesture roaming control method based on distance vector | |
| Zobl et al. | Gesture components for natural interaction with in-car devices | |
| Gaikwad et al. | Recognition of American sign language using image processing and machine learning | |
| US10095308B2 (en) | Gesture based human machine interface using marker | |
| CN107015636A (en) | The aobvious equipment gestural control method of virtual reality | |
| Dhamanskar et al. | Human computer interaction using hand gestures and voice | |
| Atwood et al. | American sign language recognition system | |
| Manresa-Yee et al. | Towards hands-free interfaces based on real-time robust facial gesture recognition | |
| Pushpalatha et al. | Sign Language Converter Using Feature Extractor and PoseNet | |
| Enikeev et al. | Russian Fingerspelling Recognition Using Leap Motion Controller | |
| Tazhigaliyeva et al. | Cyrillic manual alphabet recognition in RGB and RGB-D data for sign language interpreting robotic system (SLIRS) | |
| Tazhigaliyeva et al. | Slirs: Sign language interpreting system for human-robot interaction | |
| Garg | Converting American sign language to voice using RBFNN | |
| Vatavu et al. | Visual recognition of hand postures for interacting with virtual environments | |
| Hannuksela et al. | Adaptive motion-based gesture recognition interface for mobile phones | |
| Rai et al. | Gesture recognition system | |
| Pradeep et al. | Advancement of sign language recognition through technology using python and OpenCV |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170804 |
|
| RJ01 | Rejection of invention patent application after publication |