CN107316025A - A kind of hand gestures recognition methods and identifying system - Google Patents
A kind of hand gestures recognition methods and identifying system Download PDFInfo
- Publication number
- CN107316025A CN107316025A CN201710505926.XA CN201710505926A CN107316025A CN 107316025 A CN107316025 A CN 107316025A CN 201710505926 A CN201710505926 A CN 201710505926A CN 107316025 A CN107316025 A CN 107316025A
- Authority
- CN
- China
- Prior art keywords
- hand
- standard
- data
- preliminary
- depth
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/107—Static hand or arm
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The hand gestures recognition methods of the present invention, for solving because recognition methods robustness is low, is influenceed that the technical problem of the hand gestures of high accuracy can not be formed by depth image data quality.This method includes：Extract the depth partial gradient characteristic formation preliminary posture feature data of hand in preliminary depth image；The preliminary posture feature data of hand pass through hand gestures grader formation hand current pose data；Determine the hand standard attitude data close with hand current pose data and contrast to determine hand standard posture corresponding with hand current pose by storing index.Instant invention overcomes make it that grader predicts the outcome the larger defect of error by the depth data cavity in depth image and influence of noise.Some hand standard gesture distributions selected by storing index to cause further increase the robustness of identification process near the hand gestures predicted.Present invention additionally comprises hand gestures identifying system.
Description
Technical field
The present invention relates to the Computer Identification of real-world object and identifying system, more particularly to a kind of hand gestures identification
Method and identifying system.
Background technology
Popularization and the demand of field of human-computer interaction with depth transducer, in recent years hand gestures based on depth data
The research of identification is risen.Compared with hand gestures identification of the tradition based on RGB (i.e. RGB primary colours) image, depth data
There is provided the three-dimensional information of hand, the robustness and accuracy of hand gestures identification are improved.
But carry out hand gestures using depth data and recognize, existing depth higher to the image quality requirements of depth image
Sensor is limited by physical parameter, and the picture quality of the depth image of dynamic formation is poor, it is impossible to fully meet the number of grader
According to input requirements so that even " cavity " occur comprising substantial amounts of noise in the hand images that grader processing is obtained, seriously
Reduce the forecasting accuracy of grader.
The content of the invention
In view of this, the embodiments of the invention provide hand gestures recognition methods and identifying system, for solving due to knowing
Other method robustness is low, is influenceed that the technical problem of the hand gestures of high accuracy can not be formed by depth image data quality.
The hand gestures recognition methods of the present invention, including：
Extract the depth partial gradient characteristic formation preliminary posture feature data of hand in preliminary depth image；
The preliminary posture feature data of hand pass through hand gestures grader formation hand current pose data；
By store index determine with the close hand standard attitude data of hand current pose data and contrast determination with
The corresponding hand standard posture of hand current pose.
The hand gestures identifying system of the present invention, including following functions module：
Preliminary depth image generating means, the preliminary depth map for obtaining hand current pose by depth transducer
Picture；
The preliminary posture feature data generating device of hand, for extracting the depth partial gradient feature in preliminary depth image
The data formation preliminary posture feature data of hand；
Hand current pose data generating device, hand gestures grader shape is passed through for the preliminary posture feature data of hand
Into hand current pose data；
Posture comparison device, for being determined and the close hand standard posture of hand current pose data by storing index
Data simultaneously contrast determination hand standard posture corresponding with hand current pose.
The program module disposed in the hand gestures identifying system of the present invention, including processor, processor includes：
Preliminary depth image generating means, the preliminary depth map for obtaining hand current pose by depth transducer
Picture；
The preliminary posture feature data generating device of hand, for extracting the depth partial gradient feature in preliminary depth image
The data formation preliminary posture feature data of hand；
Hand current pose data generating device, hand gestures grader shape is passed through for the preliminary posture feature data of hand
Into hand current pose data；
Posture comparison device, for being determined and the close hand standard posture of hand current pose data by storing index
Data simultaneously contrast determination hand standard posture corresponding with hand current pose.
The hand gestures recognition methods and identifying system of the present invention is avoided directly to be lacked using with noise drawbacks and data
The preliminary posture feature data of sunken hand carry out hand gestures identification, overcome by the depth data cavity in depth image and make an uproar
Sound shadow rings the larger defect of error so that grader predicts the outcome.Even if too fast using this hand gestures recognition methods hand exercise
The Raw data quality for causing depth transducer to gather is relatively low, also will not substantially reduce final forecasting accuracy.This hand appearance
State recognition methods utilizes similitude higher-dimension degrees of data being mapped to after lower dimensional space between high dimensional data point also should be in low-dimensional
The characteristics of being embodied in the data point in space, some hand standard gesture distributions selected by storing index to cause are in prediction
Near the hand gestures gone out, the robustness of identification process is further increased.
Brief description of the drawings
Fig. 1 is the flow chart of hand gestures recognition methods of the embodiment of the present invention.
Fig. 2 is the formation flow chart of the standard depth image of hand gestures recognition methods of the embodiment of the present invention.
Fig. 3 is the formation flow of the depth characteristic of the standard depth image of hand gestures recognition methods of the embodiment of the present invention
Figure.
Fig. 4 is the formation flow chart of the hand standard posture of hand gestures recognition methods of the embodiment of the present invention.
Fig. 5 is the formation flow chart of the preliminary depth image of hand gestures recognition methods of the embodiment of the present invention.
Fig. 6 is the formation flow chart of the hand current pose of hand gestures recognition methods of the embodiment of the present invention.
Fig. 7 compares the flow chart of replacement for the hand gestures of hand gestures recognition methods of the embodiment of the present invention.
Fig. 8 is to simulate posture using the hand set up during hand gestures recognition methods of the embodiment of the present invention.
Fig. 9 is the standard depth figure that posture is simulated using hand during hand gestures recognition methods of the embodiment of the present invention
Picture.
Figure 10 is special using the two-dimentional hand standard posture of dimensionality reduction during hand gestures recognition methods of the embodiment of the present invention
Levy the visualization result of data.
Figure 11 be using during hand gestures recognition methods of the embodiment of the present invention by hand gestures grader to hand
The visualization result of ROI two-dimentional hand standard posture feature data prediction result.
Figure 12 is the hand ROI that obtains during hand gestures recognition methods of the embodiment of the present invention.
Figure 13 be hand gestures recognition methods of the embodiment of the present invention during hand ROI predict the outcome and choose it is nearest
The corresponding depth data of hand ROI postures.
Figure 14 is most like for what is chosen after hand gestures recognition methods process vacuum metrics similarity degree of the embodiment of the present invention
As a result.
Figure 15 is the structural representation of hand gestures of embodiment of the present invention identifying system or program module.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation is described, it is clear that described embodiment is only a part of embodiment of the invention, rather than whole embodiments.Based on this
Embodiment in invention, the every other reality that those of ordinary skill in the art are obtained under the premise of creative work is not made
Example is applied, the scope of protection of the invention is belonged to.
Step numbering in accompanying drawing is only used for the reference as the step, does not indicate that execution sequence.
Fig. 1 is the flow chart of hand gestures recognition methods of the embodiment of the present invention.Method and step as shown in Figure 1 includes：
Step 100：Set up hand simulation posture, generation standard depth image corresponding with hand simulation posture.
Step 200：Depth partial gradient characteristic formation hand standard posture feature in extraction standard depth image
Data.
Step 300：Grader is trained by hand standard posture feature data to form hand gestures grader, and
The storage for setting up hand standard attitude data and hand standard attitude data by hand gestures grader is indexed.
Step 400：The preliminary depth image of hand current pose is obtained by depth transducer.
Step 500：Extract the depth partial gradient characteristic formation preliminary posture feature of hand in preliminary depth image
Data.
Step 600：The preliminary posture feature data of hand pass through hand gestures grader formation hand current pose data.
Step 700：The hand standard attitude data close with hand current pose data is determined by storing index and right
Than determining hand standard posture corresponding with hand current pose.
The hand gestures recognition methods of the embodiment of the present invention, is that various hand standard postures establish tool under off-line state
There is the standard depth image of accurate depth information, and form the hand standard attitude data (hand if necessary that can be indexed
Standard attitude data can be shown with the visual image of hand standard posture), it is ensured that the standardization of hand gestures.In wire
The preliminary posture feature data of low quality depth image formation hand (the preliminary appearance of hand if necessary for obtaining depth transducer under state
State data can be shown with the visual image of the preliminary posture of hand), and with the hand standard posture number of part hand standard posture
According to directly being matched, comparison selects most suitable hand standard attitude data and then determines corresponding hand standard posture.This
The hand gestures recognition methods of inventive embodiments is avoided directly using with the preliminary appearance of the hand of noise drawbacks and data flaw
State characteristic carries out hand gestures identification, overcomes and is caused classification by the depth data cavity in depth image and influence of noise
Device predicts the outcome the larger defect of error.Even if being crossed using this hand gestures recognition methods hand exercise causes fast depth transducer
The Raw data quality of collection is relatively low, also will not substantially reduce final forecasting accuracy.The recognition methods of this hand gestures is utilized
Similitude higher-dimension degrees of data being mapped to after lower dimensional space between high dimensional data point also should be in the data point of lower dimensional space
The characteristics of embodying, some hand standard gesture distributions selected by storing index to cause are attached in the hand gestures predicted
Closely, the robustness of identification process is further increased.
Fig. 2 is the formation flow chart of the standard depth image of hand gestures recognition methods of the embodiment of the present invention.As shown in Figure 2
Step 100 includes：
Step 110：Set up hand model.
Set up the skeleton cartoon technology that hand standard posture uses computer graphics techniques.Skeleton cartoon technology is using biological
Skeleton character as three dimensional biological object basic model, according to the personal feature of three dimensional biological object to basic model carry out
The three dimensional biological object of filling and textures formation with personal feature.Flesh can be passed through using the three-dimensional bone basic model of hand
The filling of meat object, the textures formation hand model of dermatoglyph object.
Step 120：Determine hand model datum mark and Euclidean distance coordinate system.
Computer graphics can set up the two-dimensional object or three-dimensional right being made up of point, line, surface in three dimensional coordinate space
As, and obtain according to Euclidean Distance Transform scheduling algorithm the specific three-dimensional coordinate of each object such as bone.
Step 130：The movement angle in each joint of adjustment hand bone forms each hand simulation posture one by one.
Object can just be formed three by the appropriate offset of the three-dimensional coordinate application to specific object or movement rule
In dimension coordinate space along six degree of freedom change in location and whole object shape change.Respectively closed in three-dimensional bone basic model
The movement angle of section object is limited to the movement angle of the corresponding physiological joint of specific articulated objects.By adjusting in hand model
The specific orientation of each bone, it is possible to form different hand gestures.Singlehanded hand includes 27 pieces of bones, more than 15 joints,
It is determined that at least 1000 kinds of the hand standard posture (containing typical transition posture) that can be formed.
Step 140：Carry out rendering acquisition using Euclidean distance as parameter often to adjusting postarticular hand model each time
One hand simulates the standard depth image of posture.
, can be by the physical features data of the objects such as muscle, skin by the model rendering technology of Computer Animated Graph
Change, and the fixing point and physical features data by objects such as muscle, skins in three-dimensional bone basic model form mapping relations.It is logical
The form for changing three-dimensional bone basic model is crossed, acquisition renders rear corresponding hand gestures image.
Each coordinate points and the Euclidean distance of datum mark map with tonal gradation formation in hand standard posture, by each
The grey scale change of the standard depth image of hand simulation posture can just reflect depth information.
While the standard depth image of all hand standard postures is formed, in order to reflect before and after any joint change
Similarity between hand standard posture, the correlation formed according to the similarity between hand standard posture between each hand standard posture
Scale label.Degree of correlation label can be represented using the vector data of change direction or angle changing, and then be used as standard depth figure
The indexing parameter of picture.
, can also be in render process by light source and visual angle in the hand gestures recognition methods of one embodiment of the invention
Coordinate unification, according to hand each several part pixel luminance difference obtain a fixed viewpoint in hand each point apart from depth information.
Fig. 3 is the formation flow of the depth characteristic of the standard depth image of hand gestures recognition methods of the embodiment of the present invention
Figure.Step 200 as shown in Figure 3 includes：
Step 210：The depth information carried in each standard depth image using pixel determines the depth of each pixel
Angle value.
Depth information can be the European coordinate value of each pixel or the brightness value of each pixel.
Step 220：Pixel is formed in different directions and apart from upper depth partial gradient value according to the depth value of pixel.
Depth partial gradient value refers to quantify the depth difference that depth information is formed between pixel.A usual center pixel and
One center pixel of depth difference formation of some spaced pixels of adjacent pixel or same distance in different directions, it is different away from
It is the concentration gradient value in different directions dimension and range dimension from upper concentration gradient value.
Specifically, depth partial gradient feature acquisition modes are as follows：
Wherein uiPixel i depth partial gradient characteristic is represented, z (u) represents the depth value of the pixel, and u is
Random offset.
Step 230：Each standard depth image is formed according to the depth partial gradient value of each each dimension of pixel
High-dimensional hand standard posture feature data.
Different directions are embodied in using the concentration gradient value of the multiple dimensions of depth partial gradient characteristic and apart from upper each
Attraction or exclusion relationses between pixel, form changing rule of each hand standard posture in specific depth attribute.
Fig. 4 is the formation flow chart of the hand standard posture of hand gestures recognition methods of the embodiment of the present invention.As shown in Figure 4
Step 300 includes：
Step 310：By the hand standard posture feature data normalization of each standard depth image and carry out at dimensionality reduction
Reason, forms two-dimentional hand standard posture feature data.
Data normalization ensures that the measurement uniformity of hand standard posture feature data completes data normalization processing, realizes
Comparativity between data target.
Form the dimension-reduction algorithm such as t-sne algorithms (tstochastic that two-dimentional hand standard posture feature data are used
Neighbor embedding are t distribution neighborhoods embedded mobile GIS), pass through t- using hand standard posture feature data as input
Sne algorithms carry out dimension-reduction treatment, and high-dimensional hand standard posture feature data are formed into two-dimentional hand standard posture feature number
According to.
Step 320：Using the two-dimentional hand standard posture feature data of each standard depth image as input, to classification
Device is trained, and forms the hand gestures grader for hand model.
Grader can use random forest grader or depth convolutional network.For example with random forest grader,
Using two-dimentional hand standard posture feature data as the input of random forest grader, due to the pixel of each standard depth image
Depth information accurately and reliably, the parameter of algorithm can fully be adjusted so that algorithm formation grader be based on hand
Posture carries out comprehensive data test, and the reliability predicted the outcome of classifying is able to fully verify, simulation test result can be met
High duplication.
Step 330：The two-dimentional hand standard posture feature data of each standard depth image are passed through into hand gestures point
Class device exports corresponding hand standard attitude data, and the vector label set up between hand standard attitude data, and passes through vector
Label sets up the storage index of hand standard attitude data.
By the vector label of the degree of correlation label formation hand standard attitude data of hand standard posture, with vectorial degree
To represent the similitude of hand standard posture.The storage of hand standard attitude data is completed using rational data structure simultaneously.
For example with kd-tree (k-dimensional tree are k dimension spaces segmentation data tree structure) as data structure storage hand
Ministerial standard attitude data is simultaneously indexed with vector label, it is possible to achieve the quick indexing of similar hand standard posture.
Fig. 5 is the formation flow chart of the preliminary depth image of hand gestures recognition methods of the embodiment of the present invention.As shown in Figure 5
Step 500 includes：
Step 510：The depth information carried in preliminary depth image using pixel determines each pixel in different directions
On depth partial gradient value.
Depth partial gradient value refers to quantify the depth difference that depth information is formed between pixel.A usual pixel and adjacent
One center pixel of depth difference formation of some spaced pixels of adjacent pixel or same distance in different directions, it is different away from
It is the concentration gradient value in different directions dimension and range dimension from upper Grad.
Specifically, depth partial gradient feature acquisition modes are as follows：
Wherein uiPixel i depth partial gradient characteristic is represented, z (u) represents the depth value of the pixel, and u is
Random offset.
Step 520：The higher-dimension of preliminary depth image is formed according to the depth partial gradient data of multiple dimensions between pixel
The preliminary posture feature data of hand of degree.
Different directions and the suction between last pixel are embodied in using the depth partial gradient characteristic of multiple dimensions
Draw or exclusion relationses, form changing rule of each hand standard posture in specific depth attribute.
Fig. 6 is the formation flow chart of the hand current pose of hand gestures recognition methods of the embodiment of the present invention.As shown in Figure 6
Step 600 includes:
Step 610：By the preliminary posture feature data normalization of hand in preliminary depth image and carry out dimension-reduction treatment, shape
Into the preliminary posture feature data of two-dimentional hand.
Data normalization ensures that the measurement uniformity of hand standard posture feature data completes data normalization processing, realizes
Comparativity between data target.
Form the dimension-reduction algorithm such as t-sne algorithms (tstochastic that two-dimentional hand standard posture feature data are used
Neighbor embedding are t distribution neighborhoods embedded mobile GIS), pass through t- using hand standard posture feature data as input
Sne algorithms carry out dimension-reduction treatment, and high-dimensional hand standard posture feature data are formed into two-dimentional hand standard posture feature number
According to.
Step 620：The preliminary posture feature data of two-dimentional hand are exported into corresponding hand by hand gestures grader to work as
Preceding attitude data simultaneously forms corresponding vector label.
Grader can use random forest grader or depth convolutional network.For example with random forest grader,
Using two-dimentional hand standard posture feature data as the input of random forest grader, due to the pixel of each standard depth image
Depth information accurately and reliably, the parameter of algorithm can fully be adjusted so that algorithm formation grader be based on hand
Posture carries out comprehensive data test, and the reliability that result is also surveyed in classification is able to fully verify, simulation test result can be met
High duplication.
Fig. 7 compares the flow chart of replacement for the hand gestures of hand gestures recognition methods of the embodiment of the present invention.As shown in Figure 7
Step 700 includes：
Step 710：The index range of hand standard attitude data is determined according to the vector label of hand current pose data.
While hand current pose data are exported by hand gestures grader, formed and hand standard attitude data
Storage index corresponding vector label.
Step 720：Some hand standard attitude datas are obtained out of index range, hand standard attitude data and hand is utilized
Portion's current pose data form corresponding MTD image collection.
Vector label by the use of hand current pose data is used as the storage index practicing inspection in hand standard attitude data
The range parameter of rope, using obtain such as k-1 (storage dimensions of the k as hand standard attitude data) individual hand standard attitude datas with
K attitude data of hand current pose data formation renders the set for the k MTD image to be formed.
Step 730：Preliminary depth image and the depth information in MTD image collection are contrasted, with closest depth
The hand simulation posture of image replaces hand current pose.
The similarity degree measure formulas that preliminary depth image is contrasted one by one with the depth image in MTD image collection
For：
Wherein Z represents the depth data obtained from sensor, and R represents the depth data obtained from image is rendered, zi,jTable
Show the depth value of the depth image pixel for hand current pose data, ri,jRepresent for hand standard attitude data picture
The depth value of vegetarian refreshments, ρ be correspondence image in respective pixel point relevant difference value (difference).
In the hand gestures recognition methods of another embodiment of the present invention, in order to increase the accuracy finally contrasted and reduction
The data processing amount of contrast from standard depth image and in preliminary depth image, it is necessary to extract hand ROI.
As shown in figure 4, also including the step 340 performed before step 320：
Step 340：Determined according to PCA algorithms (Principal Component Analysis are Principal Component Analysis Algorithm)
The center and border of hand in hand standard posture feature data, form hand ROI two-dimentional hand standard posture according to border
Characteristic.
As shown in fig. 6, also including the step 630 performed before step 220：
Step 630：The center of hand and direction in the preliminary posture feature data of hand are determined according to PCA algorithms, bag is set
Include the fixation cubic space of the same direction of hand point data, and to towards equatorial projection；The basis on two dimensional surface
Border forms the hand ROI preliminary posture feature data of two-dimentional hand.
The hand standard attitude data formed by above step 340 and step 630 subsequent step refers to hand ROI hand
Standard attitude data, hand current pose data refer to hand ROI hand current pose data.
Therefore in step 730 as shown in Figure 7 preliminary depth image and the depth image in MTD image collection by
One contrast similarity degree measure formulas be：
Wherein ZroiRepresent the hand ROI depth datas obtained from sensor, RroiRepresent the hand obtained from image is rendered
Portion's ROI depth datas, zi,jRepresent the depth value of the depth image pixel for hand current pose data, ri,jRepresent for
The depth value of hand standard attitude data pixel.
Fig. 8 is to simulate posture using the hand set up during hand gestures recognition methods of the embodiment of the present invention.Fig. 9 is profit
The standard depth image of posture is simulated with hand during hand gestures recognition methods of the embodiment of the present invention.Figure 10 is to utilize this hair
The visualization result of the two-dimentional hand standard posture feature data of dimensionality reduction during bright embodiment hand gestures recognition methods.Figure 11
To pass through two-dimentional hand of the hand gestures grader to hand ROI during utilization hand gestures recognition methods of the embodiment of the present invention
The visualization result of standard posture feature data prediction result.It is as shown in Figs. 8 to 11, indicate and set up hand under off-line state
The process of ROI standard attitude datas.This process ensure that the hand ROI prediction data obtained by sensor have one it is high-quality
Comparison standard standard hand gestures depth image set.
Figure 12 is the hand ROI that obtains during hand gestures recognition methods of the embodiment of the present invention.Figure 13 is real for the present invention
Apply the depth corresponding with the nearest hand ROI postures chosen that predicts the outcome of hand ROI during a hand gestures recognition methods
Degrees of data.Figure 14 is the most like knot chosen after hand gestures recognition methods process vacuum metrics similarity degree of the embodiment of the present invention
Really.As shown in Figure 12 to Figure 14, indicate collection hand ROI depth datas under presence and set up hand prediction attitude data
Process.The contrast of attitude data is predicted by hand ROI standards attitude data and hand, eliminates that to only rely on low quality hand pre-
Survey attitude data prediction accuracy low, the defect of poor robustness, what the hand ROI postures and sensor for comparing selection result were gathered
Hand ROI postures are consistent.
Figure 15 is the structural representation of hand gestures of embodiment of the present invention identifying system or program module.Wrap as shown in figure 15
Include following device：
Standard depth video generation device 10, for setting up hand simulation posture, generation is corresponding with hand simulation posture
Standard depth image；
Hand standard posture feature data generating device 20, it is special for the depth partial gradient in extraction standard depth image
Levy data formation hand standard posture feature data；
Hand standard attitude data indexing unit 30, for being instructed by hand standard posture feature data to grader
White silk forms hand gestures grader, and sets up hand standard attitude data and hand standard posture number by hand gestures grader
According to storage index；
Preliminary depth image generating means 40, the preliminary depth map for obtaining hand current pose by depth transducer
Picture；
The preliminary posture feature data generating device 50 of hand, it is special for extracting the depth partial gradient in preliminary depth image
Levy the data formation preliminary posture feature data of hand；
Hand current pose data generating device 60, hand gestures grader is passed through for the preliminary posture feature data of hand
Form hand current pose data；
Posture comparison device 70, for being determined and the close hand standard appearance of hand current pose data by storing index
State data simultaneously contrast determination hand standard posture corresponding with hand current pose.
Standard depth video generation device 10 in the hand gestures identifying system of the embodiment of the present invention as shown in figure 15 is wrapped
Include：
Model building module 11, for setting up hand model；
Distance sets up module 12, for determining hand model datum mark and Euclidean distance coordinate system；
Attitude-simulating module 13, each hand simulation appearance is formed for adjusting the movement angle in each joint of hand bone one by one
State；
Standard depth image generation module 14, for adjust each time postarticular hand model using Euclidean distance as
Parameter, which render, obtains the standard depth image that each hand simulates posture；
Hand standard posture feature data generation in the hand gestures identifying system of the embodiment of the present invention as shown in figure 15
Device 20 includes：
First pixel depth generation module 21, for the depth letter carried in each standard depth image using pixel
Breath determines the depth value of each pixel；
First concentration gradient generation module 22, for according to the depth value of pixel formed pixel different directions and apart from
Depth partial gradient value；
Standard posture feature data generation module 23, for the depth partial gradient value shape according to each each dimension of pixel
Into the high-dimensional hand standard posture feature data of each standard depth image；
Hand standard attitude data indexing unit in the hand gestures identifying system of the embodiment of the present invention as shown in figure 15
30 include：
Two-dimentional standard posture feature data generation module 31, for by the hand standard posture of each standard depth image
Characteristic normalizes and carries out dimension-reduction treatment, forms two-dimentional hand standard posture feature data.
Hand gestures classifier training module 32, for the two-dimentional hand standard posture of each standard depth image is special
Data are levied as input, grader is trained, the hand gestures grader for hand model is formed.
Storage index generation module 33, for by the two-dimentional hand standard posture feature data of each standard depth image
Corresponding hand standard attitude data, and the vector mark set up between hand standard attitude data are exported by hand gestures grader
Sign, and the storage for setting up hand standard attitude data by vector label is indexed.
ROI two dimension standard posture features data generation module 34, for determining hand standard posture feature according to PCA algorithms
The center and border of hand in data, form hand ROI two-dimentional hand standard posture feature data according to border.
The preliminary posture feature data generation of hand in the hand gestures identifying system of the embodiment of the present invention as shown in figure 15
Device 50 includes：
Second pixel depth generation module 51, the depth information for being carried in preliminary depth image using pixel is determined
The each depth partial gradient value of pixel in different directions.
Second concentration gradient generation module 52, is formed for the depth partial gradient data according to multiple dimensions between pixel
The preliminary posture feature data of high-dimensional hand of preliminary depth image.
Hand current pose data generating device in the hand gestures identifying system of the embodiment of the present invention as shown in figure 15
60 include：
The two-dimentional preliminary posture feature data generation module 61 of hand, for by the preliminary posture of hand in preliminary depth image
Characteristic normalizes and carries out dimension-reduction treatment, forms the preliminary posture feature data of two-dimentional hand.
Hand current pose data generation module 62, for the preliminary posture feature data of two-dimentional hand to be passed through into hand gestures
Grader exports corresponding hand current pose data and forms corresponding vector label.
ROI two dimension standard posture features data generation module 63, for determining the preliminary posture feature of hand according to PCA algorithms
The center of hand and direction in data, set the fixation cubic space for the same direction for including hand point data, and to same direction
Equatorial projection；The hand ROI preliminary posture feature data of two-dimentional hand are formed according to border on two dimensional surface.
Posture comparison device 70 in the hand gestures identifying system of the embodiment of the present invention as shown in figure 15 includes：
Index range enquiry module 71, for determining hand standard posture according to the vector label of hand current pose data
The index range of data.
MTD image collection generation module 72, for obtaining some hand standard attitude datas out of index range,
Corresponding MTD image collection is formed using hand standard attitude data and hand current pose data.
Depth information contrast module 73, believes for contrasting preliminary depth image with the depth in MTD image collection
Breath, hand current pose is replaced with the hand simulation posture of closest depth image.
Hand gestures identifying system implements with beneficial effect reference can be made to hand gestures are recognized in the embodiment of the present invention
Method, will not be repeated here.
Figure 15 is the structural representation of hand gestures of embodiment of the present invention identifying system or program module.As shown in figure 15 originally
The program module that the hand gestures identifying system of inventive embodiments includes disposing in processor, processor includes：
Standard depth video generation device 10, for setting up hand simulation posture, generation is corresponding with hand simulation posture
Standard depth image.
Hand standard posture feature data generating device 20, it is special for the depth partial gradient in extraction standard depth image
Levy data formation hand standard posture feature data.
Hand standard attitude data indexing unit 30, for being instructed by hand standard posture feature data to grader
White silk forms hand gestures grader, and sets up hand standard attitude data and hand standard posture number by hand gestures grader
According to storage index.
Preliminary depth image generating means 40, the preliminary depth map for obtaining hand current pose by depth transducer
Picture.
The preliminary posture feature data generating device 50 of hand, it is special for extracting the depth partial gradient in preliminary depth image
Levy the data formation preliminary posture feature data of hand.
Hand current pose data generating device 60, hand gestures grader is passed through for the preliminary posture feature data of hand
Form hand current pose data.
Posture comparison device 70, for being determined and the close hand standard appearance of hand current pose data by storing index
State data simultaneously contrast determination hand standard posture corresponding with hand current pose.
Standard depth video generation device 10 includes：
Model building module 11, for setting up hand model.
Distance sets up module 12, for determining hand model datum mark and Euclidean distance coordinate system.
Attitude-simulating module 13, each hand simulation appearance is formed for adjusting the movement angle in each joint of hand bone one by one
State.
Standard depth image generation module 14, for adjust each time postarticular hand model using Euclidean distance as
Parameter, which render, obtains the standard depth image that each hand simulates posture.
Hand standard posture feature data generating device 20 includes：
First pixel depth generation module 21, for the depth letter carried in each standard depth image using pixel
Breath determines the depth value of each pixel.
First concentration gradient generation module 22, for according to the depth value of pixel formed pixel different directions and apart from
Depth partial gradient value.
Standard posture feature data generation module 23, for the depth partial gradient value shape according to each each dimension of pixel
Into the high-dimensional hand standard posture feature data of each standard depth image.
Hand standard attitude data indexing unit 30 includes：
Two-dimentional standard posture feature data generation module 31, for by the hand standard posture of each standard depth image
Characteristic normalizes and carries out dimension-reduction treatment, forms two-dimentional hand standard posture feature data.
Hand gestures classifier training module 32, for the two-dimentional hand standard posture of each standard depth image is special
Data are levied as input, grader is trained, the hand gestures grader for hand model is formed.
Storage index generation module 33, for by the two-dimentional hand standard posture feature data of each standard depth image
Corresponding hand standard attitude data, and the vector mark set up between hand standard attitude data are exported by hand gestures grader
Sign, and the storage for setting up hand standard attitude data by vector label is indexed.
ROI two dimension standard posture features data generation module 34, for determining hand standard posture feature according to PCA algorithms
The center and border of hand in data, form hand ROI two-dimentional hand standard posture feature data according to border.
The preliminary posture feature data generating device 50 of hand includes：
Second pixel depth generation module 51, the depth information for being carried in preliminary depth image using pixel is determined
The each depth partial gradient value of pixel in different directions.
Second concentration gradient generation module 52, is formed for the depth partial gradient data according to multiple dimensions between pixel
The preliminary posture feature data of high-dimensional hand of preliminary depth image.
Hand current pose data generating device 60 includes：
The two-dimentional preliminary posture feature data generation module 61 of hand, for by the preliminary posture of hand in preliminary depth image
Characteristic normalizes and carries out dimension-reduction treatment, forms the preliminary posture feature data of two-dimentional hand.
Hand current pose data generation module 62, for the preliminary posture feature data of two-dimentional hand to be passed through into hand gestures
Grader exports corresponding hand current pose data and forms corresponding vector label.
ROI two dimension standard posture features data generation module 63, for determining the preliminary posture feature of hand according to PCA algorithms
The center of hand and direction in data, set the fixation cubic space for the same direction for including hand point data, and to same direction
Equatorial projection；The hand ROI preliminary posture feature data of two-dimentional hand are formed according to border on two dimensional surface.
Posture comparison device 70 includes：
Index range enquiry module 71, for determining hand standard posture according to the vector label of hand current pose data
The index range of data.
MTD image collection generation module 72, for obtaining some hand standard attitude datas out of index range,
Corresponding MTD image collection is formed using hand standard attitude data and hand current pose data.
Depth information contrast module 73, believes for contrasting preliminary depth image with the depth in MTD image collection
Breath, hand current pose is replaced with the hand simulation posture of closest depth image.
Hand gestures identifying system implements with beneficial effect reference can be made to hand gestures are recognized in the embodiment of the present invention
Method, will not be repeated here.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all essences in the present invention
God is with principle, and any modification, equivalent substitution for being made etc. should be included in the scope of the protection.
Claims (12)
1. a kind of hand gestures recognition methods, including：
Extract the depth partial gradient characteristic formation preliminary posture feature data of hand in preliminary depth image；
Classification is carried out to the preliminary posture feature data of hand by hand gestures grader and forms hand current pose data；
By storing index determination and the close hand standard attitude data of hand current pose data and contrasting determination and hand
The corresponding hand standard posture of current pose.
2. hand gestures recognition methods as claimed in claim 1, it is characterised in that also include：
Set up hand simulation posture, generation standard depth image corresponding with hand simulation posture；
Depth partial gradient characteristic formation hand standard posture feature data in extraction standard depth image；
Grader is trained to form hand gestures grader by hand standard posture feature data, and passes through hand gestures
Grader sets up the storage index of the hand standard attitude data and the hand standard attitude data.
3. hand gestures recognition methods as claimed in claim 2, it is characterised in that described to set up hand simulation posture, generation with
The corresponding standard depth image of hand simulation posture includes：
Set up hand model；
Determine the hand model datum mark and Euclidean distance coordinate system；
The movement angle formation hand simulation posture of the skeletal joint of the hand model is adjusted one by one；
Hand simulation posture render using Euclidean distance as parameter and obtains each hand simulation posture
The standard depth image.
4. hand gestures recognition methods as claimed in claim 2, it is characterised in that the depth in the extraction standard depth image
Degree partial gradient characteristic formation hand standard posture feature data include：
The depth information carried in each described standard depth image using pixel determines the depth value of each pixel；
The pixel is formed in different directions and apart from upper depth partial gradient value according to the depth value of the pixel；
Each standard depth image is formed according to the depth partial gradient value of each each dimension of pixel
The high-dimensional hand standard posture feature data.
5. hand gestures recognition methods as claimed in claim 2, it is characterised in that described to pass through hand standard posture feature number
Hand gestures grader is formed according to being trained to grader, and the hand standard posture is set up by hand gestures grader
The storage index of data and the hand standard attitude data includes：
By the hand standard posture feature data normalization of each standard depth image and carry out dimension-reduction treatment, shape
Into two-dimentional hand standard posture feature data；
Using the two-dimentional hand standard posture feature data of each standard depth image as input, grader is entered
Row training, forms the hand gestures grader for hand model；
The two-dimentional hand standard posture feature data of each standard depth image are passed through into the hand gestures point
Class device exports the corresponding hand standard attitude data, and the vector label set up between the hand standard attitude data, and
The storage for setting up hand standard attitude data by the vector label is indexed.
6. hand gestures recognition methods as claimed in claim 2, it is characterised in that the depth in the preliminary depth image of extraction
The degree partial gradient characteristic formation preliminary posture feature data of hand include：
The depth information that is carried in the preliminary depth image using pixel determines each pixel in different directions
Depth partial gradient value；
The high-dimensional of the preliminary depth image is formed according to the depth partial gradient value of multiple dimensions between the pixel
The preliminary posture feature data of the hand.
7. hand gestures recognition methods as claimed in claim 2, it is characterised in that the preliminary posture feature data of hand are led to
Crossing hand gestures grader formation hand current pose data includes：
By the preliminary posture feature data normalization of the hand in preliminary depth image and dimension-reduction treatment is carried out, form described two
Tie up the preliminary posture feature data of hand；
The two-dimentional preliminary posture feature data of hand are exported into the corresponding hand by the hand gestures grader to work as
Preceding attitude data simultaneously forms the corresponding vector label.
8. hand gestures recognition methods as claimed in claim 2, it is characterised in that described to be determined and hand by storing index
The close hand standard attitude data of current pose data simultaneously contrasts determination hand standard posture corresponding with hand current pose
Including：
The index range of the hand standard attitude data is determined according to the vector label of the hand current pose data；
Some hand standard attitude datas are obtained out of described index range, the hand standard attitude data and institute is utilized
State hand current pose data and form corresponding MTD image collection；
The preliminary depth image and the depth information in the MTD image collection are contrasted, described in closest
The hand simulation posture of depth image replaces the hand current pose.
9. hand gestures recognition methods as claimed in claim 2, it is characterised in that described to pass through hand standard posture feature number
Hand gestures grader is formed according to being trained to grader, and the hand standard posture is set up by hand gestures grader
The storage index of data and the hand standard attitude data includes：
The center and border of hand in the hand standard posture feature data are determined according to PCA algorithms, according to the border shape
Into hand ROI two-dimentional hand standard posture feature data；
Using each hand ROI two-dimentional hand standard posture feature data as input, grader is trained, shape
Into the hand gestures grader for hand model；
Hand ROI two dimensions hand standard posture feature data each described are exported by the hand gestures grader
The corresponding hand standard attitude data, and the vector label set up between the hand standard attitude data, and by described
Vector label sets up the storage index of hand standard attitude data.
10. hand gestures recognition methods as claimed in claim 2, it is characterised in that the preliminary posture feature data of hand are led to
Crossing hand gestures grader formation hand current pose data includes：
The center of hand and direction in the preliminary posture feature data of the hand are determined according to PCA algorithms, setting includes hand point
The fixation cubic space of the same direction of data, and to towards equatorial projection；
The hand ROI preliminary posture feature data of two-dimentional hand are formed according to border on the two dimensional surface；
The preliminary posture feature data of the two-dimentional hand of the hand ROI are exported into correspondence by the hand gestures grader
The hand current pose data and form the corresponding vector label.
11. a kind of hand gestures identifying system, including：
Preliminary depth image generating means, the preliminary depth image for obtaining hand current pose by depth transducer；
The preliminary posture feature data generating device of hand, for extracting the depth partial gradient characteristic in preliminary depth image
Form the preliminary posture feature data of hand；
Hand current pose data generating device, hand gestures grader formation hand is passed through for the preliminary posture feature data of hand
Portion's current pose data；
Posture comparison device, for being determined and the close hand standard attitude data of hand current pose data by storing index
And contrast determination hand standard posture corresponding with hand current pose.
12. the program module disposed in a kind of hand gestures identifying system, including processor, processor includes：
Preliminary depth image generating means, the preliminary depth image for obtaining hand current pose by depth transducer；
The preliminary posture feature data generating device of hand, for extracting the depth partial gradient characteristic in preliminary depth image
Form the preliminary posture feature data of hand；
Hand current pose data generating device, hand gestures grader formation hand is passed through for the preliminary posture feature data of hand
Portion's current pose data；
Posture comparison device, for being determined and the close hand standard attitude data of hand current pose data by storing index
And contrast determination hand standard posture corresponding with hand current pose.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710505926.XA CN107316025B (en) | 2017-06-27 | 2017-06-27 | Hand gesture recognition method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710505926.XA CN107316025B (en) | 2017-06-27 | 2017-06-27 | Hand gesture recognition method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107316025A true CN107316025A (en) | 2017-11-03 |
| CN107316025B CN107316025B (en) | 2021-04-06 |
Family
ID=60180321
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710505926.XA Active CN107316025B (en) | 2017-06-27 | 2017-06-27 | Hand gesture recognition method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107316025B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108154176A (en) * | 2017-12-22 | 2018-06-12 | 北京工业大学 | A kind of 3D human body attitude algorithm for estimating for single depth image |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070002015A1 (en) * | 2003-01-31 | 2007-01-04 | Olympus Corporation | Movement detection device and communication apparatus |
| EP2821916A1 (en) * | 2012-02-27 | 2015-01-07 | NEC CASIO Mobile Communications, Ltd. | Voice input device, voice input method and program |
| CN104317391A (en) * | 2014-09-24 | 2015-01-28 | 华中科技大学 | Stereoscopic vision-based three-dimensional palm posture recognition interactive method and system |
| CN104517097A (en) * | 2014-09-24 | 2015-04-15 | 浙江大学 | Kinect-based moving human body posture recognition method |
| CN106030610A (en) * | 2014-01-05 | 2016-10-12 | 马诺手势股份公司 | Real-time 3D gesture recognition and tracking system for mobile devices |
-
2017
- 2017-06-27 CN CN201710505926.XA patent/CN107316025B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070002015A1 (en) * | 2003-01-31 | 2007-01-04 | Olympus Corporation | Movement detection device and communication apparatus |
| EP2821916A1 (en) * | 2012-02-27 | 2015-01-07 | NEC CASIO Mobile Communications, Ltd. | Voice input device, voice input method and program |
| CN106030610A (en) * | 2014-01-05 | 2016-10-12 | 马诺手势股份公司 | Real-time 3D gesture recognition and tracking system for mobile devices |
| CN104317391A (en) * | 2014-09-24 | 2015-01-28 | 华中科技大学 | Stereoscopic vision-based three-dimensional palm posture recognition interactive method and system |
| CN104517097A (en) * | 2014-09-24 | 2015-04-15 | 浙江大学 | Kinect-based moving human body posture recognition method |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108154176A (en) * | 2017-12-22 | 2018-06-12 | 北京工业大学 | A kind of 3D human body attitude algorithm for estimating for single depth image |
| CN108154176B (en) * | 2017-12-22 | 2021-11-05 | 北京工业大学 | 3D human body posture estimation algorithm aiming at single depth image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107316025B (en) | 2021-04-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5976103B2 (en) | Generate an avatar that reflects the player's appearance | |
| US8471848B2 (en) | System and method for tracking three dimensional objects | |
| CN108345869A (en) | Driver's gesture recognition method based on depth image and virtual data | |
| WO2022095721A1 (en) | Parameter estimation model training method and apparatus, and device and storage medium | |
| WO2016011834A1 (en) | Image processing method and system | |
| CN108510583A (en) | The generation method of facial image and the generating means of facial image | |
| US20180357819A1 (en) | Method for generating a set of annotated images | |
| CN108491776A (en) | Assembly part identification method and device based on pixel classification and monitoring system | |
| CN106447713A (en) | Automatic measurement method and device based on point cloud human body model | |
| EP3997670A1 (en) | Methods of estimating a bare body shape from a concealed scan of the body | |
| CN109886153A (en) | A kind of real-time face detection method based on depth convolutional neural networks | |
| CN113436303A (en) | Method of rendering a volume and embedding a surface in the volume | |
| CN114419320A (en) | Fish shoal detection method, system and device based on image recognition | |
| KR20230085931A (en) | Method and system for extracting color from face images | |
| CN111080754A (en) | Character animation production method and device for connecting characteristic points of head and limbs | |
| CN109919128A (en) | Acquisition methods, device and the electronic equipment of control instruction | |
| CN116721104B (en) | Live three-dimensional model defect detection method and device, electronic equipment and storage medium | |
| CN107316025A (en) | A kind of hand gestures recognition methods and identifying system | |
| US11361467B2 (en) | Pose selection and animation of characters using video data and training techniques | |
| CN108108648A (en) | A kind of new gesture recognition system device and method | |
| JP2017122993A (en) | Image processor, image processing method and program | |
| Putri et al. | Artistic Style Characterization of Vincent Van Gogh’s Paintings using Extracted Features from Visible Brush Strokes | |
| Gao et al. | Automatic construction of 3D animatable facial avatars | |
| JP6593928B2 (en) | Information processing apparatus and program | |
| JP7526412B2 (en) | Method for training a parameter estimation model, apparatus for training a parameter estimation model, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |