Summary of the invention
The object of the present invention is to provide the extracting method of a kind of wrist based on depth camera point and arm point, solve the problems referred to above that prior art exists.The three-dimensional position of the depth map that the present invention is provided by Kinect depth camera and the palm of the hand point that OpeNI/NITE provides, first by hand region segmentation out, then using the bianry image that is partitioned into as mask figure, from coloured image, split region of selling, utilize complexion model and Bayes classifier in the coloured image in hand region, extract accurate hand region.In the accurate hand region extracted, utilize wrist point extracting method and arm point extracting method to extract wrist point and arm point, then utilize wrist point, arm point and palm of the hand point to design various gestures.
Above-mentioned purpose of the present invention is achieved through the following technical solutions:
Based on the wrist point of depth camera and the extracting method of arm point, the palm of the hand point three-dimensional position that the cromogram gathered by Kinect depth camera, depth map and OpenNI/NITE are provided, first hand region is split, then by Bayes's complexion model by accurate hand extracted region out, by visible sensation method, wrist point and arm point are extracted; The multiple man-machine interaction gesture of palm of the hand point design that the wrist point extracted, arm point and OpenNI/NITE provide, is then applied to virtual reality and augmented reality field by the gesture of design;
Described hand region segmentation is: first split region of selling in the two-dimensional direction, then split in the depth direction, again using the hand area image that is partitioned into as mask figure, in the coloured image that Kinect provides by hand region segmentation out, then accurate hand region is extracted by Bayes's complexion model;
The method that described wrist point extracts is: the end points first extracting wrist, then with this end points for the cornerwise summit of rectangle, do with each point on handwheel exterior feature the inscribe rectangle being parallel to coordinate axis, the central point of the inscribe rectangle selecting diagonal line the shortest is wrist point;
The method that described arm point extracts is: first ask for all incircles calculating profile, in all incircles, inscribe radius of a circle is selected to be greater than the threshold value (parameter training see inscribed circle diameter threshold value) of setting to prevent from getting on finger, palm of the hand point and wrist point be not in incircle, and the center of circle of distance palm of the hand point and wrist point incircle is farthest arm point.
according to the position of palm of the hand point, wrist point and arm point and straight angle devise 3 class gestures, and to combine with virtual reality and augmented reality, the gesture designed be applied in virtual reality and augmented reality.
Beneficial effect of the present invention is: the method for existing extraction wrist and arm, and major part arranges aid mark in wrist and arm place, is such as a belt being different from the colour of skin in wrist, wrist from extracting.Method wrist point and arm point extracted based on the method for pure vision is rarely found.In existing gesture identification investigation and application, using wrist point and arm point as trace point and to design mutual example also rarely found.This method is simple, and calculated amount is little, can real-time stabilization extract wrist point and arm point, and by the wrist point extracted and arm point design gesture be applied to virtual reality and augmented reality field.
Accompanying drawing explanation
Accompanying drawing described herein is used to provide a further understanding of the present invention, forms a application's part, and illustrative example of the present invention and explanation thereof, for explaining the present invention, do not form inappropriate limitation of the present invention.
Fig. 1 is the process flow diagram in the accurate hand region of extraction of the present invention;
The cromogram that Fig. 2 inputs when being the accurate hand region of extraction of the present invention;
Depth map when Fig. 3 is extraction of the present invention accurate hand region;
When Fig. 4 is extraction of the present invention accurate hand region, depth map is split to the coarse hand region obtained;
Coarse hand and cromogram corresponding to arm regions when Fig. 5 is extraction of the present invention accurate hand region;
The accurate hand area image that Fig. 6 obtains when being extraction of the present invention accurate hand region;
Fig. 7 is the design sketch that wrist end points of the present invention extracts;
Fig. 8 is that wrist end points of the present invention is extracted as inscribe rectangle schematic diagram;
Fig. 9 is that wrist end points of the present invention is extracted as non-inscribe rectangle schematic diagram;
Figure 10 is the wrist point schematic diagram that various gesture of the present invention is extracted;
Figure 11 is that arm of the present invention point is extracted as non-incircle schematic diagram;
Figure 12 is that arm of the present invention point is extracted as incircle schematic diagram;
Figure 13 is the palm of the hand of the present invention some schematic diagram in incircle;
Figure 14 is that wrist of the present invention point is at incircle schematic diagram;
The parameter training figure of Figure 15 inscribed circle diameter threshold value;
Figure 16 is arm point of trying to achieve various gesture by calculating incircle of the present invention;
Figure 17 is that arm of the present invention sets level gesture schematic diagram;
Figure 18 is that arm of the present invention is clenched fist and lifted gesture schematic diagram;
Figure 19 is that arm of the present invention is set level to clench fist with arm and lifted the application schematic diagram of gesture;
Figure 20 be of the present invention downwards and be bent upwards gesture schematic diagram;
Figure 21 is the application schematic diagram of wrist flex gesture of the present invention;
Figure 22 is of the present invention downwards with to Back stroke arm gesture schematic diagram;
Figure 23 is the application schematic diagram waving arm gesture of the present invention.
Embodiment
Detailed content of the present invention and embodiment thereof is further illustrated below in conjunction with accompanying drawing.
See shown in Fig. 1 to Figure 23, the extracting method of the point of the wrist based on depth camera of the present invention and arm point, the three-dimensional position of the palm of the hand point that the depth map provided by Kinect depth camera and OpenNI/NITE are provided, first by hand region segmentation out, then using the bianry image that is partitioned into as mask figure, from coloured image, split region of selling, utilize complexion model and Bayes classifier in the coloured image in hand region, extract accurate hand region.In the accurate hand region extracted, utilize wrist point extracting method and arm point extracting method to extract wrist point and arm point, then utilize wrist point, arm point and palm of the hand point design various gestures.
1, the segmentation in hand region
According to the three-dimensional coordinate of the palm of the hand point that OpenNI/NITE provides, first split in the two-dimensional direction.Palm of the hand point is the center of rectangle, splits the rectangle that comprises hand and arm regions.
。
for palm of the hand point coordinate in the two-dimensional direction.
represent the length of side of rectangle.
Segmentation at depth direction: the depth map provided according to Kinect, extracts the depth value of palm of the hand point
, determine hand and arm regions.If
and this depth value
meet
,
the point in hand and arm regions, that is:
wherein
whether marker image vegetarian refreshments is hand and arm regions,
represent the depth value of the palm of the hand point traced into,
represent pixel
the depth value at place,
represent the depth capacity scope of hand and arm, the present invention is taken as 160mm.
The extraction of area of skin color: due to Kinect data, cause the edge sawtooth of hand and arm regions obvious, this can affect the accuracy of contours extract below.Therefore, after obtaining in one's hands and arm regions, application Bayes complexion model, removes the place that those are not the colours of skin, obtains accurate hand and arm regions.
The present invention adopts YCbCr color space to build complexion model, and its reason is the homogeneity of its discreteness and the visually-perceptible of people, brightness and chrominance separation and the characteristic such as colour of skin aggregation zone is compact.Then calculating each Cb*Cr component with Bayes classifier is skin color probability, is greater than 0.6 for the colour of skin, sets up colour of skin look-up table.When given one will be extracted the image of area of skin color, be translated into YCbCr color space, in the colour of skin look-up table set up, whether inquire about the Cb*Cr component of each pixel for the colour of skin.
2, the extraction of wrist point
2.1, from the bianry image of hand and arm, extract profile, then extract the convex closure of profile.Convex closure is made up of some straight-line segments, and the end points of line segment is exactly the intersection point with convex closure and profile.
2.2, the extraction of wrist end points
In the line segment forming convex closure, choose the longest line segment, obtain the profile that this line segment is corresponding, calculate each point on this profile to the distance of nose section, get the point P that ultimate range is corresponding, namely P point is an end points of wrist.
2.3, the extraction of wrist point
With extract wrist end points be a summit, on contouring, any point is another summit, and the line segment connecting two summits is the diagonal line of rectangle, does rectangle, judge whether this rectangle is profile inscribe rectangle, and the point namely on rectangle is all on profile or at contoured interior.Institute in this approach on traversal profile a little, is cornerwise two summits of rectangle with the wrist end points extracted, does rectangle, ask inscribe rectangle.
Ask in all inscribe rectangles, select the inscribe rectangle that diagonal line is the shortest, the central point of inscribe rectangle is wrist point.
3, the extraction of arm point
Wrist is a narrow region, so the range areas of wrist point is very little.Arm is then a sizable region, so the extraction also quite flexible of arm point.Calculate the incircle of profile, inscribe diameter of a circle be greater than the threshold value (preventing incircle from getting on finger) of setting and palm of the hand point and wrist point scarcely in circle (preventing incircle to be taken in palm and wrist).Chosen distance palm of the hand point and wrist point incircle farthest in all incircles, the center of circle of this circle is exactly arm point.
3.1, the incircle of profile is calculated
On contouring a bit, with this point for basic point, contouring a bit does line segment, with the mid point of line segment for the center of circle, line segment length is that diameter does circle, judge on profile a little to the distance in the center of circle, if profile has a point or multiple point be less than radius to the distance in the center of circle, just prove that this circle is not incircle.
The center of continuing to get next point and basic point line is the center of circle, and the distance between 2 is that diameter does circle, determines whether incircle.Until the institute on profile has a little all traveled through, basic point has just got next point as basic point, then does circle with each point on profile, asks for incircle, until basic point is got all over whole profile.
3.2 calculate arm point
The center of circle calculating each incircle is to the distance of palm of the hand point and wrist point, if palm of the hand point and wrist point are less than radius to the distance in the center of circle, prove that hand point and wrist point are in incircle, this incircle is undesirable.Otherwise record this centre point respectively, radius and the center of circle are to the distance of palm of the hand point and wrist point.In all qualified incircles, radius is selected to be greater than certain threshold value and distance of center circle is arm point from the center of circle of palm of the hand point and wrist point incircle farthest.
The parameter training of 3.3 inscribed circle diameter threshold values
The parameter training process of inscribed circle diameter threshold value of the present invention: in selected depth range, stretch out a finger at every turn, finger will stretch and not bend, slowly mobile before and after in this depth range, in the process of movement, adopt the method based on profile to extract finger tip point, with the finger tip point extracted for the center of circle, preseting length r=20 is that radius does circle, this circle has two intersection points with finger, calculate the distance of two intersection points, namely the width pointed, in this depth range, other fingers in the five fingers are changed every a period of time, repeatedly test, extract the width in this depth range of different finger.The present invention have selected six depth rangees, the finger width value of each depth range is exported, tries to achieve the scope of finger width, and the intermediate value of getting finger width scope adds the threshold value in the process that 3px extracts as arm point.
The width range square length of side that depth value scope is pointed is chosen (
)
[1001mm,1100mm] [10px,13px] 12px+
[901mm,1000mm] [12px,16px] 14px+
[801mm,900mm] [15px, 20px] 17px+
[701mm,800mm] [18px,23px] 20px+
[601mm,700mm] [23px,26px] 24px+
[501mm,600mm] [24px,28px] 26px+
4, the design of the palm of the hand, wrist and handle composite gesture
If palm of the hand point is
, wrist point is
, arm point is
if, straight line
with straight line
obtuse angle in angle is
, straight line
with the angle of the transverse axis of coordinate axis be
.According to
,
and 3 class gestures are designed in the position of palm of the hand point and arm point.
4.1, arm is stretched flat and lifts the design of gesture
When
and
time, be set as that arm is stretched flat gesture.
When
and
time, for clenching fist of arm lifts gesture,
4.2, the design of wrist flex gesture
Work as straight line
with straight line
obtuse angle in angle is
and
for hand is bent downwardly gesture along wrist.When
and
for hand is bent upwards gesture along wrist.
4.3, the design of arm gesture is waved
When
for arm is brandished downwards.
When
for arm is upwards brandished.
Embodiment 1: the segmentation in hand region
Shown in Fig. 1 and Fig. 2, the palm of the hand point three-dimensional position that the present invention utilizes Kinect depth camera to provide depth map and OpenNI/NITE to provide carries out the segmentation in hand region.Then the look-up table utilizing Bayes's complexion model to set up, extracts accurate hand region.
Embodiment 2: the extraction of wrist point
See shown in Fig. 3 to Fig. 5, first extract the profile in hand region, and then extract the convex closure of profile, try to achieve the profile of the nose section correspondence of convex closure, calculating the distance of each point on this profile to nose section, is an end points of wrist to the point that nose segment distance is maximum.
Wrist end points is a summit, and appointing on contouring is a bit another summit, and two summits on the diagonal line being rectangle with two summits, do the rectangle being parallel to coordinate axis, then to judge on this rectangle each point whether on profile or contoured interior.If rectangle exists the point outside profile, then this rectangle is not inscribe rectangle.Travel through the point on each profile in this approach, be diagonal line two summits with wrist end points, do rectangle, ask for inscribe rectangle.The inscribe rectangle selecting diagonal line the shortest in all inscribe rectangles, the central point of this rectangle is exactly wrist point.
Embodiment 3: the extraction of arm point
See shown in Fig. 6 to Fig. 8, on contouring a bit, with this point for basic point, contouring a bit does line segment, with the mid point of line segment for the center of circle, line segment length is that diameter does circle, judge on profile a little to the distance in the center of circle, if profile has a point or multiple point be less than radius to the distance in the center of circle, just prove that this circle is not incircle.
The center of continuing to get next point and basic point line is the center of circle, and the distance between 2 is that diameter does circle, determines whether incircle.Until the institute on profile has a little all traveled through, basic point has just got next point as basic point, then does circle with each point on profile, asks for incircle, until basic point is got all over whole profile.
The center of circle calculating each incircle is to the distance of palm of the hand point and wrist point, if palm of the hand point and wrist point are less than radius to the distance in the center of circle, prove that hand point and wrist point are in incircle, this incircle is undesirable.Otherwise record this centre point respectively, radius and the center of circle are to the distance of palm of the hand point and wrist point.In all qualified incircles, radius is selected to be greater than certain threshold value and distance of center circle is arm point from the center of circle of palm of the hand point and wrist point incircle farthest.
Embodiment 4: palm of the hand point, the design of wrist point and arm point combination gesture and application
See shown in Fig. 9 to Figure 14, if palm of the hand point is
, wrist point is
, arm point is
if, straight line
with straight line
obtuse angle in angle is
, straight line
with the angle of the transverse axis of coordinate axis be
.According to
,
and 3 class gestures are designed in the position of palm of the hand point and arm point.
Gesture identification combined with augmented reality, be stretched flat with the arm designed and clench fist the stretching, extension of virtual portrait model arm and lifting of arm of lifting gesture to drive.Along the reclinate gesture of wrist, hand represents that virtual hand model in augmented reality prepares the action of opening wine bottle, hand is bent upwards along wrist the unlatching that gesture represents bottle.Wave arm gesture to play table tennis to drive the hand in virtual scene and virtual portrait.
The foregoing is only preferred embodiment of the present invention, be not limited to the present invention, for a person skilled in the art, the present invention can have various modifications and variations.All any amendments made for the present invention, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.