CN108133206B - Static gesture recognition method and device and readable storage medium - Google Patents
Static gesture recognition method and device and readable storage medium Download PDFInfo
- Publication number
- CN108133206B CN108133206B CN201810142872.XA CN201810142872A CN108133206B CN 108133206 B CN108133206 B CN 108133206B CN 201810142872 A CN201810142872 A CN 201810142872A CN 108133206 B CN108133206 B CN 108133206B
- Authority
- CN
- China
- Prior art keywords
- image
- gesture
- points
- point
- characteristic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/107—Static hand or arm
- G06V40/113—Recognition of static hand signs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/107—Static hand or arm
- G06V40/117—Biometrics derived from hands
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention provides a static gesture recognition method, a static gesture recognition device and a readable storage medium. According to the method and the device, gesture segmentation is carried out on a static gesture image to be recognized to obtain a gesture segmentation image, then an integral image of the gesture segmentation image is calculated, a scale space corresponding to the gesture segmentation image is constructed according to the integral image, all extreme points meeting judgment conditions of the extreme points based on a Hessian matrix are searched in the scale space, target feature points are screened from all the searched extreme points, finally feature values of the target feature points are extracted, and the feature values of the target feature points are recognized based on a Hu invariant moment algorithm to obtain a static gesture recognition result. Therefore, the robustness and the recognition rate of static gesture recognition can be effectively improved, and the problem of characteristic information loss caused by angle and scale change in the gesture segmentation process is solved.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a static gesture recognition method and device and a readable storage medium.
Background
At present, research based on static gesture recognition is mainly based on the conditions of good illumination, simple background and no angle and scale change of gesture input, but the recognition rate is greatly reduced when the illumination change or the complex background occurs in the static gesture recognition, and meanwhile, when the angle and the scale change, characteristic information is lost in the gesture segmentation process, so that the problem of error recognition is easily caused, how to improve the robustness and the recognition rate of the static gesture recognition is a technical problem to be solved urgently by technical personnel in the field.
Disclosure of Invention
In order to overcome the above disadvantages in the prior art, an object of the present invention is to provide a static gesture recognition method, device and readable storage medium, which can effectively improve the robustness and recognition rate of static gesture recognition and solve the problem of feature information loss caused by angle and scale changes in the gesture segmentation process.
In order to achieve the above object, the preferred embodiment of the present invention adopts the following technical solutions:
the invention provides a static gesture recognition method, which is applied to electronic equipment and comprises the following steps:
performing gesture segmentation on a static gesture image to be recognized to obtain a gesture segmentation image, wherein the static gesture image comprises a depth image and a color image;
calculating an integral image of the gesture segmentation image, and constructing a scale space corresponding to the gesture segmentation image according to the integral image;
searching all extreme points which accord with the judgment condition of the extreme points based on the Hessian matrix in the scale space, and screening target characteristic points from all the searched extreme points;
and extracting the characteristic value of the target characteristic point, and identifying the characteristic value of the target characteristic point based on a Hu invariant moment algorithm to obtain a static gesture identification result.
In a preferred embodiment of the present invention, the step of performing gesture segmentation on the static gesture image to be recognized to obtain a gesture segmented image includes:
performing image segmentation on the depth image based on a gray level histogram to obtain a first gesture segmentation result aiming at the depth image;
performing image segmentation on the color image based on the first gesture segmentation result to obtain a second gesture segmentation result for the color image;
and fusing the first gesture segmentation result and the second gesture segmentation result to obtain a gesture segmentation image.
In a preferred embodiment of the present invention, the step of performing image segmentation on the depth image based on a gray histogram to obtain a first gesture segmentation result for the depth image includes:
and carrying out gray threshold segmentation on the depth image to obtain a binary image of the static gesture, wherein the binary image is used as the first gesture segmentation result.
In a preferred embodiment of the present invention, the step of performing image segmentation on the color image based on the first gesture segmentation result to obtain a second gesture segmentation result for the color image includes:
calculating the minimum circumscribed rectangle of the static gesture according to the binary image, acquiring two-dimensional coordinates of the circumscribed rectangle, and mapping the two-dimensional coordinates to a corresponding color image to obtain the minimum circumscribed rectangle comprising the static gesture;
performing skin color segmentation on the minimum circumscribed rectangle to obtain a skin color binary image of the minimum circumscribed rectangle;
and segmenting a second gesture segmentation result from the color image through the binary image and the skin color binary image.
In a preferred embodiment of the present invention, the step of searching all extreme points in the scale space that meet the determination condition of the extreme points based on the Hessian matrix, and selecting the target feature point from all the searched extreme points includes:
searching all extreme points which accord with the judgment condition of the extreme points based on the Hessian matrix in the scale space;
searching a target extreme point under a target scale from all the extreme points;
carrying out non-maximum suppression processing of a three-dimensional neighborhood on the target extreme point to obtain positioning information of a local extreme point;
performing characteristic symbol description on each characteristic point based on the positioning information of the local extreme point to obtain a characteristic vector of each characteristic point;
calculating SURF similarity measure and Euclidean distance similarity measure corresponding to each feature vector to obtain a primary feature point screening result;
sorting the feature points according to the Euclidean distance of the feature vector of each feature point obtained by calculation, and selecting at least two groups of feature points with the top ranking as reference points;
calculating the distance from all the characteristic points except the reference points to each reference point and the included angle between all the characteristic points except the reference points and each reference point;
and screening target characteristic points from all the searched extreme points according to the distance and the included angle.
In a preferred embodiment of the present invention, the step of identifying the feature values of the target feature points based on the Hu invariant moment algorithm includes:
converting the characteristic value of the target characteristic point into a Hu moment characteristic value by adopting a Hu invariant moment algorithm;
and identifying the characteristic value of the target characteristic point through the converted Hu moment characteristic value to obtain a static gesture identification result.
The preferred embodiment of the present invention further provides a static gesture recognition apparatus, which is applied to an electronic device, and the apparatus includes:
and the gesture segmentation module is used for performing gesture segmentation on the static gesture image to be recognized to obtain a gesture segmentation image, and the static gesture image comprises a depth image and a color image.
And the construction module is used for calculating an integral image of the gesture segmentation image and constructing a scale space corresponding to the gesture segmentation image according to the integral image.
And the searching module is used for searching all extreme points which accord with the judgment condition of the extreme points based on the Hessian matrix in the scale space and screening the target characteristic points from all the searched extreme points.
And the recognition module is used for extracting the characteristic value of the target characteristic point and recognizing the characteristic value of the target characteristic point based on a Hu invariant moment algorithm so as to obtain a static gesture recognition result.
The preferred embodiment of the present invention further provides a readable storage medium, in which a computer program is stored, and the computer program, when executed, implements the static gesture recognition method described above.
Compared with the prior art, the invention has the following beneficial effects:
the embodiment of the invention provides a static gesture recognition method, a static gesture recognition device and a readable storage medium, wherein a gesture segmentation image is obtained by performing gesture segmentation on a static gesture image to be recognized, then an integral image of the gesture segmentation image is calculated, a scale space corresponding to the gesture segmentation image is constructed according to the integral image, all extreme points meeting the judgment condition of the extreme points based on a Hessian matrix are searched in the scale space, target feature points are screened from all the searched extreme points, finally feature values of the target feature points are extracted, and the feature values of the target feature points are recognized based on a Hu invariant moment algorithm to obtain a static gesture recognition result. Therefore, the anti-interference performance in the static gesture recognition process can be improved, the robustness and the recognition rate are higher under the interference of illumination change, complex background and the like, and the problem of characteristic information loss caused by angle and scale change in the gesture segmentation process is solved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
FIG. 1 is a flowchart illustrating a static gesture recognition method according to a preferred embodiment of the present invention;
FIG. 2 is a schematic flow chart illustrating the sub-steps included in step S210 shown in FIG. 1;
FIG. 3 is a functional block diagram of a static gesture recognition apparatus according to a preferred embodiment of the present invention;
fig. 4 is a schematic block diagram of an electronic device for implementing the static gesture recognition method according to a preferred embodiment of the present invention.
Icon: 100-an electronic device; 110-a bus; 120-a processor; 130-a storage medium; 140-bus interface; 150-a network adapter; 160-a user interface; 200-static gesture recognition means; 210-a gesture segmentation module; 220-a building block; 230-a lookup module; 240-identification module.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present invention, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
Fig. 1 is a schematic flow chart of a static gesture recognition method according to a preferred embodiment of the invention. It should be noted that the static gesture recognition method provided by the embodiment of the present invention is not limited by the specific sequence described in fig. 1 and below. The method comprises the following specific steps:
step S210, performing gesture segmentation on the static gesture image to be recognized to obtain a gesture segmentation image.
In this embodiment, the static gesture image includes a depth image and a color image. In detail, the present embodiment may acquire image information including static gesture information through a Kinect sensor, where the image information includes a depth image and an RGB color image. The depth image has object three-dimensional feature information, i.e., depth information. Because the depth image is not influenced by the irradiation direction of the light source and the emission characteristic of the surface of the object, and meanwhile, shadow does not exist, the three-dimensional depth information of the acquired target surface can be more accurately represented.
The inventor researches and finds that when the depth image is processed independently, the wrist or elbow part can be detected by mistake as a gesture, or the segmentation result is influenced by the shadow generated by object occlusion and hand microseismic. And the color image is processed independently to be subjected to gesture segmentation based on skin color characteristics, and the gesture segmentation is easily interfered by illumination and skin color object-like objects. In order to solve the above problem, in one embodiment, referring to fig. 2, the step S210 can be implemented by the following sub-steps:
and a substep S211 of performing image segmentation on the depth image based on the gray histogram to obtain a first gesture segmentation result for the depth image.
In this embodiment, a binary image of a static gesture may be obtained by performing grayscale thresholding on the depth image, where the binary image is used as the first gesture segmentation result. For example, the depth image may be processed using a threshold-based grayscale image segmentation algorithm. The dynamic gesture and the background are distinguished by determining a gray threshold, and pixels are divided into dynamic gesture areas by comparing the gray value of the pixels with the threshold value to obtain a gray histogram. And selecting a proper segmentation threshold value according to the gray level histogram, and segmenting the dynamic gesture to obtain a binary image comprising the dynamic gesture.
And a substep S212 of performing image segmentation on the color image based on the first gesture segmentation result to obtain a second gesture segmentation result for the color image.
And a substep S213, fusing the first gesture segmentation result and the second gesture segmentation result to obtain a gesture segmentation image.
In this embodiment, a minimum circumscribed rectangle of a static gesture may be calculated according to the binary image, a two-dimensional coordinate of the circumscribed rectangle may be obtained, the two-dimensional coordinate is mapped to a corresponding color image, the minimum circumscribed rectangle including the static gesture is obtained, then, skin color segmentation may be performed on the minimum circumscribed rectangle, a skin color binary image of the minimum circumscribed rectangle may be obtained, and finally, a second gesture segmentation result may be segmented from the color image through the binary image and the skin color binary image, specifically, the static gesture information may be segmented from the image information by performing an and operation on the binary image and the skin color binary image.
Step S220, calculating an integral image of the gesture segmentation image, and constructing a scale space corresponding to the gesture segmentation image according to the integral image.
Step S230, searching all extreme points which accord with the judgment condition of the extreme points based on the Hessian matrix in the scale space, and screening target characteristic points from all the searched extreme points.
Through careful research, aiming at the problem of error identification caused by angle transformation and scale transformation in the gesture segmentation process, the invention provides a feature extraction method for screening error feature points and combining with SURF algorithm, and in detail, the feature extraction method can be realized by the following modes:
firstly, all extreme points which accord with the judgment condition of the extreme points based on the Hessian matrix are searched in the scale space, then target extreme points under the target scale are searched in all the extreme points, non-maximum suppression processing of a three-dimensional neighborhood is carried out on the target extreme points, and the positioning information of local extreme points is obtained. Then, feature character description is carried out on each feature point based on the positioning information of the local extreme points to obtain a feature vector of each feature point, SURF similarity measure and Euclidean distance similarity measure corresponding to each feature vector are calculated to obtain a primary feature point screening result, the feature points are ranked according to the Euclidean distance of the feature vector of each feature point obtained through calculation, and at least two groups of feature points with the top ranking are selected as reference points. And then, calculating the distance from all the characteristic points except the reference points to each reference point and the included angle between all the characteristic points except the reference points and each reference point, and finally screening target characteristic points from all the searched extreme points according to the distance and the included angle.
In detail, in the above process, an integral image of the gesture segmentation image is calculated and a scale space is constructed, then all extreme points meeting the Hessian extreme point judgment condition are searched in the scale space by calculating the detection response, and non-maximum suppression such as three-dimensional stereo neighborhood is performed on all extreme points under a certain scale, so as to obtain accurate positioning information of local extreme points. Then, feature character description is carried out to obtain four-dimensional feature vectors of each feature point, and then SURF similarity measure and Euclidean distance similarity measure of the feature points are calculated to obtain a primary feature point screening result. And then, performing ascending arrangement according to the Euclidean distance of the characteristic vectors of the characteristic points obtained by calculation to obtain two point sets, and selecting two groups of characteristic points arranged at the top as reference points. Then, distances D1 and D1 'from all feature points except the reference point to the reference points p1 and p 1' are calculated, and in order to ensure the accuracy of distance comparison under the condition that the scale of the image changes, the gesture contour perimeter L is introduced to weight the distance D to obtain the weighted distance D, and whether the distance D is greater or less is judged. And finally, checking the consistency of angle parameters, namely respectively calculating the distance from all the feature points except the reference points to each reference point and the included angle between all the feature points except the reference points and each reference point, if the corresponding included angle and the distance are established, namely within an error range, judging that the feature point is a correct feature point, otherwise, judging that the feature point is an error feature point, and rejecting the error feature point, thereby solving the problem of feature information loss caused by angle and scale change in the gesture segmentation process.
And S240, extracting the characteristic value of the target characteristic point, and identifying the characteristic value of the target characteristic point based on a Hu invariant moment algorithm to obtain a static gesture identification result.
In this embodiment, the characteristic value of the target characteristic point is converted into a Hu moment characteristic value by using a Hu moment invariant algorithm, and the characteristic value of the target characteristic point is identified by using the converted Hu moment characteristic value, so as to obtain a static gesture identification result.
In detail, firstly, the static gesture recognition based on the Hu moment algorithm comprises the following steps:
it is easy to see from the above expression that, in a discrete state, when the scale of the image changes, the normalized center-to-center distance function value is not only related to the order of the moment, but also affected by the scale factor, which all affect the Hu invariant moment. Therefore, when the space vector matching alignment for constructing the Hu moment is performed, the recognition rate and the robustness are reduced.
Therefore, in order to solve the above problem, the inventor of the present application improves the above expression, and eliminates the influence of the scale factor on the Hu moment by using a normalization method, and constructs a new set of moment features as follows:
matching images by using the constructed moment characteristic value, and selecting the improved M1-M6And the original
Static gesture recognition is carried out, and the recognition rate and robustness of the static gesture can be effectively improved.
Further, referring to fig. 3, a static gesture recognition apparatus 200 according to a preferred embodiment of the present invention may include:
the gesture segmentation module 210 is configured to perform gesture segmentation on the static gesture image to be recognized to obtain a gesture segmentation image, where the gesture segmentation image includes a depth image and a color image.
The constructing module 220 is configured to calculate an integral image of the gesture segmentation image, and construct a scale space corresponding to the gesture segmentation image according to the integral image.
And the searching module 230 is configured to search all extreme points in the scale space, which meet the judgment condition based on the extreme points of the Hessian matrix, and screen the target feature points from all the searched extreme points.
And the identification module 240 is configured to extract a feature value of the target feature point, and identify the feature value of the target feature point based on a Hu invariant moment algorithm to obtain a static gesture identification result.
In an embodiment, the searching all extreme points that meet the determination condition based on the extreme points of the Hessian matrix in the scale space and screening the target feature points from all the found extreme points includes:
searching all extreme points which accord with the judgment condition of the extreme points based on the Hessian matrix in the scale space;
searching a target extreme point under a target scale from all the extreme points;
carrying out non-maximum suppression processing of a three-dimensional neighborhood on the target extreme point to obtain positioning information of a local extreme point;
performing characteristic symbol description on each characteristic point based on the positioning information of the local extreme point to obtain a characteristic vector of each characteristic point;
calculating SURF similarity measure and Euclidean distance similarity measure corresponding to each feature vector to obtain a primary feature point screening result;
sorting the feature points according to the calculated Euclidean distance of the feature vector of each feature point, and selecting at least two groups with the top ranking as reference points;
calculating the distance from all the characteristic points except the reference points to each reference point and the included angle between all the characteristic points except the reference points and each reference point;
and screening target characteristic points from all the searched extreme points according to the distance and the included angle.
In one embodiment, the method for identifying the feature value of the target feature point based on the Hu invariant moment algorithm includes:
converting the characteristic value of the target characteristic point into a Hu moment characteristic value by adopting a Hu invariant moment algorithm;
and identifying the characteristic value of the target characteristic point through the converted Hu moment characteristic value to obtain a static gesture identification result.
The detailed description of the corresponding steps in the above method embodiments can be referred to for the specific operation method of each functional module in this embodiment, and will not be repeated herein.
Further, please refer to fig. 4, which is a block diagram illustrating a structure of an electronic device 100 according to a preferred embodiment of the present invention. In this embodiment, the electronic device 100 may be a mobile terminal, a server, or other terminal devices with computing capabilities.
As shown in FIG. 4, the electronic device 100 may be implemented by a bus 110 as a general bus architecture. Bus 110 may include any number of interconnecting buses and bridges depending on the specific application of electronic device 100 and the overall design constraints. Bus 110 connects various circuits together, including processor 120, storage medium 130, and bus interface 140. Alternatively, the electronic apparatus 100 may connect a network adapter 150 or the like via the bus 110 using the bus interface 140. The network adapter 150 may be used to implement signal processing functions of a physical layer in a wireless communication network and implement transmission and reception of radio frequency signals through an antenna. The user interface 160 may connect external devices such as: a keyboard, a display, a mouse or a joystick, etc. The bus 110 may also connect various other circuits such as timing sources, peripherals, voltage regulators, or power management circuits, which are well known in the art, and therefore, will not be described in detail.
Alternatively, the electronic device 100 may be configured as a general purpose processing system, for example, commonly referred to as a chip, including: one or more microprocessors providing processing functions, and an external memory providing at least a portion of storage medium 130, all connected together with other support circuits through an external bus architecture.
Alternatively, the electronic device 100 may be implemented using: an ASIC (application specific integrated circuit) having a processor 120, a bus interface 140, a user interface 160; and at least a portion of the storage medium 130 integrated in a single chip, or the electronic device 100 may be implemented using one or more FPGAs (field programmable gate arrays), PLDs (programmable logic devices), controllers, state machines, gated logic, discrete hardware components, any other suitable circuitry, or any combination of circuitry capable of performing the various functions described throughout this disclosure.
Among other things, processor 120 is responsible for managing bus 110 and general processing (including the execution of software stored on storage medium 130). Processor 120 may be implemented using one or more general-purpose processors and/or special-purpose processors. Examples of processor 120 include microprocessors, microcontrollers, DSP processors, and other circuits capable of executing software. Software should be construed broadly to mean instructions, data, or any combination thereof, whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.
The processor 120 may execute the above embodiments, specifically, the storage medium 130 may store the static gesture recognition apparatus 200 therein, and the processor 120 may be configured to execute the static gesture recognition apparatus 200.
In summary, embodiments of the present invention provide a static gesture recognition method, an apparatus, and a readable storage medium, where a static gesture image to be recognized is subjected to gesture segmentation to obtain a gesture segmentation image, then an integral image of the gesture segmentation image is calculated, a scale space corresponding to the gesture segmentation image is constructed according to the integral image, then all extreme points meeting a criterion based on an extreme point of a Hessian matrix are searched for in the scale space, a target feature point is screened from all the found extreme points, finally a feature value of the target feature point is extracted, and a feature value of the target feature point is recognized based on a Hu invariant moment algorithm to obtain a static gesture recognition result. Therefore, the anti-interference performance in the static gesture recognition process can be improved, the robustness and the recognition rate are higher under the interference of illumination change, complex background and the like, and the problem of characteristic information loss caused by angle and scale change in the gesture segmentation process is solved.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus and method embodiments described above are illustrative only, as the flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, the functional modules in the embodiments of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
Alternatively, all or part of the implementation may be in software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Claims (8)
1. A static gesture recognition method is applied to an electronic device, and comprises the following steps:
performing gesture segmentation on a static gesture image to be recognized to obtain a gesture segmentation image, wherein the static gesture image comprises a depth image and a color image;
calculating an integral image of the gesture segmentation image, and constructing a scale space corresponding to the gesture segmentation image according to the integral image;
searching all extreme points which accord with the judgment condition of the extreme points based on the Hessian matrix in the scale space, and screening target characteristic points from all the searched extreme points;
extracting the characteristic value of the target characteristic point, identifying the characteristic value of the target characteristic point based on a Hu invariant moment algorithm to obtain a static gesture identification result, searching all extreme points which meet the judgment condition of the extreme point based on the Hessian matrix in the scale space, and screening the target characteristic point from all the searched extreme points, wherein the step comprises the following steps:
searching all extreme points which accord with the judgment condition of the extreme points based on the Hessian matrix in the scale space;
searching a target extreme point under a target scale from all the extreme points;
carrying out non-maximum suppression processing of a three-dimensional neighborhood on the target extreme point to obtain positioning information of a local extreme point;
performing characteristic symbol description on each characteristic point based on the positioning information of the local extreme point to obtain a characteristic vector of each characteristic point;
calculating SURF similarity measure and Euclidean distance similarity measure corresponding to every two feature vectors to obtain a primary feature point screening result;
sorting the feature points according to the Euclidean distance of the feature vector of each feature point obtained by calculation, and selecting at least two groups of feature points with the top ranking as reference points;
calculating the distance from all the characteristic points except the reference points to each reference point and the included angle between all the characteristic points except the reference points and each reference point;
and screening target characteristic points from all the searched extreme points according to the distance and the included angle.
2. The static gesture recognition method according to claim 1, wherein the step of performing gesture segmentation on the static gesture image to be recognized to obtain a gesture segmentation image comprises:
performing image segmentation on the depth image based on a gray level histogram to obtain a first gesture segmentation result aiming at the depth image;
performing image segmentation on the color image based on the first gesture segmentation result to obtain a second gesture segmentation result for the color image;
and fusing the first gesture segmentation result and the second gesture segmentation result to obtain a gesture segmentation image.
3. The static gesture recognition method according to claim 2, wherein the step of performing image segmentation on the depth image based on a gray histogram to obtain a first gesture segmentation result for the depth image comprises:
and carrying out gray threshold segmentation on the depth image to obtain a binary image of the static gesture, wherein the binary image is used as the first gesture segmentation result.
4. The static gesture recognition method according to claim 3, wherein the step of performing image segmentation on the color image based on the first gesture segmentation result to obtain a second gesture segmentation result for the color image comprises:
calculating the minimum circumscribed rectangle of the static gesture according to the binary image, acquiring two-dimensional coordinates of the circumscribed rectangle, and mapping the two-dimensional coordinates to a corresponding color image to obtain the minimum circumscribed rectangle comprising the static gesture;
performing skin color segmentation on the minimum circumscribed rectangle to obtain a skin color binary image of the minimum circumscribed rectangle;
and segmenting a second gesture segmentation result from the color image through the binary image and the skin color binary image.
5. The static gesture recognition method according to claim 1, wherein the step of recognizing the feature value of the target feature point based on the Hu invariant moment algorithm includes:
converting the characteristic value of the target characteristic point into a Hu moment characteristic value by adopting a Hu invariant moment algorithm;
and identifying the characteristic value of the target characteristic point through the converted Hu moment characteristic value to obtain a static gesture identification result.
6. A static gesture recognition apparatus applied to an electronic device, the apparatus comprising:
the gesture segmentation module is used for performing gesture segmentation on a static gesture image to be recognized to obtain a gesture segmentation image, wherein the static gesture image comprises a depth image and a color image;
the construction module is used for calculating an integral image of the gesture segmentation image and constructing a scale space corresponding to the gesture segmentation image according to the integral image;
the searching module is used for searching all extreme points which accord with the judgment condition of the extreme points based on the Hessian matrix in the scale space and screening target characteristic points from all the searched extreme points;
the identification module is configured to extract a feature value of the target feature point, identify the feature value of the target feature point based on a Hu invariant moment algorithm to obtain a static gesture identification result, search all extreme points that meet a criterion based on an extreme point of a Hessian matrix in the scale space, and screen the target feature point from the searched all extreme points, and includes:
searching all extreme points which accord with the judgment condition of the extreme points based on the Hessian matrix in the scale space;
searching a target extreme point under a target scale from all the extreme points;
carrying out non-maximum suppression processing of a three-dimensional neighborhood on the target extreme point to obtain positioning information of a local extreme point;
performing characteristic symbol description on each characteristic point based on the positioning information of the local extreme point to obtain a characteristic vector of each characteristic point;
calculating SURF similarity measure and Euclidean distance similarity measure corresponding to every two feature vectors to obtain a primary feature point screening result;
sorting the feature points according to the Euclidean distance of the feature vector of each feature point obtained by calculation, and selecting at least two groups of feature points with the top ranking as reference points;
calculating the distance from all the characteristic points except the reference points to each reference point and the included angle between all the characteristic points except the reference points and each reference point;
and screening target characteristic points from all the searched extreme points according to the distance and the included angle.
7. The static gesture recognition apparatus according to claim 6, wherein the manner of recognizing the feature value of the target feature point based on the Hu invariant moment algorithm includes:
converting the characteristic value of the target characteristic point into a Hu moment characteristic value by adopting a Hu invariant moment algorithm;
and identifying the characteristic value of the target characteristic point through the converted Hu moment characteristic value to obtain a static gesture identification result.
8. A readable storage medium, wherein a computer program is stored, which when executed implements the static gesture recognition method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810142872.XA CN108133206B (en) | 2018-02-11 | 2018-02-11 | Static gesture recognition method and device and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810142872.XA CN108133206B (en) | 2018-02-11 | 2018-02-11 | Static gesture recognition method and device and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108133206A CN108133206A (en) | 2018-06-08 |
| CN108133206B true CN108133206B (en) | 2020-03-06 |
Family
ID=62430828
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810142872.XA Active CN108133206B (en) | 2018-02-11 | 2018-02-11 | Static gesture recognition method and device and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108133206B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110738118B (en) * | 2019-09-16 | 2023-07-07 | 平安科技（深圳）有限公司 | Gesture recognition method, gesture recognition system, management terminal and computer readable storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140204013A1 (en) * | 2013-01-18 | 2014-07-24 | Microsoft Corporation | Part and state detection for gesture recognition |
| CN103927555B (en) * | 2014-05-07 | 2018-01-05 | 重庆邮电大学 | Static manual alphabet identifying system and method based on Kinect sensor |
| CN106557173B (en) * | 2016-11-29 | 2019-10-18 | 重庆重智机器人研究院有限公司 | Dynamic gesture identification method and device |
-
2018
- 2018-02-11 CN CN201810142872.XA patent/CN108133206B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108133206A (en) | 2018-06-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111178250B (en) | Object identification positioning method and device and terminal equipment | |
| US20220383535A1 (en) | Object Tracking Method and Device, Electronic Device, and Computer-Readable Storage Medium | |
| US11854237B2 (en) | Human body identification method, electronic device and storage medium | |
| CN110807473B (en) | Target detection method, device and computer storage medium | |
| CN107239794B (en) | Point cloud data segmentation method and terminal | |
| TWI821671B (en) | A method and device for positioning text areas | |
| CN109343920B (en) | Image processing method and device, equipment and storage medium thereof | |
| CN110059596B (en) | Image identification method, device, medium and electronic equipment | |
| CN108021847B (en) | Apparatus and method for recognizing facial expression, image processing apparatus and system | |
| CN114677565B (en) | Training method and image processing method and device for feature extraction network | |
| CN111400528B (en) | Image compression method, device, server and storage medium | |
| CN109711374B (en) | Human body bone point identification method and device | |
| US20180247152A1 (en) | Method and apparatus for distance measurement | |
| CN112639872B (en) | Method and device for difficult mining in target detection | |
| CN114972947B (en) | Depth scene text detection method and device based on fuzzy semantic modeling | |
| CN111373393B (en) | Image retrieval method and device and image library generation method and device | |
| CN109375833B (en) | Touch instruction generation method and device | |
| CN108133206B (en) | Static gesture recognition method and device and readable storage medium | |
| CN113610809A (en) | Fracture detection method, fracture detection device, electronic device, and storage medium | |
| CN113627298A (en) | Training method of target detection model and method and device for detecting target object | |
| CN117372415A (en) | Laryngoscope image recognition method, device, computer equipment and storage medium | |
| CN111951211B (en) | Target detection method, device and computer readable storage medium | |
| CN114387600A (en) | Text feature recognition method and device, computer equipment and storage medium | |
| CN113569731A (en) | Intelligent identification method, system, equipment and medium for satellite image tower | |
| CN109146893B (en) | Oil light area segmentation method and device and mobile terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |