WO2023020268A1 - Gesture recognition method and apparatus, and device and medium - Google Patents

Gesture recognition method and apparatus, and device and medium Download PDF

Info

Publication number
WO2023020268A1
WO2023020268A1 PCT/CN2022/109467 CN2022109467W WO2023020268A1 WO 2023020268 A1 WO2023020268 A1 WO 2023020268A1 CN 2022109467 W CN2022109467 W CN 2022109467W WO 2023020268 A1 WO2023020268 A1 WO 2023020268A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
target hand
gesture recognition
hand
preset
Prior art date
Application number
PCT/CN2022/109467
Other languages
French (fr)
Chinese (zh)
Inventor
李海洋
安龙飞
赵晓旭
颜世秦
侯俊杰
聂超
熊巧奇
张新田
王伟
杨文瀚
李进进
王照顺
刘高强
王鹏飞
慕岳衷
Original Assignee
北京有竹居网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京有竹居网络技术有限公司 filed Critical 北京有竹居网络技术有限公司
Publication of WO2023020268A1 publication Critical patent/WO2023020268A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/136Segmentation; Edge detection involving thresholding
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/194Segmentation; Edge detection involving foreground-background segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30196Human being; Person

Definitions

  • the present disclosure relates to the technical field of image recognition, and in particular to a gesture recognition method, device, equipment and medium.
  • gesture recognition As an important part of human-computer interaction, gesture recognition has attracted widespread attention in more and more fields.
  • the method of gesture recognition usually uses sensors to extract hand features to obtain the position information corresponding to the gesture.
  • sensors to extract hand features to obtain the position information corresponding to the gesture.
  • there is interference due to different motion states which makes it impossible to accurately locate and recognize user gestures.
  • the present disclosure provides a gesture recognition method, device, equipment and medium.
  • An embodiment of the present disclosure provides a gesture recognition method, the method comprising:
  • the target image When it is determined based on the horizontal motion stabilization data that the target hand is stable in the horizontal direction, and based on the vertical distance between the target hand and a preset plane, it is determined that the target hand is stable in the vertical direction, then the target image The target hand performs gesture recognition.
  • An embodiment of the present disclosure also provides a gesture recognition device, the device comprising:
  • An image acquisition module configured to acquire a target image
  • the horizontal data module is used to determine the horizontal motion stability data of the target hand by performing motion recognition on the target image
  • a vertical data module configured to determine the vertical distance between the target hand and a preset plane
  • a gesture recognition module configured to determine that the target hand is stable in the horizontal direction based on the horizontal motion stabilization data, and determine that the target hand is stable in the vertical direction based on the vertical distance between the target hand and a preset plane, Then perform gesture recognition on the target hand of the target image.
  • An embodiment of the present disclosure also provides an electronic device, which includes: a processor; a memory for storing instructions executable by the processor; and the processor, for reading the instruction from the memory. Instructions can be executed, and the instructions are executed to implement the gesture recognition method provided by the embodiments of the present disclosure.
  • the embodiment of the present disclosure also provides a computer-readable storage medium, the storage medium stores a computer program, and the computer program is used to execute the gesture recognition method provided by the embodiment of the present disclosure.
  • the gesture recognition solution provided by the embodiment of the present disclosure acquires the target image, and determines the horizontal motion stability data of the target hand by performing motion recognition on the target image, Determine the vertical distance between the target hand and the preset plane; when the target hand is determined to be stable in the horizontal direction based on the horizontal motion stability data, and the target hand is determined to be stable in the vertical direction based on the vertical distance between the target hand and the preset plane, then the The target hand of the target image is used for gesture recognition.
  • the gesture recognition is performed after the hand is stable, so as to avoid the interference caused by the movement interference of the hand in the horizontal and/or vertical direction in the related technology. Larger error, thereby improving the accuracy of gesture recognition.
  • FIG. 1 is a schematic flowchart of a gesture recognition method provided by an embodiment of the present disclosure
  • FIG. 2 is a schematic flowchart of another gesture recognition method provided by an embodiment of the present disclosure
  • FIG. 3 is a schematic diagram of gesture recognition provided by an embodiment of the present disclosure.
  • FIG. 4 is a schematic structural diagram of a gesture recognition device provided by an embodiment of the present disclosure.
  • FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
  • the term “comprise” and its variations are open-ended, ie “including but not limited to”.
  • the term “based on” is “based at least in part on”.
  • the term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one further embodiment”; the term “some embodiments” means “at least some embodiments.” Relevant definitions of other terms will be given in the description below.
  • FIG. 1 is a schematic flow chart of a gesture recognition method provided by an embodiment of the present disclosure.
  • the method can be executed by a gesture recognition device, where the device can be implemented by software and/or hardware, and generally can be integrated into an electronic device.
  • the gesture recognition method in the embodiment of the present disclosure can be applied to any electronic device that needs gesture recognition, for example, the electronic device can be a mobile phone, a tablet computer, a desktop computer, a notebook computer, a vehicle terminal, a wearable device, an all-in-one machine , smart home devices and other devices with communication functions.
  • the method includes:
  • Step 101 acquiring a target image.
  • the target image may be an image including the current user's hand collected by a preset image collector, or an image frame including the current user's hand extracted from a video.
  • Target images can include extracted RGB image frames and depth images from videos.
  • Embodiments of the present disclosure are not limited to specific image collectors, and different types of image collectors are used to collect corresponding images, for example, depth image collectors are used to collect the above-mentioned depth images.
  • Step 102 Determine the horizontal movement stability data of the target hand by performing motion recognition on the target image.
  • the motion recognition may be the recognition of the motion state of the hand in the target image, specifically the recognition of the stability of the horizontal motion state, and the horizontal motion stability data may be the result of the recognition.
  • the target image includes the current RGB image frame and the previous RGB image frame extracted from the video
  • determining the horizontal motion stabilization data of the target hand includes: based on the current RGB image frame and For the previous RGB image frame, the optical flow algorithm is used to calculate the optical flow field of the current RGB image frame, and the optical flow field is thresholded to obtain the foreground area and background area including the target hand; the velocity vector of the foreground area and the background area The velocity vector of is determined as the horizontal motion stabilization data of the target hand.
  • Optical flow is the instantaneous velocity of the pixel movement of a spatially moving object on the observation imaging plane.
  • the time interval is small, it is approximately equivalent to the displacement of the target point.
  • the instantaneous rate of change of gray level at a specific coordinate point on a two-dimensional image plane is defined as the optical flow vector.
  • the optical flow method uses the changes of pixels in the image sequence in the time domain and the correlation between adjacent frames to find the corresponding relationship between the previous frame and the current frame, thereby calculating the motion of objects between adjacent frames.
  • Optical flow contains information about object motion.
  • the optical flow field is a two-dimensional vector field, which reflects the change trend of each gray point on the image, and can be regarded as the instantaneous velocity field generated by the movement of gray pixels on the image plane.
  • the information it contains is the instantaneous motion velocity vector information of each pixel.
  • an optical flow field corresponds to a motion field.
  • the gesture recognition device can use the optical flow algorithm for motion recognition. Specifically, after resampling and denoising preprocessing of the current RGB image frame and the previous RGB image frame, the optical flow method can be used to calculate the current RGB image frame. The optical flow value of each point is obtained, and then the optical flow field is thresholded to distinguish the foreground area and the background area. The foreground area includes the target hand. After that, the velocity vector of the foreground area and the velocity vector of the background area can be combined The velocity vector is determined as the horizontal motion stabilization data of the target hand.
  • the gesture recognition device can also use a continuous adaptive MeanShift (Continuously Adaptive Mean-SHIFT, CamShift) algorithm or an active contour tracking algorithm to perform motion recognition on the target image, and the specific process will not be repeated here.
  • a continuous adaptive MeanShift Continuous Adaptive Mean-SHIFT, CamShift
  • an active contour tracking algorithm to perform motion recognition on the target image, and the specific process will not be repeated here.
  • Step 103 determining the vertical distance between the target hand and the preset plane.
  • the preset plane may be a plane where the electronic device currently performing gesture recognition is located, and the preset plane may be a horizontal plane or a vertical plane, which is specifically determined according to an actual scene.
  • the preset plane is the horizontal plane where the electronic device is located; or, when the electronic device is placed vertically and gesture recognition needs to be performed on the user in front, the preset plane is The vertical plane where the electronic equipment is located, such as a vertical wall.
  • the vertical distance between the target hand and the preset plane can be determined in various ways, for example, it can be determined by extracting a depth image or based on a distance sensor, which is only an example and not a limitation.
  • the target image includes a first depth image at the first moment and a second depth image at the second moment
  • determining the vertical distance between the target hand and the preset plane includes: based on the first depth image and the second depth image, respectively extracting the first vertical distance and the second vertical distance between the target hand and the preset plane at the first moment and the second moment, and both the first depth image and the second depth image include the target hand and the preset plane.
  • the second moment is after the first moment, and there is a preset time interval between the first moment and the second moment, for example, 30 seconds may be detected.
  • Depth image is also called distance image, which refers to the image with the distance (depth) from the image collector to each point in the scene as the pixel value, which directly reflects the geometry of the visible surface of the object.
  • the gesture recognition device can acquire a first depth image and a second depth image including the target hand and a preset plane through the depth sensor, the first depth image corresponds to the first moment, and the second depth image corresponds to the second moment. Then, the first vertical distance and the second vertical distance between the preset point in the target hand and the preset plane at the first moment and the second moment can be respectively extracted from the first depth image and the second depth image.
  • the preset point can be set as a position point in the target hand, for example, the preset point can be the fingertip or palm center of any finger of the target hand.
  • determining the vertical distance between the target hand and the preset plane may include: using a distance sensor to respectively determine the first vertical distance and the second vertical distance between the target hand and the preset plane at the first moment and the second moment distance.
  • the distance sensor may be a sensor for sensing the distance between it and an object, and the distance sensor in the embodiment of the present disclosure may be set in the above-mentioned electronic device.
  • the gesture recognition device may also acquire the vertical distances between the target hand and the preset plane collected by the distance sensor at the first moment and the second moment.
  • Step 104 when it is determined that the target hand is stable in the horizontal direction based on the horizontal motion stabilization data, and it is determined that the target hand is stable in the vertical direction based on the vertical distance between the target hand and the preset plane, perform gesture recognition on the target hand in the target image .
  • gesture recognition is performed when the hand is determined to be stable in both horizontal and vertical directions.
  • determining that the target hand is stable in the horizontal direction based on the horizontal motion stabilization data may include: if the difference between the velocity vector of the foreground area and the velocity vector of the background area is less than a preset threshold, determining that the target hand is stable in the horizontal direction Steady direction.
  • the difference between the velocity vectors between the foreground area and the background area can be determined in the embodiment of the present disclosure value, and the difference is compared with the preset threshold, if the difference is less than the preset threshold, it is determined that the movement range of the target hand in the horizontal direction is very small, and it is considered stable.
  • determining that the target hand is stable in the vertical direction based on the vertical distance between the target hand and the preset plane includes: if the difference between the first vertical distance and the second vertical distance between the target hand and the preset plane If the value is smaller than the first preset difference, it is determined that the target hand is stable in the vertical direction.
  • the first preset difference can be set according to actual conditions, for example, the preset difference can be 1 cm.
  • the distance between the first vertical distance and the second vertical distance may be determined and compare the difference with the first preset difference. If the difference is smaller than the first preset difference, it means that the target hand is stable within a small distance in the vertical direction, and then the target is determined. The hand is stabilized in the vertical direction.
  • the gesture recognition device can determine whether the target hand is stable in the horizontal direction according to the horizontal movement stability data of the target hand, And judging whether the target hand is stable in the vertical direction according to the vertical distance between the target hand and the preset plane, if it is determined that the target hand is stable in both the horizontal direction and the vertical direction, gesture recognition can be performed on the target hand of the target image, specifically There may be various gesture recognition manners adopted, which are not limited in this embodiment of the present disclosure.
  • it before performing gesture recognition on the target hand of the target image, it further includes: judging whether the difference between the vertical distance between the target hand and the preset plane and the preset distance is smaller than the second preset difference; When the difference between the vertical distance between the target hand and the preset plane and the preset distance is smaller than the second preset difference, perform gesture recognition on the target hand in the target image.
  • the preset distance can be a preset recognition distance between a hand and a preset plane, which can be set according to actual use scenarios.
  • the preset distance can be Farther, for example, the preset distance may be 10 cm; and the preset distance may be shorter when performing gesture recognition in a point-and-read scene.
  • the difference between the vertical distance and the preset distance can be determined, and the difference can be compared with the second preset difference , if the difference is smaller than the second preset difference, it means that the target hand meets a distance requirement for gesture recognition, and then gesture recognition can be performed on the target image.
  • the second preset difference may be the same as or different from the above-mentioned first preset difference.
  • performing gesture recognition on the target hand of the target image may include: performing gesture segmentation and feature extraction on the target image, and then performing gesture recognition using a gesture recognition algorithm based on the extracted features.
  • the aforementioned preset gesture recognition algorithm may include a template matching algorithm, a statistical analysis algorithm, a neural network algorithm, etc., and is not specifically limited.
  • the gesture recognition device can perform gesture segmentation on the target image, specifically, threshold method, edge detection method, or physical feature method can be used to perform gesture segmentation; then feature extraction can be performed on the segmented gesture area, and the extracted features It can include contours, edges, image moments, image feature vectors, and regional histogram features, etc., and is not limited in detail; then based on the extracted features, a preset gesture recognition algorithm can be used for gesture recognition to obtain the final recognition result.
  • the gesture recognition scheme acquires the target image, and determines the horizontal motion stability data of the target hand by performing motion recognition on the target image, and determines the vertical distance between the target hand and the preset plane; when based on the horizontal motion stability data If it is determined that the target hand is stable in the horizontal direction, and based on the vertical distance between the target hand and the preset plane, it is determined that the target hand is stable in the vertical direction, and gesture recognition is performed on the target hand in the target image.
  • the gesture recognition is performed after the hand is stable, so as to avoid the interference caused by the movement interference of the hand in the horizontal and/or vertical direction in the related technology. Larger error, thereby improving the accuracy of gesture recognition.
  • FIG. 2 is a schematic flowchart of another gesture recognition method provided by an embodiment of the present disclosure. On the basis of the above embodiments, this embodiment further specifically describes the above gesture recognition method. As shown in Figure 2, the method includes:
  • Step 201 acquiring a target image.
  • the target image may include an RGB image frame and a depth image.
  • step 202-step 203 can be executed first, and then step 204-step 205 can be executed; step 204-step 205 can also be executed first; step 202-step 203 can be executed first; step 202-step 203 can also be executed first Step 202 and step 204 (the sequence is not limited), and then execute step 203 and step 205 (the sequence is not limited), which is determined according to the actual situation.
  • the order of execution in Figure 2 is just an example.
  • Step 202 Determine the horizontal motion stability data of the target hand by performing motion recognition on the target image.
  • the target image includes the current RGB image frame and the previous RGB image frame extracted from the video
  • the horizontal motion stabilization data of the target hand is determined by performing motion recognition on the target image, including: based on the current RGB image frame and the previous RGB image frame RGB image frame, using the optical flow algorithm to calculate the optical flow field of the current RGB image frame, and thresholding the optical flow field to obtain the foreground area and background area including the target hand; the velocity vector of the foreground area and the velocity of the background area The vector is determined as the horizontal motion stabilization data of the target hand.
  • Step 203 determine whether the target hand is stable in the horizontal direction based on the horizontal motion stability data, if yes, execute step 204 ; otherwise, return to execute step 201 .
  • step 204 if the difference between the velocity vector of the foreground area and the velocity vector of the background area is less than the preset threshold, it is determined that the target hand is stable in the horizontal direction, and step 204 is performed; if the velocity vector of the foreground area and the velocity vector of the background area If the difference is greater than or equal to the preset threshold, it is determined that the target hand is unstable in the horizontal direction, and the execution returns to step 201 .
  • Step 204 determining the vertical distance between the target hand and the preset plane.
  • the preset plane is a horizontal plane or a vertical plane.
  • the target image includes a first depth image at the first moment and a second depth image at the second moment
  • determining the vertical distance between the target hand and the preset plane includes: based on the first depth image and the second depth image, The first vertical distance and the second vertical distance between the target hand and the preset plane are respectively extracted at the first moment and the second moment, and both the first depth image and the second depth image include the target hand and the preset plane.
  • determining the vertical distance between the target hand and the preset plane includes: using a distance sensor to respectively determine a first vertical distance and a second vertical distance between the target hand and the preset plane at the first moment and the second moment.
  • Step 205 determine whether the target hand is stable in the vertical direction based on the vertical distance between the target hand and the preset plane, if yes, perform step 206 ; otherwise, return to step 201 .
  • step 206 can be performed; If the difference between the first vertical distance and the second vertical distance between the target hand and the preset plane is greater than or equal to the first preset difference, it is determined that the target hand is unstable in the vertical direction, and then the execution of step 201 may be returned .
  • Step 206 judging whether the difference between the vertical distance between the target hand and the preset plane and the preset distance is smaller than the second preset difference, if yes, go to step 207 ; otherwise, go back to step 201 .
  • Step 207 performing gesture recognition on the target hand in the target image.
  • performing gesture recognition on the target hand of the target image may include: performing gesture segmentation and feature extraction on the target image, and performing gesture recognition using a preset gesture recognition algorithm based on the extracted features.
  • FIG. 3 is a schematic diagram of a gesture recognition provided by an embodiment of the present disclosure.
  • the gesture recognition process may include: Step 21, start. Step 22, acquiring the RGB image frame and the depth image in the video. That is, the above-mentioned target image is acquired, and the target image includes an RGB image frame and a depth image.
  • Step 23 Based on the current RGB image frame and the previous RGB image frame, the optical flow algorithm is used to perform motion recognition on the current RGB image frame, and determine the horizontal motion stability data of the target hand.
  • Step 24 Determine whether the target hand is stable in the horizontal direction based on the horizontal movement stability data, if yes, execute step 25; otherwise, return to execute step 22. Step 25.
  • Step 26 Determine the vertical distance between the target hand and the preset plane based on the depth image.
  • Step 26 Determine whether the target hand is stable in the vertical direction based on the vertical distance between the target hand and the preset plane. If yes, perform step 27; otherwise, return to step 22.
  • Step 27 whether the vertical distance between the target hand and the preset plane reaches the preset distance, if yes, go to step 28; otherwise, go back to step 22.
  • step 28 is executed; otherwise, step 22 is executed.
  • gesture recognition When the target hand is stable in both the horizontal direction and the vertical direction, and the target hand reaches the preset distance from the preset plane, gesture recognition is started. Step 29, subsequent processing. Specifically, the gesture recognized in real time may be matched with a preset gesture, and if the matching is successful, the gesture recognition is completed. Step 30, end.
  • the horizontal motion recognition of the hand is carried out through the optical flow algorithm, and the vertical distance between the hand and the measured plane is determined based on the depth information, and then the gesture is performed when the target hand is stable in both the horizontal and vertical directions. Recognition, gesture recognition results with higher accuracy can be obtained.
  • the gesture recognition scheme acquires the target image, and determines the horizontal motion stability data of the target hand by performing motion recognition on the target image, and determines the vertical distance between the target hand and the preset plane; when based on the horizontal motion stability data If it is determined that the target hand is stable in the horizontal direction, and based on the vertical distance between the target hand and the preset plane, it is determined that the target hand is stable in the vertical direction, and gesture recognition is performed on the target hand in the target image.
  • the gesture recognition is performed after the hand is stable, so as to avoid the interference caused by the movement interference of the hand in the horizontal and/or vertical direction in the related technology. Larger error, thereby improving the accuracy of gesture recognition.
  • FIG. 4 is a schematic structural diagram of a gesture recognition device provided by an embodiment of the present disclosure.
  • the device can be implemented by software and/or hardware, and generally can be integrated into an electronic device. As shown in Figure 4, the device includes:
  • An image acquisition module 301 configured to acquire a target image
  • the horizontal data module 302 is used to determine the horizontal motion stability data of the target hand by performing motion recognition on the target image
  • a vertical data module 303 configured to determine the vertical distance between the target hand and a preset plane
  • Gesture recognition module 304 configured to determine that the target hand is stable in the horizontal direction based on the horizontal motion stability data, and determine that the target hand is stable in the vertical direction based on the vertical distance between the target hand and a preset plane , performing gesture recognition on the target hand of the target image.
  • the target image includes the current RGB image frame and the previous RGB image frame extracted from the video
  • the horizontal data module 302 is specifically used for:
  • an optical flow algorithm is used to calculate the optical flow field of the current RGB image frame, and the optical flow field is thresholded to obtain the target hand the foreground and background regions of
  • the velocity vector of the foreground area and the velocity vector of the background area are determined as the horizontal motion stabilization data of the target hand.
  • the gesture recognition module 304 is specifically configured to:
  • the difference between the velocity vector of the foreground area and the velocity vector of the background area is smaller than a preset threshold, it is determined that the target hand is stable in the horizontal direction.
  • the target image includes a first depth image at a first moment and a second depth image at a second moment
  • the vertical data module 303 is specifically used for:
  • the first Both the depth image and the second depth image include the target hand and the preset plane.
  • the vertical data module 303 is specifically used for:
  • a distance sensor is used to determine a first vertical distance and a second vertical distance between the target hand and the preset plane at the first moment and the second moment respectively.
  • the gesture recognition module 304 is specifically configured to:
  • the difference between the first vertical distance and the second vertical distance between the target hand and the preset plane is smaller than a first preset difference, it is determined that the target hand is stable in the vertical direction .
  • the preset plane is a horizontal plane or a vertical plane.
  • the device further includes a vertical judging module, configured to: before performing gesture recognition on the target hand of the target image,
  • the gesture recognition module 304 is specifically configured to:
  • gesture recognition is performed using a preset gesture recognition algorithm based on the extracted features.
  • the gesture recognition device provided by the embodiments of the present disclosure can execute the gesture recognition method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.
  • An embodiment of the present disclosure further provides a computer program product, including a computer program/instruction, and when the computer program/instruction is executed by a processor, the gesture recognition method provided in any embodiment of the present disclosure is implemented.
  • FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. Referring specifically to FIG. 5 , it shows a schematic structural diagram of an electronic device 400 suitable for implementing an embodiment of the present disclosure.
  • the electronic device 400 in the embodiment of the present disclosure may include, but is not limited to, mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), vehicle-mounted terminals ( Mobile terminals such as car navigation terminals) and stationary terminals such as digital TVs, desktop computers and the like.
  • the electronic device shown in FIG. 5 is only an example, and should not limit the functions and scope of use of the embodiments of the present disclosure.
  • an electronic device 400 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 401, which may be randomly accessed according to a program stored in a read-only memory (ROM) 402 or loaded from a storage device 408. Various appropriate actions and processes are executed by programs in the memory (RAM) 403 . In the RAM 403, various programs and data necessary for the operation of the electronic device 400 are also stored.
  • the processing device 401 , ROM 402 and RAM 403 are connected to each other through a bus 404 .
  • An input/output (I/O) interface 405 is also connected to bus 404 .
  • the following devices can be connected to the I/O interface 405: input devices 406 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speaker, vibration an output device 407 such as a computer; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409.
  • the communication means 409 may allow the electronic device 400 to perform wireless or wired communication with other devices to exchange data. While FIG. 5 shows electronic device 400 having various means, it should be understood that implementing or having all of the means shown is not a requirement. More or fewer means may alternatively be implemented or provided.
  • embodiments of the present disclosure include a computer program product, which includes a computer program carried on a non-transitory computer readable medium, where the computer program includes program code for executing the method shown in the flowchart.
  • the computer program may be downloaded and installed from a network via communication means 409, or from storage means 408, or from ROM 402.
  • the processing device 401 When the computer program is executed by the processing device 401, the above-mentioned functions defined in the gesture recognition method of the embodiment of the present disclosure are executed.
  • the above-mentioned computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two.
  • a computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device .
  • Program code embodied on a computer readable medium may be transmitted by any appropriate medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
  • the client and the server can communicate using any currently known or future network protocols such as HTTP (HyperText Transfer Protocol, Hypertext Transfer Protocol), and can communicate with digital data in any form or medium Communications (eg, communication networks) are interconnected.
  • Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network of.
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device.
  • the above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device: acquires the target image; determines the target hand by performing motion recognition on the target image The horizontal motion stabilization data; determine the vertical distance between the target hand and the preset plane; when it is determined based on the horizontal motion stabilization data that the target hand is stable in the horizontal direction, and based on the target hand and the preset plane If the vertical distance of the target hand is determined to be stable in the vertical direction, gesture recognition is performed on the target hand in the target image.
  • Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, or combinations thereof, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and Included are conventional procedural programming languages - such as the "C" language or similar programming languages.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through an Internet service provider). Internet connection).
  • LAN local area network
  • WAN wide area network
  • Internet service provider such as AT&T, MCI, Sprint, EarthLink, MSN, GTE, etc.
  • each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. Wherein, the name of a unit does not constitute a limitation of the unit itself under certain circumstances.
  • FPGAs Field Programmable Gate Arrays
  • ASICs Application Specific Integrated Circuits
  • ASSPs Application Specific Standard Products
  • SOCs System on Chips
  • CPLD Complex Programmable Logical device
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing.
  • machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read only memory
  • EPROM or flash memory erasable programmable read only memory
  • CD-ROM compact disk read only memory
  • magnetic storage or any suitable combination of the foregoing.
  • the present disclosure provides a gesture recognition method, including:
  • the target image When it is determined based on the horizontal motion stabilization data that the target hand is stable in the horizontal direction, and based on the vertical distance between the target hand and a preset plane, it is determined that the target hand is stable in the vertical direction, then the target image The target hand performs gesture recognition.
  • the target image includes the current RGB image frame and the previous RGB image frame extracted from the video, and by performing motion recognition on the target image, Determine horizontal motion stabilization data for the target hand, including:
  • an optical flow algorithm is used to calculate the optical flow field of the current RGB image frame, and the optical flow field is thresholded to obtain the target hand the foreground and background regions of
  • the velocity vector of the foreground area and the velocity vector of the background area are determined as the horizontal motion stabilization data of the target hand.
  • determining that the target hand is stable in the horizontal direction based on the horizontal motion stability data includes:
  • the difference between the velocity vector of the foreground area and the velocity vector of the background area is smaller than a preset threshold, it is determined that the target hand is stable in the horizontal direction.
  • the target image includes a first depth image at a first moment and a second depth image at a second moment, and the target hand and The vertical distance of the preset plane, including:
  • the first Both the depth image and the second depth image include the target hand and the preset plane.
  • determining the vertical distance between the target hand and the preset plane includes:
  • a distance sensor is used to determine a first vertical distance and a second vertical distance between the target hand and the preset plane at the first moment and the second moment respectively.
  • determining that the target hand is stable in the vertical direction based on the vertical distance between the target hand and a preset plane includes:
  • the difference between the first vertical distance and the second vertical distance between the target hand and the preset plane is smaller than a first preset difference, it is determined that the target hand is stable in the vertical direction .
  • the preset plane is a horizontal plane or a vertical plane.
  • the gesture recognition method provided in the present disclosure before performing gesture recognition on the target hand of the target image, further includes:
  • performing gesture recognition on the target hand of the target image includes:
  • gesture recognition is performed using a preset gesture recognition algorithm based on the extracted features.
  • the present disclosure provides a gesture recognition device, including:
  • An image acquisition module configured to acquire a target image
  • the horizontal data module is used to determine the horizontal motion stability data of the target hand by performing motion recognition on the target image
  • a vertical data module configured to determine the vertical distance between the target hand and a preset plane
  • the gesture recognition module is used to determine that the target hand is stable in the horizontal direction based on the horizontal motion stability data, and determine that the target hand is stable in the vertical direction based on the vertical distance between the target hand and a preset plane, Then perform gesture recognition on the target hand of the target image.
  • the target image includes the current RGB image frame and the previous RGB image frame extracted from the video
  • the horizontal data module is specifically used for:
  • an optical flow algorithm is used to calculate the optical flow field of the current RGB image frame, and the optical flow field is thresholded to obtain the target hand the foreground and background regions of
  • the velocity vector of the foreground area and the velocity vector of the background area are determined as the horizontal motion stabilization data of the target hand.
  • the gesture recognition module is specifically used for:
  • the difference between the velocity vector of the foreground area and the velocity vector of the background area is smaller than a preset threshold, it is determined that the target hand is stable in the horizontal direction.
  • the target image includes a first depth image at a first moment and a second depth image at a second moment
  • the vertical data module is specifically used At:
  • the first Both the depth image and the second depth image include the target hand and the preset plane.
  • the vertical data module is specifically used for:
  • a distance sensor is used to determine a first vertical distance and a second vertical distance between the target hand and the preset plane at the first moment and the second moment respectively.
  • the gesture recognition module is specifically used for:
  • the difference between the first vertical distance and the second vertical distance between the target hand and the preset plane is smaller than a first preset difference, it is determined that the target hand is stable in the vertical direction .
  • the preset plane is a horizontal plane or a vertical plane.
  • the device further includes a vertical judgment module, configured to: before performing gesture recognition on the target hand of the target image,
  • the gesture recognition module is specifically used for:
  • gesture recognition is performed using a preset gesture recognition algorithm based on the extracted features.
  • the present disclosure provides an electronic device, including:
  • the processor is configured to read the executable instructions from the memory, and execute the instructions to implement any gesture recognition method provided in the present disclosure.
  • the present disclosure provides a computer-readable storage medium, the storage medium stores a computer program, and the computer program is used to perform any of the gestures provided in the present disclosure recognition methods.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • User Interface Of Digital Computer (AREA)
  • Image Analysis (AREA)

Abstract

The embodiments of the present disclosure relate to a gesture recognition method and apparatus, and a device and a medium. The method comprises: acquiring a target image; determining horizontal motion stability data of a target hand by means of performing motion recognition on the target image; determining the vertical distance between the target hand and a preset plane; and when it is determined, on the basis of the horizontal motion stability data, that the target hand is stable in the horizontal direction, and it is determined, on the basis of the vertical distance between the target hand and the preset plane, that the target hand is stable in the vertical direction, performing gesture recognition on the target hand of the target image. By means of the technical solution, the stability in the horizontal direction and in the vertical direction is determined before gesture recognition is performed, and gesture recognition is performed after a hand becomes stable, thereby avoiding the relatively large error caused by motion interference of the hand in the horizontal and/or vertical direction in the related art, and thus improving the accuracy of gesture recognition.

Description

一种手势识别方法、装置、设备及介质Gesture recognition method, device, equipment and medium
相关申请的交叉引用Cross References to Related Applications
本申请基于申请号为202110962932.4、申请日为2021年08月20日,名称为“一种手势识别方法、装置、设备及介质”的中国专利申请提出,并要求该中国专利申请的优先权,该中国专利申请的全部内容在此引入本申请作为参考。This application is based on the Chinese patent application with the application number 202110962932.4 and the filing date of August 20, 2021, entitled "A Gesture Recognition Method, Device, Equipment, and Medium", and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby incorporated by reference into this application.
技术领域technical field
本公开涉及图像识别技术领域,尤其涉及一种手势识别方法、装置、设备及介质。The present disclosure relates to the technical field of image recognition, and in particular to a gesture recognition method, device, equipment and medium.
背景技术Background technique
手势识别作为人机交互的一个重要部分,在越来越多的领域引起了广泛关注。As an important part of human-computer interaction, gesture recognition has attracted widespread attention in more and more fields.
目前,手势识别的方法通常是使用传感器对手部特征进行提取,从而获得手势对应的位置信息,但是实际识别过程中,存在因不同运动状态的干扰,导致无法对用户手势进行准确定位和识别。At present, the method of gesture recognition usually uses sensors to extract hand features to obtain the position information corresponding to the gesture. However, in the actual recognition process, there is interference due to different motion states, which makes it impossible to accurately locate and recognize user gestures.
发明内容Contents of the invention
为了解决上述技术问题或者至少部分地解决上述技术问题,本公开提供了一种手势识别方法、装置、设备及介质。In order to solve the above technical problems or at least partly solve the above technical problems, the present disclosure provides a gesture recognition method, device, equipment and medium.
本公开实施例提供了一种手势识别方法,所述方法包括:An embodiment of the present disclosure provides a gesture recognition method, the method comprising:
获取目标图像;Get the target image;
通过对所述目标图像进行运动识别,确定目标手部的水平运动稳定数据;Determine the horizontal motion stability data of the target hand by performing motion recognition on the target image;
确定所述目标手部与预设平面的垂直距离;determining the vertical distance between the target hand and a preset plane;
当基于所述水平运动稳定数据确定所述目标手部在水平方向稳定,并且基于所述目标手部与预设平面的垂直距离确定所述目标手部在垂直方向稳定,则对所述目标图像的所述目标手部进行手势识别。When it is determined based on the horizontal motion stabilization data that the target hand is stable in the horizontal direction, and based on the vertical distance between the target hand and a preset plane, it is determined that the target hand is stable in the vertical direction, then the target image The target hand performs gesture recognition.
本公开实施例还提供了一种手势识别装置,所述装置包括:An embodiment of the present disclosure also provides a gesture recognition device, the device comprising:
图像获取模块,用于获取目标图像;An image acquisition module, configured to acquire a target image;
水平数据模块,用于通过对所述目标图像进行运动识别,确定目标手部的水平运动稳定数据;The horizontal data module is used to determine the horizontal motion stability data of the target hand by performing motion recognition on the target image;
垂直数据模块,用于确定所述目标手部与预设平面的垂直距离;a vertical data module, configured to determine the vertical distance between the target hand and a preset plane;
手势识别模块,用于当基于所述水平运动稳定数据确定所述目标手部在水平方向稳定, 并且基于所述目标手部与预设平面的垂直距离确定所述目标手部在垂直方向稳定,则对所述目标图像的所述目标手部进行手势识别。A gesture recognition module, configured to determine that the target hand is stable in the horizontal direction based on the horizontal motion stabilization data, and determine that the target hand is stable in the vertical direction based on the vertical distance between the target hand and a preset plane, Then perform gesture recognition on the target hand of the target image.
本公开实施例还提供了一种电子设备,所述电子设备包括:处理器;用于存储所述处理器可执行指令的存储器;所述处理器,用于从所述存储器中读取所述可执行指令,并执行所述指令以实现如本公开实施例提供的手势识别方法。An embodiment of the present disclosure also provides an electronic device, which includes: a processor; a memory for storing instructions executable by the processor; and the processor, for reading the instruction from the memory. Instructions can be executed, and the instructions are executed to implement the gesture recognition method provided by the embodiments of the present disclosure.
本公开实施例还提供了一种计算机可读存储介质,所述存储介质存储有计算机程序,所述计算机程序用于执行如本公开实施例提供的手势识别方法。The embodiment of the present disclosure also provides a computer-readable storage medium, the storage medium stores a computer program, and the computer program is used to execute the gesture recognition method provided by the embodiment of the present disclosure.
本公开实施例提供的技术方案与现有技术相比具有如下优点:本公开实施例提供的手势识别方案,获取目标图像,通过对目标图像进行运动识别,确定目标手部的水平运动稳定数据,确定目标手部与预设平面的垂直距离;当基于水平运动稳定数据确定目标手部在水平方向稳定,并且基于目标手部与预设平面的垂直距离确定目标手部在垂直方向稳定,则对目标图像的目标手部进行手势识别。采用上述技术方案,通过在手势识别之前对水平方向和垂直方向的稳定判断,在手部稳定之后再进行手势识别,避免相关技术中因手部在水平和/或垂直方向的运动干扰而造成的较大误差,进而提升了手势识别的准确率。Compared with the prior art, the technical solution provided by the embodiment of the present disclosure has the following advantages: the gesture recognition solution provided by the embodiment of the present disclosure acquires the target image, and determines the horizontal motion stability data of the target hand by performing motion recognition on the target image, Determine the vertical distance between the target hand and the preset plane; when the target hand is determined to be stable in the horizontal direction based on the horizontal motion stability data, and the target hand is determined to be stable in the vertical direction based on the vertical distance between the target hand and the preset plane, then the The target hand of the target image is used for gesture recognition. By adopting the above-mentioned technical solution, through the stable judgment of the horizontal direction and the vertical direction before the gesture recognition, the gesture recognition is performed after the hand is stable, so as to avoid the interference caused by the movement interference of the hand in the horizontal and/or vertical direction in the related technology. Larger error, thereby improving the accuracy of gesture recognition.
附图说明Description of drawings
结合附图并参考以下具体实施方式,本公开各实施例的上述和其他特征、优点及方面将变得更加明显。贯穿附图中,相同或相似的附图标记表示相同或相似的元素。应当理解附图是示意性的,原件和元素不一定按照比例绘制。The above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic and that elements and elements are not necessarily drawn to scale.
图1为本公开实施例提供的一种手势识别方法的流程示意图;FIG. 1 is a schematic flowchart of a gesture recognition method provided by an embodiment of the present disclosure;
图2为本公开实施例提供的另一种手势识别方法的流程示意图;FIG. 2 is a schematic flowchart of another gesture recognition method provided by an embodiment of the present disclosure;
图3为本公开实施例提供的一种手势识别的示意图;FIG. 3 is a schematic diagram of gesture recognition provided by an embodiment of the present disclosure;
图4为本公开实施例提供的一种手势识别装置的结构示意图;FIG. 4 is a schematic structural diagram of a gesture recognition device provided by an embodiment of the present disclosure;
图5为本公开实施例提供的一种电子设备的结构示意图。FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
具体实施方式Detailed ways
下面将参照附图更详细地描述本公开的实施例。虽然附图中显示了本公开的某些实施例,然而应当理解的是,本公开可以通过各种形式来实现,而且不应该被解释为限于这里阐述的实施例,相反提供这些实施例是为了更加透彻和完整地理解本公开。应当理解的是,本公开的附图及实施例仅用于示例性作用,并非用于限制本公开的保护范围。Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the drawings, it should be understood that the disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein; A more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for exemplary purposes only, and are not intended to limit the protection scope of the present disclosure.
应当理解,本公开的方法实施方式中记载的各个步骤可以按照不同的顺序执行,和/或并行执行。此外,方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。本公开的范围 在此方面不受限制。It should be understood that the various steps described in the method implementations of the present disclosure may be executed in different orders, and/or executed in parallel. Additionally, method embodiments may include additional steps and/or omit performing illustrated steps. The scope of the present disclosure is not limited in this regard.
本文使用的术语“包括”及其变形是开放性包括,即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”;术语“另一实施例”表示“至少一个另外的实施例”;术语“一些实施例”表示“至少一些实施例”。其他术语的相关定义将在下文描述中给出。As used herein, the term "comprise" and its variations are open-ended, ie "including but not limited to". The term "based on" is "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one further embodiment"; the term "some embodiments" means "at least some embodiments." Relevant definitions of other terms will be given in the description below.
需要注意,本公开中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分,并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。It should be noted that concepts such as "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the sequence of functions performed by these devices, modules or units or interdependence.
需要注意,本公开中提及的“一个”、“多个”的修饰是示意性而非限制性的,本领域技术人员应当理解,除非在上下文另有明确指出,否则应该理解为“一个或多个”。It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative and not restrictive, and those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more" multiple".
本公开实施方式中的多个装置之间所交互的消息或者信息的名称仅用于说明性的目的,而并不是用于对这些消息或信息的范围进行限制。The names of messages or information exchanged between multiple devices in the embodiments of the present disclosure are used for illustrative purposes only, and are not used to limit the scope of these messages or information.
图1为本公开实施例提供的一种手势识别方法的流程示意图,该方法可以由手势识别装置执行,其中该装置可以采用软件和/或硬件实现,一般可集成在电子设备中。本公开实施例中的手势识别方法可以应用于任意一种需要进行手势识别的电子设备中,例如电子设备可以是移动电话、平板电脑、台式计算机、笔记本电脑、车载终端、可穿戴设备、一体机、智能家居设备等具有通信功能的设备。FIG. 1 is a schematic flow chart of a gesture recognition method provided by an embodiment of the present disclosure. The method can be executed by a gesture recognition device, where the device can be implemented by software and/or hardware, and generally can be integrated into an electronic device. The gesture recognition method in the embodiment of the present disclosure can be applied to any electronic device that needs gesture recognition, for example, the electronic device can be a mobile phone, a tablet computer, a desktop computer, a notebook computer, a vehicle terminal, a wearable device, an all-in-one machine , smart home devices and other devices with communication functions.
如图1所示,该方法包括:As shown in Figure 1, the method includes:
步骤101、获取目标图像。 Step 101, acquiring a target image.
其中,目标图像可以是预设的图像采集器采集得到的包括当前用户的手部的图像,或者视频中提取的包括当前用户的手部的图像帧。目标图像可以包括视频中提取的RGB图像帧和深度图像。本公开实施例对具体的图像采集器不限,分别采用不同类型的图像采集器件采集对应的图像,例如采用深度图像采集器采集上述深度图像。Wherein, the target image may be an image including the current user's hand collected by a preset image collector, or an image frame including the current user's hand extracted from a video. Target images can include extracted RGB image frames and depth images from videos. Embodiments of the present disclosure are not limited to specific image collectors, and different types of image collectors are used to collect corresponding images, for example, depth image collectors are used to collect the above-mentioned depth images.
步骤102、通过对目标图像进行运动识别,确定目标手部的水平运动稳定数据。Step 102: Determine the horizontal movement stability data of the target hand by performing motion recognition on the target image.
其中,运动识别可以是对目标图像中手部的运动状态的识别,具体是对水平运动状态的稳定性的识别,水平运动稳定数据可以是识别得到的结果。The motion recognition may be the recognition of the motion state of the hand in the target image, specifically the recognition of the stability of the horizontal motion state, and the horizontal motion stability data may be the result of the recognition.
在一些实施例中,目标图像包括视频中提取的当前RGB图像帧和上一RGB图像帧,通过对目标图像进行运动识别,确定目标手部的水平运动稳定数据,包括:基于当前RGB图像帧和上一RGB图像帧,采用光流算法计算当前RGB图像帧的光流场,并对光流场进行阈值分割,得到包括目标手部的前景区域和背景区域;将前景区域的速度矢量和背景区域的速度矢量确定为目标手部的水平运动稳定数据。In some embodiments, the target image includes the current RGB image frame and the previous RGB image frame extracted from the video, and by performing motion recognition on the target image, determining the horizontal motion stabilization data of the target hand includes: based on the current RGB image frame and For the previous RGB image frame, the optical flow algorithm is used to calculate the optical flow field of the current RGB image frame, and the optical flow field is thresholded to obtain the foreground area and background area including the target hand; the velocity vector of the foreground area and the background area The velocity vector of is determined as the horizontal motion stabilization data of the target hand.
光流(optical flow)是空间运动物体在观察成像平面上的像素运动的瞬时速度,在时间间隔很小时,约等同于目标点的位移。通常将二维图像平面特定坐标点上的灰度瞬时变化率定义为光流矢量。光流法是利用图像序列中像素在时间域上的变化以及相邻帧之间的相关性来找到上一帧跟当前帧之间存在的对应关系,从而计算出相邻帧之间物体的运动信息的一种方法。光流中包含了物体运动的相关信息。Optical flow is the instantaneous velocity of the pixel movement of a spatially moving object on the observation imaging plane. When the time interval is small, it is approximately equivalent to the displacement of the target point. Usually, the instantaneous rate of change of gray level at a specific coordinate point on a two-dimensional image plane is defined as the optical flow vector. The optical flow method uses the changes of pixels in the image sequence in the time domain and the correlation between adjacent frames to find the corresponding relationship between the previous frame and the current frame, thereby calculating the motion of objects between adjacent frames. A method of information. Optical flow contains information about object motion.
光流场是一个二维矢量场,它反映了图像上每一点灰度的变化趋势,可看成是带有灰度的像素点在图像平面上运动而产生的瞬时速度场。它包含的信息即是各像素点的瞬时运动速度矢量信息。在理想情况下,光流场对应于运动场。The optical flow field is a two-dimensional vector field, which reflects the change trend of each gray point on the image, and can be regarded as the instantaneous velocity field generated by the movement of gray pixels on the image plane. The information it contains is the instantaneous motion velocity vector information of each pixel. Ideally, an optical flow field corresponds to a motion field.
具体的,手势识别装置可以采用光流算法进行运动识别,具体可以对当前RGB图像帧和上一RGB图像帧进行重采样和去噪预处理后,利用光流法计算出当前RGB图像帧各点的光流值,得到各点的光流场,然后对光流场进行阈值分割,区分出前景区域与背景区域,前景区域中包括目标手部,之后可以将前景区域的速度矢量和背景区域的速度矢量确定为目标手部的水平运动稳定数据。Specifically, the gesture recognition device can use the optical flow algorithm for motion recognition. Specifically, after resampling and denoising preprocessing of the current RGB image frame and the previous RGB image frame, the optical flow method can be used to calculate the current RGB image frame. The optical flow value of each point is obtained, and then the optical flow field is thresholded to distinguish the foreground area and the background area. The foreground area includes the target hand. After that, the velocity vector of the foreground area and the velocity vector of the background area can be combined The velocity vector is determined as the horizontal motion stabilization data of the target hand.
在另一种实施例中,手势识别装置还可以使用连续自适应的MeanShift(Continuously Adaptive Mean-SHIFT,CamShift)算法或主动轮廓的跟踪算法来对目标图像进行运动识别,具体过程在此不进行赘述。In another embodiment, the gesture recognition device can also use a continuous adaptive MeanShift (Continuously Adaptive Mean-SHIFT, CamShift) algorithm or an active contour tracking algorithm to perform motion recognition on the target image, and the specific process will not be repeated here. .
步骤103、确定目标手部与预设平面的垂直距离。 Step 103, determining the vertical distance between the target hand and the preset plane.
其中,预设平面可以是当前进行手势识别的电子设备所在的平面,预设平面可以为水平平面或竖直平面,具体根据实际场景确定。例如当电子设备放置于水平桌面,需要对上方用户进行手势识别,则预设平面为电子设备所在水平平面;或者,当电子设备竖直放置,需要对前方用户进行手势识别,则预设平面为电子设备所在竖直平面,例如竖直墙面。Wherein, the preset plane may be a plane where the electronic device currently performing gesture recognition is located, and the preset plane may be a horizontal plane or a vertical plane, which is specifically determined according to an actual scene. For example, when an electronic device is placed on a horizontal desktop and gesture recognition needs to be performed on the upper user, the preset plane is the horizontal plane where the electronic device is located; or, when the electronic device is placed vertically and gesture recognition needs to be performed on the user in front, the preset plane is The vertical plane where the electronic equipment is located, such as a vertical wall.
本公开实施例,可以通过多种方式确定目标手部与预设平面的垂直距离,例如可以通过深度图像提取或者基于距离传感器确定,仅为示例,而非限定。In the embodiment of the present disclosure, the vertical distance between the target hand and the preset plane can be determined in various ways, for example, it can be determined by extracting a depth image or based on a distance sensor, which is only an example and not a limitation.
在一些实施例中,目标图像包括第一时刻的第一深度图像和第二时刻的第二深度图像,确定目标手部与预设平面的垂直距离,包括:基于第一深度图像和第二深度图像,分别提取第一时刻和第二时刻目标手部与预设平面的第一垂直距离和第二垂直距离,第一深度图像和第二深度图像中均包括目标手部和预设平面。第二时刻在第一时刻之后,第一时刻和第二时刻之间间隔预设时间,例如可以检测30秒。In some embodiments, the target image includes a first depth image at the first moment and a second depth image at the second moment, and determining the vertical distance between the target hand and the preset plane includes: based on the first depth image and the second depth image, respectively extracting the first vertical distance and the second vertical distance between the target hand and the preset plane at the first moment and the second moment, and both the first depth image and the second depth image include the target hand and the preset plane. The second moment is after the first moment, and there is a preset time interval between the first moment and the second moment, for example, 30 seconds may be detected.
深度图像也被称为距离图像,是指将从图像采集器到场景中各点的距离(深度)作为像素值的图像,它直接反映了物体可见表面的几何形状。手势识别装置可以通过深度传感器获 取包括目标手部和预设平面的第一深度图像和第二深度图像,第一深度图像对应于第一时刻,第二深度图像对应于第二时刻。之后可以从第一深度图像和第二深度图像中,分别提取第一时刻和第二时刻目标手部中预设点与预设平面的第一垂直距离和第二垂直距离。预设点可以设置为目标手部中的一个位置点,例如预设点可以为目标手部任意一个手指的指尖或手掌中心等。Depth image is also called distance image, which refers to the image with the distance (depth) from the image collector to each point in the scene as the pixel value, which directly reflects the geometry of the visible surface of the object. The gesture recognition device can acquire a first depth image and a second depth image including the target hand and a preset plane through the depth sensor, the first depth image corresponds to the first moment, and the second depth image corresponds to the second moment. Then, the first vertical distance and the second vertical distance between the preset point in the target hand and the preset plane at the first moment and the second moment can be respectively extracted from the first depth image and the second depth image. The preset point can be set as a position point in the target hand, for example, the preset point can be the fingertip or palm center of any finger of the target hand.
在另一些实施例中,确定目标手部与预设平面的垂直距离,可以包括:采用距离传感器分别确定第一时刻和第二时刻目标手部与预设平面的第一垂直距离和第二垂直距离。In some other embodiments, determining the vertical distance between the target hand and the preset plane may include: using a distance sensor to respectively determine the first vertical distance and the second vertical distance between the target hand and the preset plane at the first moment and the second moment distance.
距离传感器可以是用于感应其与某物体间的距离的传感器,本公开实施例中的距离传感器可以设置于上述电子设备中。本公开实施例中手势识别装置还可以获取通过距离传感器采集的在第一时刻和第二时刻的目标手部与预设平面的垂直距离。The distance sensor may be a sensor for sensing the distance between it and an object, and the distance sensor in the embodiment of the present disclosure may be set in the above-mentioned electronic device. In the embodiment of the present disclosure, the gesture recognition device may also acquire the vertical distances between the target hand and the preset plane collected by the distance sensor at the first moment and the second moment.
步骤104、当基于水平运动稳定数据确定目标手部在水平方向稳定,并且基于目标手部与预设平面的垂直距离确定目标手部在垂直方向稳定,则对目标图像的目标手部进行手势识别。 Step 104, when it is determined that the target hand is stable in the horizontal direction based on the horizontal motion stabilization data, and it is determined that the target hand is stable in the vertical direction based on the vertical distance between the target hand and the preset plane, perform gesture recognition on the target hand in the target image .
目标手部在水平方向或在垂直方向稳定可以理解为目标手部在水平方向或垂直方向没有较大的运动幅度。当手部运动幅度较大时,会造成手势识别的干扰,对手势识别的准确率的影响较大,因此本公开实施例中当确定手部在水平方向和垂直方向均稳定时进行手势识别。The fact that the target hand is stable in the horizontal direction or in the vertical direction can be understood as that the target hand does not have a large range of motion in the horizontal direction or in the vertical direction. When the hand movement range is large, it will cause interference to gesture recognition and have a great impact on the accuracy of gesture recognition. Therefore, in the embodiment of the present disclosure, gesture recognition is performed when the hand is determined to be stable in both horizontal and vertical directions.
本公开实施例中,基于水平运动稳定数据确定目标手部在水平方向稳定,可以包括:如果前景区域的速度矢量和背景区域的速度矢量的差值小于预设阈值,则确定目标手部在水平方向稳定。In the embodiment of the present disclosure, determining that the target hand is stable in the horizontal direction based on the horizontal motion stabilization data may include: if the difference between the velocity vector of the foreground area and the velocity vector of the background area is less than a preset threshold, determining that the target hand is stable in the horizontal direction Steady direction.
获取前景区域的速度矢量和背景区域的速度矢量之后,当图像中有运动物体时,前景区域和背景区域存在相对运动,本公开实施例中可以确定前景区域和背景区域之间的速度矢量的差值,并将差值与预设阈值进行比对,如果差值小于预设阈值,则确定目标手部在水平方向的运动幅度很小,视为稳定。After obtaining the velocity vector of the foreground area and the velocity vector of the background area, when there is a moving object in the image, there is relative motion between the foreground area and the background area, and the difference between the velocity vectors between the foreground area and the background area can be determined in the embodiment of the present disclosure value, and the difference is compared with the preset threshold, if the difference is less than the preset threshold, it is determined that the movement range of the target hand in the horizontal direction is very small, and it is considered stable.
本公开实施例中,基于目标手部与预设平面的垂直距离确定目标手部在垂直方向稳定,包括:如果目标手部与预设平面的第一垂直距离和第二垂直距离之间的差值小于第一预设差值,则确定目标手部在垂直方向稳定。In the embodiment of the present disclosure, determining that the target hand is stable in the vertical direction based on the vertical distance between the target hand and the preset plane includes: if the difference between the first vertical distance and the second vertical distance between the target hand and the preset plane If the value is smaller than the first preset difference, it is determined that the target hand is stable in the vertical direction.
第一预设差值可以根据实际情况设置,例如预设差值可以为1cm。本公开实施例中,在确定目标手部与预设平面之间分别在预设时间之前和之后的第一垂直距离和第二垂直距离之后,可以将确定第一垂直距离和第二垂直距离之间的差值,并将差值与第一预设差值进行对比,如果差值小于第一预设差值,则说明目标手部在垂直方向稳定在一个小的距离范围内,进而确定目标手部在垂直方向稳定。The first preset difference can be set according to actual conditions, for example, the preset difference can be 1 cm. In the embodiment of the present disclosure, after determining the first vertical distance and the second vertical distance between the target hand and the preset plane before and after the preset time respectively, the distance between the first vertical distance and the second vertical distance may be determined and compare the difference with the first preset difference. If the difference is smaller than the first preset difference, it means that the target hand is stable within a small distance in the vertical direction, and then the target is determined. The hand is stabilized in the vertical direction.
具体的,手势识别装置在确定目标手部的水平运动稳定数据以及目标手部与预设平面的垂直距离之后,可以分别根据目标手部的水平运动稳定数据判断目标手部在水平方向是否稳定,以及根据目标手部与预设平面的垂直距离判断目标手部在垂直方向是否稳定,若确定目标手部在水平方向以及垂直方向均稳定,则可以对目标图像的目标手部进行手势识别,具体采用的手势识别的方式可以为多种,本公开实施例中对此不作限定。Specifically, after determining the horizontal movement stability data of the target hand and the vertical distance between the target hand and the preset plane, the gesture recognition device can determine whether the target hand is stable in the horizontal direction according to the horizontal movement stability data of the target hand, And judging whether the target hand is stable in the vertical direction according to the vertical distance between the target hand and the preset plane, if it is determined that the target hand is stable in both the horizontal direction and the vertical direction, gesture recognition can be performed on the target hand of the target image, specifically There may be various gesture recognition manners adopted, which are not limited in this embodiment of the present disclosure.
在本公开实施例中,在对目标图像的目标手部进行手势识别之前,还包括:判断目标手部与预设平面的垂直距离与预设距离的差值是否小于第二预设差值;当目标手部与预设平面的垂直距离与预设距离的差值小于第二预设差值,则执行对目标图像的目标手部进行手势识别。In the embodiment of the present disclosure, before performing gesture recognition on the target hand of the target image, it further includes: judging whether the difference between the vertical distance between the target hand and the preset plane and the preset distance is smaller than the second preset difference; When the difference between the vertical distance between the target hand and the preset plane and the preset distance is smaller than the second preset difference, perform gesture recognition on the target hand in the target image.
其中,预设距离可以是预先设置的一个手部与预设平面之间的识别距离,可以根据实际使用场景来设置,例如在对悬停下的手部进行手势识别时,该预设距离可以较远,例如预设距离可以为10cm;而在对手指点读场景下的手势识别时,该预设距离可以较近。Wherein, the preset distance can be a preset recognition distance between a hand and a preset plane, which can be set according to actual use scenarios. For example, when performing gesture recognition on a hovering hand, the preset distance can be Farther, for example, the preset distance may be 10 cm; and the preset distance may be shorter when performing gesture recognition in a point-and-read scene.
本公开实施例中,在确定目标手部与预设平面之间的垂直距离之后,可以确定该垂直距离与预设距离之间的差值,并将差值与第二预设差值进行对比,如果差值小于第二预设差值,则说明目标手部满足手势识别的一个距离要求,之后可以对目标图像进行手势识别。第二预设差值可以与上述第一预设差值相同,也可以不同。In the embodiment of the present disclosure, after determining the vertical distance between the target hand and the preset plane, the difference between the vertical distance and the preset distance can be determined, and the difference can be compared with the second preset difference , if the difference is smaller than the second preset difference, it means that the target hand meets a distance requirement for gesture recognition, and then gesture recognition can be performed on the target image. The second preset difference may be the same as or different from the above-mentioned first preset difference.
可选的,对目标图像的目标手部进行手势识别,可以包括:对目标图像进行手势分割、特征提取之后,基于提取的特征采用手势识别算法进行手势识别。Optionally, performing gesture recognition on the target hand of the target image may include: performing gesture segmentation and feature extraction on the target image, and then performing gesture recognition using a gesture recognition algorithm based on the extracted features.
上述预设手势识别算法可以包括模版匹配算法、统计分析算法以及神经网络算法等,具体不限。具体的,手势识别装置可以对目标图像进行手势分割,具体可以采用阈值法、边缘检测法或物理特征法等分割方式进行手势分割;然后可以对分割出的手势区域进行特征提取,具体提取的特征可以包括轮廓、边缘、图像矩、图像特征向量以及区域直方图特征等等,具体不限;之后可以基于提取的特征采用预设手势识别算法进行手势识别,得到最终的识别结果。The aforementioned preset gesture recognition algorithm may include a template matching algorithm, a statistical analysis algorithm, a neural network algorithm, etc., and is not specifically limited. Specifically, the gesture recognition device can perform gesture segmentation on the target image, specifically, threshold method, edge detection method, or physical feature method can be used to perform gesture segmentation; then feature extraction can be performed on the segmented gesture area, and the extracted features It can include contours, edges, image moments, image feature vectors, and regional histogram features, etc., and is not limited in detail; then based on the extracted features, a preset gesture recognition algorithm can be used for gesture recognition to obtain the final recognition result.
本公开实施例提供的手势识别方案,获取目标图像,通过对目标图像进行运动识别,确定目标手部的水平运动稳定数据,确定目标手部与预设平面的垂直距离;当基于水平运动稳定数据确定目标手部在水平方向稳定,并且基于目标手部与预设平面的垂直距离确定目标手部在垂直方向稳定,则对目标图像的目标手部进行手势识别。采用上述技术方案,通过在手势识别之前对水平方向和垂直方向的稳定判断,在手部稳定之后再进行手势识别,避免相关技术中因手部在水平和/或垂直方向的运动干扰而造成的较大误差,进而提升了手势识别的准 确率。The gesture recognition scheme provided by the embodiments of the present disclosure acquires the target image, and determines the horizontal motion stability data of the target hand by performing motion recognition on the target image, and determines the vertical distance between the target hand and the preset plane; when based on the horizontal motion stability data If it is determined that the target hand is stable in the horizontal direction, and based on the vertical distance between the target hand and the preset plane, it is determined that the target hand is stable in the vertical direction, and gesture recognition is performed on the target hand in the target image. By adopting the above-mentioned technical solution, through the stable judgment of the horizontal direction and the vertical direction before the gesture recognition, the gesture recognition is performed after the hand is stable, so as to avoid the interference caused by the movement interference of the hand in the horizontal and/or vertical direction in the related technology. Larger error, thereby improving the accuracy of gesture recognition.
图2为本公开实施例提供的另一种手势识别方法的流程示意图,本实施例在上述实施例的基础上,进一步对上述手势识别方法进行具体说明。如图2所示,该方法包括:FIG. 2 is a schematic flowchart of another gesture recognition method provided by an embodiment of the present disclosure. On the basis of the above embodiments, this embodiment further specifically describes the above gesture recognition method. As shown in Figure 2, the method includes:
步骤201、获取目标图像。 Step 201, acquiring a target image.
其中,目标图像可以包括RGB图像帧与深度图像。Wherein, the target image may include an RGB image frame and a depth image.
步骤201之后,可以如图2所示,先执行步骤202-步骤203,再执行步骤204-步骤205;也可以先执行步骤204-步骤205;再执行步骤202-步骤203;还可以先执行步骤202和步骤204(先后顺序不限),再执行步骤203和步骤205(先后顺序不限),具体根据实际情况确定。图2中的执行顺序仅为示例。After step 201, as shown in Figure 2, step 202-step 203 can be executed first, and then step 204-step 205 can be executed; step 204-step 205 can also be executed first; step 202-step 203 can be executed first; step 202-step 203 can also be executed first Step 202 and step 204 (the sequence is not limited), and then execute step 203 and step 205 (the sequence is not limited), which is determined according to the actual situation. The order of execution in Figure 2 is just an example.
步骤202、通过对目标图像进行运动识别,确定目标手部的水平运动稳定数据。Step 202: Determine the horizontal motion stability data of the target hand by performing motion recognition on the target image.
可选的,目标图像包括视频中提取的当前RGB图像帧和上一RGB图像帧,通过对目标图像进行运动识别,确定目标手部的水平运动稳定数据,包括:基于当前RGB图像帧和上一RGB图像帧,采用光流算法计算当前RGB图像帧的光流场,并对光流场进行阈值分割,得到包括目标手部的前景区域和背景区域;将前景区域的速度矢量和背景区域的速度矢量确定为目标手部的水平运动稳定数据。Optionally, the target image includes the current RGB image frame and the previous RGB image frame extracted from the video, and the horizontal motion stabilization data of the target hand is determined by performing motion recognition on the target image, including: based on the current RGB image frame and the previous RGB image frame RGB image frame, using the optical flow algorithm to calculate the optical flow field of the current RGB image frame, and thresholding the optical flow field to obtain the foreground area and background area including the target hand; the velocity vector of the foreground area and the velocity of the background area The vector is determined as the horizontal motion stabilization data of the target hand.
步骤203、基于水平运动稳定数据确定目标手部在水平方向是否稳定,若是,则执行步骤204;否则,返回执行步骤201。 Step 203 , determine whether the target hand is stable in the horizontal direction based on the horizontal motion stability data, if yes, execute step 204 ; otherwise, return to execute step 201 .
具体的,如果前景区域的速度矢量和背景区域的速度矢量的差值小于预设阈值,则确定目标手部在水平方向稳定,执行步骤204;如果前景区域的速度矢量和背景区域的速度矢量的差值大于或等于预设阈值,则确定目标手部在水平方向不稳定,返回执行步骤201。Specifically, if the difference between the velocity vector of the foreground area and the velocity vector of the background area is less than the preset threshold, it is determined that the target hand is stable in the horizontal direction, and step 204 is performed; if the velocity vector of the foreground area and the velocity vector of the background area If the difference is greater than or equal to the preset threshold, it is determined that the target hand is unstable in the horizontal direction, and the execution returns to step 201 .
步骤204、确定目标手部与预设平面的垂直距离。 Step 204, determining the vertical distance between the target hand and the preset plane.
其中,预设平面为水平平面或竖直平面。Wherein, the preset plane is a horizontal plane or a vertical plane.
可选的,目标图像包括第一时刻的第一深度图像和第二时刻的第二深度图像,确定目标手部与预设平面的垂直距离,包括:基于第一深度图像和第二深度图像,分别提取第一时刻和第二时刻目标手部与预设平面的第一垂直距离和第二垂直距离,第一深度图像和第二深度图像中均包括目标手部和预设平面。Optionally, the target image includes a first depth image at the first moment and a second depth image at the second moment, and determining the vertical distance between the target hand and the preset plane includes: based on the first depth image and the second depth image, The first vertical distance and the second vertical distance between the target hand and the preset plane are respectively extracted at the first moment and the second moment, and both the first depth image and the second depth image include the target hand and the preset plane.
可选的,确定目标手部与预设平面的垂直距离,包括:采用距离传感器分别确定第一时刻和第二时刻目标手部与预设平面的第一垂直距离和第二垂直距离。Optionally, determining the vertical distance between the target hand and the preset plane includes: using a distance sensor to respectively determine a first vertical distance and a second vertical distance between the target hand and the preset plane at the first moment and the second moment.
步骤205、基于目标手部与预设平面的垂直距离确定目标手部在垂直方向是否稳定,若是,则执行步骤206;否则返回执行步骤201。 Step 205 , determine whether the target hand is stable in the vertical direction based on the vertical distance between the target hand and the preset plane, if yes, perform step 206 ; otherwise, return to step 201 .
具体的,如果目标手部与预设平面的第一垂直距离和第二垂直距离之间的差值小于第一预设差值,则确定目标手部在垂直方向稳定,之后可以执行步骤206;如果目标手部与预设平面的第一垂直距离和第二垂直距离之间的差值大于或等于第一预设差值,则确定目标手部在垂直方向不稳定,之后可以返回执行步骤201。Specifically, if the difference between the first vertical distance and the second vertical distance between the target hand and the preset plane is less than the first preset difference, it is determined that the target hand is stable in the vertical direction, and then step 206 can be performed; If the difference between the first vertical distance and the second vertical distance between the target hand and the preset plane is greater than or equal to the first preset difference, it is determined that the target hand is unstable in the vertical direction, and then the execution of step 201 may be returned .
步骤206、判断目标手部与预设平面的垂直距离与预设距离的差值是否小于第二预设差值,若是,则执行步骤207;否则,返回执行步骤201。 Step 206 , judging whether the difference between the vertical distance between the target hand and the preset plane and the preset distance is smaller than the second preset difference, if yes, go to step 207 ; otherwise, go back to step 201 .
步骤207、对目标图像的目标手部进行手势识别。 Step 207, performing gesture recognition on the target hand in the target image.
具体的,对目标图像的目标手部进行手势识别,可以包括:对目标图像进行手势分割、特征提取之后,基于提取的特征采用预设手势识别算法进行手势识别。Specifically, performing gesture recognition on the target hand of the target image may include: performing gesture segmentation and feature extraction on the target image, and performing gesture recognition using a preset gesture recognition algorithm based on the extracted features.
示例的,图3为本公开实施例提供的一种手势识别的示意图,如图3所示,手势识别的过程可以包括:步骤21、开始。步骤22、获取视频中的RGB图像帧和深度图像。也即获取上述目标图像,目标图像中包括RGB图像帧和深度图像。步骤23、基于当前RGB图像帧和上一RGB图像帧,采用光流算法对当前RGB图像帧进行运动识别,确定目标手部的水平运动稳定数据。步骤24、基于水平运动稳定数据确定目标手部在水平方向是否稳定,若是,则执行步骤25;否则返回执行步骤22。步骤25、基于深度图像确定目标手部与预设平面的垂直距离。步骤26、基于目标手部与预设平面的垂直距离确定目标手部在垂直方向是否稳定,若是,则执行步骤27;否则,返回执行步骤22。步骤27、目标手部与预设平面的垂直距离是否达到预设距离,若是,则执行步骤28;否则,返回执行步骤22。当目标手部与预设平面的垂直距离与预设距离的差值小于第二预设差值,则执行步骤28;否则返回执行步骤22。步骤28、手势识别。当目标手部在水平方向和垂直方向均达到稳定,并且目标手部与预设平面也达到预设距离后,启动手势识别。步骤29、后续处理。具体可以为将实时识别的手势与预设手势匹配,如果匹配成功,则完成手势识别。步骤30、结束。As an example, FIG. 3 is a schematic diagram of a gesture recognition provided by an embodiment of the present disclosure. As shown in FIG. 3 , the gesture recognition process may include: Step 21, start. Step 22, acquiring the RGB image frame and the depth image in the video. That is, the above-mentioned target image is acquired, and the target image includes an RGB image frame and a depth image. Step 23: Based on the current RGB image frame and the previous RGB image frame, the optical flow algorithm is used to perform motion recognition on the current RGB image frame, and determine the horizontal motion stability data of the target hand. Step 24: Determine whether the target hand is stable in the horizontal direction based on the horizontal movement stability data, if yes, execute step 25; otherwise, return to execute step 22. Step 25. Determine the vertical distance between the target hand and the preset plane based on the depth image. Step 26. Determine whether the target hand is stable in the vertical direction based on the vertical distance between the target hand and the preset plane. If yes, perform step 27; otherwise, return to step 22. Step 27, whether the vertical distance between the target hand and the preset plane reaches the preset distance, if yes, go to step 28; otherwise, go back to step 22. When the difference between the vertical distance between the target hand and the preset plane and the preset distance is less than the second preset difference, step 28 is executed; otherwise, step 22 is executed. Step 28, gesture recognition. When the target hand is stable in both the horizontal direction and the vertical direction, and the target hand reaches the preset distance from the preset plane, gesture recognition is started. Step 29, subsequent processing. Specifically, the gesture recognized in real time may be matched with a preset gesture, and if the matching is successful, the gesture recognition is completed. Step 30, end.
本方案中,通过光流算法进行手部的水平运动识别,并基于深度信息确定手部与被测平面的垂直距离,进而当目标手部在水平方向和垂直方向均达到稳定时,再进行手势识别,可以得到准确率更高的手势识别结果。In this solution, the horizontal motion recognition of the hand is carried out through the optical flow algorithm, and the vertical distance between the hand and the measured plane is determined based on the depth information, and then the gesture is performed when the target hand is stable in both the horizontal and vertical directions. Recognition, gesture recognition results with higher accuracy can be obtained.
本公开实施例提供的手势识别方案,获取目标图像,通过对目标图像进行运动识别,确定目标手部的水平运动稳定数据,确定目标手部与预设平面的垂直距离;当基于水平运动稳定数据确定目标手部在水平方向稳定,并且基于目标手部与预设平面的垂直距离确定目标手 部在垂直方向稳定,则对目标图像的目标手部进行手势识别。采用上述技术方案,通过在手势识别之前对水平方向和垂直方向的稳定判断,在手部稳定之后再进行手势识别,避免相关技术中因手部在水平和/或垂直方向的运动干扰而造成的较大误差,进而提升了手势识别的准确率。The gesture recognition scheme provided by the embodiments of the present disclosure acquires the target image, and determines the horizontal motion stability data of the target hand by performing motion recognition on the target image, and determines the vertical distance between the target hand and the preset plane; when based on the horizontal motion stability data If it is determined that the target hand is stable in the horizontal direction, and based on the vertical distance between the target hand and the preset plane, it is determined that the target hand is stable in the vertical direction, and gesture recognition is performed on the target hand in the target image. By adopting the above-mentioned technical solution, through the stable judgment of the horizontal direction and the vertical direction before the gesture recognition, the gesture recognition is performed after the hand is stable, so as to avoid the interference caused by the movement interference of the hand in the horizontal and/or vertical direction in the related technology. Larger error, thereby improving the accuracy of gesture recognition.
图4为本公开实施例提供的一种手势识别装置的结构示意图,该装置可由软件和/或硬件实现,一般可集成在电子设备中。如图4所示,该装置包括:FIG. 4 is a schematic structural diagram of a gesture recognition device provided by an embodiment of the present disclosure. The device can be implemented by software and/or hardware, and generally can be integrated into an electronic device. As shown in Figure 4, the device includes:
图像获取模块301,用于获取目标图像;An image acquisition module 301, configured to acquire a target image;
水平数据模块302,用于通过对所述目标图像进行运动识别,确定目标手部的水平运动稳定数据;The horizontal data module 302 is used to determine the horizontal motion stability data of the target hand by performing motion recognition on the target image;
垂直数据模块303,用于确定所述目标手部与预设平面的垂直距离;A vertical data module 303, configured to determine the vertical distance between the target hand and a preset plane;
手势识别模块304,用于当基于所述水平运动稳定数据确定所述目标手部在水平方向稳定,并且基于所述目标手部与预设平面的垂直距离确定所述目标手部在垂直方向稳定,则对所述目标图像的所述目标手部进行手势识别。 Gesture recognition module 304, configured to determine that the target hand is stable in the horizontal direction based on the horizontal motion stability data, and determine that the target hand is stable in the vertical direction based on the vertical distance between the target hand and a preset plane , performing gesture recognition on the target hand of the target image.
可选的,所述目标图像包括视频中提取的当前RGB图像帧和上一RGB图像帧,所述水平数据模块302具体用于:Optionally, the target image includes the current RGB image frame and the previous RGB image frame extracted from the video, and the horizontal data module 302 is specifically used for:
基于所述当前RGB图像帧和所述上一RGB图像帧,采用光流算法计算所述当前RGB图像帧的光流场,并对所述光流场进行阈值分割,得到包括所述目标手部的前景区域和背景区域;Based on the current RGB image frame and the last RGB image frame, an optical flow algorithm is used to calculate the optical flow field of the current RGB image frame, and the optical flow field is thresholded to obtain the target hand the foreground and background regions of
将所述前景区域的速度矢量和所述背景区域的速度矢量确定为所述目标手部的水平运动稳定数据。The velocity vector of the foreground area and the velocity vector of the background area are determined as the horizontal motion stabilization data of the target hand.
可选的,所述手势识别模块304具体用于:Optionally, the gesture recognition module 304 is specifically configured to:
如果所述前景区域的速度矢量和所述背景区域的速度矢量的差值小于预设阈值,则确定所述目标手部在水平方向稳定。If the difference between the velocity vector of the foreground area and the velocity vector of the background area is smaller than a preset threshold, it is determined that the target hand is stable in the horizontal direction.
可选的,所述目标图像包括第一时刻的第一深度图像和第二时刻的第二深度图像,所述垂直数据模块303具体用于:Optionally, the target image includes a first depth image at a first moment and a second depth image at a second moment, and the vertical data module 303 is specifically used for:
基于所述第一深度图像和所述第二深度图像,分别提取第一时刻和第二时刻所述目标手部与所述预设平面的第一垂直距离和第二垂直距离,所述第一深度图像和所述第二深度图像中均包括所述目标手部和所述预设平面。Based on the first depth image and the second depth image, respectively extract a first vertical distance and a second vertical distance between the target hand and the preset plane at the first moment and the second moment, the first Both the depth image and the second depth image include the target hand and the preset plane.
可选的,所述垂直数据模块303具体用于:Optionally, the vertical data module 303 is specifically used for:
采用距离传感器分别确定第一时刻和第二时刻所述目标手部与所述预设平面的第一垂直距离和第二垂直距离。A distance sensor is used to determine a first vertical distance and a second vertical distance between the target hand and the preset plane at the first moment and the second moment respectively.
可选的,所述手势识别模块304具体用于:Optionally, the gesture recognition module 304 is specifically configured to:
如果所述目标手部与所述预设平面的所述第一垂直距离和所述第二垂直距离之间的差值小于第一预设差值,则确定所述目标手部在垂直方向稳定。If the difference between the first vertical distance and the second vertical distance between the target hand and the preset plane is smaller than a first preset difference, it is determined that the target hand is stable in the vertical direction .
可选的,所述预设平面为水平平面或竖直平面。Optionally, the preset plane is a horizontal plane or a vertical plane.
可选的,所述装置还包括垂直判断模块,用于:在对所述目标图像的所述目标手部进行手势识别之前,Optionally, the device further includes a vertical judging module, configured to: before performing gesture recognition on the target hand of the target image,
判断所述目标手部与所述预设平面的垂直距离与预设距离的差值是否小于第二预设差值;judging whether the difference between the vertical distance between the target hand and the preset plane and the preset distance is smaller than a second preset difference;
当所述目标手部与所述预设平面的垂直距离与所述预设距离的差值小于所述第二预设差值,则执行所述对所述目标图像的所述目标手部进行手势识别。When the difference between the vertical distance between the target hand and the preset plane and the preset distance is smaller than the second preset difference, then performing the processing of the target hand in the target image Gesture Recognition.
可选的,所述手势识别模块304具体用于:Optionally, the gesture recognition module 304 is specifically configured to:
对所述目标图像进行手势分割、特征提取之后,基于提取的特征采用预设手势识别算法进行手势识别。After gesture segmentation and feature extraction are performed on the target image, gesture recognition is performed using a preset gesture recognition algorithm based on the extracted features.
本公开实施例所提供的手势识别装置可执行本公开任意实施例所提供的手势识别方法,具备执行方法相应的功能模块和有益效果。The gesture recognition device provided by the embodiments of the present disclosure can execute the gesture recognition method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.
本公开实施例还提供了一种计算机程序产品,包括计算机程序/指令,该计算机程序/指令被处理器执行时实现本公开任意实施例所提供的手势识别方法。An embodiment of the present disclosure further provides a computer program product, including a computer program/instruction, and when the computer program/instruction is executed by a processor, the gesture recognition method provided in any embodiment of the present disclosure is implemented.
图5为本公开实施例提供的一种电子设备的结构示意图。下面具体参考图5,其示出了适于用来实现本公开实施例中的电子设备400的结构示意图。本公开实施例中的电子设备400可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图5示出的电子设备仅仅是一个示例,不应对本公开实施例的功能和使用范围带来任何限制。FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. Referring specifically to FIG. 5 , it shows a schematic structural diagram of an electronic device 400 suitable for implementing an embodiment of the present disclosure. The electronic device 400 in the embodiment of the present disclosure may include, but is not limited to, mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), vehicle-mounted terminals ( Mobile terminals such as car navigation terminals) and stationary terminals such as digital TVs, desktop computers and the like. The electronic device shown in FIG. 5 is only an example, and should not limit the functions and scope of use of the embodiments of the present disclosure.
如图5所示,电子设备400可以包括处理装置(例如中央处理器、图形处理器等)401, 其可以根据存储在只读存储器(ROM)402中的程序或者从存储装置408加载到随机访问存储器(RAM)403中的程序而执行各种适当的动作和处理。在RAM 403中,还存储有电子设备400操作所需的各种程序和数据。处理装置401、ROM402以及RAM403通过总线404彼此相连。输入/输出(I/O)接口405也连接至总线404。As shown in FIG. 5, an electronic device 400 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 401, which may be randomly accessed according to a program stored in a read-only memory (ROM) 402 or loaded from a storage device 408. Various appropriate actions and processes are executed by programs in the memory (RAM) 403 . In the RAM 403, various programs and data necessary for the operation of the electronic device 400 are also stored. The processing device 401 , ROM 402 and RAM 403 are connected to each other through a bus 404 . An input/output (I/O) interface 405 is also connected to bus 404 .
通常,以下装置可以连接至I/O接口405:包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置406;包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置407;包括例如磁带、硬盘等的存储装置408;以及通信装置409。通信装置409可以允许电子设备400与其他设备进行无线或有线通信以交换数据。虽然图5示出了具有各种装置的电子设备400,但是应理解的是,并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。Typically, the following devices can be connected to the I/O interface 405: input devices 406 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speaker, vibration an output device 407 such as a computer; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409. The communication means 409 may allow the electronic device 400 to perform wireless or wired communication with other devices to exchange data. While FIG. 5 shows electronic device 400 having various means, it should be understood that implementing or having all of the means shown is not a requirement. More or fewer means may alternatively be implemented or provided.
特别地,根据本公开的实施例,上文参考流程图描述的过程可以被实现为计算机软件程序。例如,本公开的实施例包括一种计算机程序产品,其包括承载在非暂态计算机可读介质上的计算机程序,该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中,该计算机程序可以通过通信装置409从网络上被下载和安装,或者从存储装置408被安装,或者从ROM 402被安装。在该计算机程序被处理装置401执行时,执行本公开实施例的手势识别方法中限定的上述功能。In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product, which includes a computer program carried on a non-transitory computer readable medium, where the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via communication means 409, or from storage means 408, or from ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the gesture recognition method of the embodiment of the present disclosure are executed.
需要说明的是,本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的***、装置或器件,或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于:具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中,计算机可读存储介质可以是任何包含或存储程序的有形介质,该程序可以被指令执行***、装置或者器件使用或者与其结合使用。而在本公开中,计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号,其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式,包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质,该计算机可读信号介质可以发送、传播或者传输用于由指令 执行***、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输,包括但不限于:电线、光缆、RF(射频)等等,或者上述的任意合适的组合。It should be noted that the above-mentioned computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted by any appropriate medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
在一些实施方式中,客户端、服务器可以利用诸如HTTP(HyperText Transfer Protocol,超文本传输协议)之类的任何当前已知或未来研发的网络协议进行通信,并且可以与任意形式或介质的数字数据通信(例如,通信网络)互连。通信网络的示例包括局域网(“LAN”),广域网(“WAN”),网际网(例如,互联网)以及端对端网络(例如,ad hoc端对端网络),以及任何当前已知或未来研发的网络。In some embodiments, the client and the server can communicate using any currently known or future network protocols such as HTTP (HyperText Transfer Protocol, Hypertext Transfer Protocol), and can communicate with digital data in any form or medium Communications (eg, communication networks) are interconnected. Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network of.
上述计算机可读介质可以是上述电子设备中所包含的;也可以是单独存在,而未装配入该电子设备中。The above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device.
上述计算机可读介质承载有一个或者多个程序,当上述一个或者多个程序被该电子设备执行时,使得该电子设备:获取目标图像;通过对所述目标图像进行运动识别,确定目标手部的水平运动稳定数据;确定所述目标手部与预设平面的垂直距离;当基于所述水平运动稳定数据确定所述目标手部在水平方向稳定,并且基于所述目标手部与预设平面的垂直距离确定所述目标手部在垂直方向稳定,则对所述目标图像的所述目标手部进行手势识别。The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device: acquires the target image; determines the target hand by performing motion recognition on the target image The horizontal motion stabilization data; determine the vertical distance between the target hand and the preset plane; when it is determined based on the horizontal motion stabilization data that the target hand is stable in the horizontal direction, and based on the target hand and the preset plane If the vertical distance of the target hand is determined to be stable in the vertical direction, gesture recognition is performed on the target hand in the target image.
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码,上述程序设计语言包括但不限于面向对象的程序设计语言-诸如Java、Smalltalk、C++,还包括常规的过程式程序设计语言-诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中,远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)-连接到用户计算机,或者,可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, or combinations thereof, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and Included are conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through an Internet service provider). Internet connection).
附图中的流程图和框图,图示了按照本公开各种实施例的***、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上,流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分,该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意,在有些作为替换的实现中,方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如,两个接连地表示的方框实际上可以基 本并行地执行,它们有时也可以按相反的顺序执行,这依所涉及的功能而定。也要注意的是,框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合,可以用执行规定的功能或操作的专用的基于硬件的***来实现,或者可以用专用硬件与计算机指令的组合来实现。The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现,也可以通过硬件的方式来实现。其中,单元的名称在某种情况下并不构成对该单元本身的限定。The units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. Wherein, the name of a unit does not constitute a limitation of the unit itself under certain circumstances.
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如,非限制性地,可以使用的示范类型的硬件逻辑部件包括:现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上***(SOC)、复杂可编程逻辑设备(CPLD)等等。The functions described herein above may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), System on Chips (SOCs), Complex Programmable Logical device (CPLD) and so on.
在本公开的上下文中,机器可读介质可以是有形的介质,其可以包含或存储以供指令执行***、装置或设备使用或与指令执行***、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体***、装置或设备,或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.
根据本公开的一个或多个实施例,本公开提供了一种手势识别方法,包括:According to one or more embodiments of the present disclosure, the present disclosure provides a gesture recognition method, including:
获取目标图像;Get the target image;
通过对所述目标图像进行运动识别,确定目标手部的水平运动稳定数据;Determine the horizontal motion stability data of the target hand by performing motion recognition on the target image;
确定所述目标手部与预设平面的垂直距离;determining the vertical distance between the target hand and a preset plane;
当基于所述水平运动稳定数据确定所述目标手部在水平方向稳定,并且基于所述目标手部与预设平面的垂直距离确定所述目标手部在垂直方向稳定,则对所述目标图像的所述目标手部进行手势识别。When it is determined based on the horizontal motion stabilization data that the target hand is stable in the horizontal direction, and based on the vertical distance between the target hand and a preset plane, it is determined that the target hand is stable in the vertical direction, then the target image The target hand performs gesture recognition.
根据本公开的一个或多个实施例,本公开提供的手势识别方法中,所述目标图像包括视频中提取的当前RGB图像帧和上一RGB图像帧,通过对所述目标图像进行运动识别,确定目标手部的水平运动稳定数据,包括:According to one or more embodiments of the present disclosure, in the gesture recognition method provided by the present disclosure, the target image includes the current RGB image frame and the previous RGB image frame extracted from the video, and by performing motion recognition on the target image, Determine horizontal motion stabilization data for the target hand, including:
基于所述当前RGB图像帧和所述上一RGB图像帧,采用光流算法计算所述当前RGB图 像帧的光流场,并对所述光流场进行阈值分割,得到包括所述目标手部的前景区域和背景区域;Based on the current RGB image frame and the last RGB image frame, an optical flow algorithm is used to calculate the optical flow field of the current RGB image frame, and the optical flow field is thresholded to obtain the target hand the foreground and background regions of
将所述前景区域的速度矢量和所述背景区域的速度矢量确定为所述目标手部的水平运动稳定数据。The velocity vector of the foreground area and the velocity vector of the background area are determined as the horizontal motion stabilization data of the target hand.
根据本公开的一个或多个实施例,本公开提供的手势识别方法中,基于所述水平运动稳定数据确定所述目标手部在水平方向稳定,包括:According to one or more embodiments of the present disclosure, in the gesture recognition method provided by the present disclosure, determining that the target hand is stable in the horizontal direction based on the horizontal motion stability data includes:
如果所述前景区域的速度矢量和所述背景区域的速度矢量的差值小于预设阈值,则确定所述目标手部在水平方向稳定。If the difference between the velocity vector of the foreground area and the velocity vector of the background area is smaller than a preset threshold, it is determined that the target hand is stable in the horizontal direction.
根据本公开的一个或多个实施例,本公开提供的手势识别方法中,所述目标图像包括第一时刻的第一深度图像和第二时刻的第二深度图像,确定所述目标手部与预设平面的垂直距离,包括:According to one or more embodiments of the present disclosure, in the gesture recognition method provided by the present disclosure, the target image includes a first depth image at a first moment and a second depth image at a second moment, and the target hand and The vertical distance of the preset plane, including:
基于所述第一深度图像和所述第二深度图像,分别提取第一时刻和第二时刻所述目标手部与所述预设平面的第一垂直距离和第二垂直距离,所述第一深度图像和所述第二深度图像中均包括所述目标手部和所述预设平面。Based on the first depth image and the second depth image, respectively extract a first vertical distance and a second vertical distance between the target hand and the preset plane at the first moment and the second moment, the first Both the depth image and the second depth image include the target hand and the preset plane.
根据本公开的一个或多个实施例,本公开提供的手势识别方法中,确定所述目标手部与预设平面的垂直距离,包括:According to one or more embodiments of the present disclosure, in the gesture recognition method provided in the present disclosure, determining the vertical distance between the target hand and the preset plane includes:
采用距离传感器分别确定第一时刻和第二时刻所述目标手部与所述预设平面的第一垂直距离和第二垂直距离。A distance sensor is used to determine a first vertical distance and a second vertical distance between the target hand and the preset plane at the first moment and the second moment respectively.
根据本公开的一个或多个实施例,本公开提供的手势识别方法中,基于所述目标手部与预设平面的垂直距离确定所述目标手部在垂直方向稳定,包括:According to one or more embodiments of the present disclosure, in the gesture recognition method provided in the present disclosure, determining that the target hand is stable in the vertical direction based on the vertical distance between the target hand and a preset plane includes:
如果所述目标手部与所述预设平面的所述第一垂直距离和所述第二垂直距离之间的差值小于第一预设差值,则确定所述目标手部在垂直方向稳定。If the difference between the first vertical distance and the second vertical distance between the target hand and the preset plane is smaller than a first preset difference, it is determined that the target hand is stable in the vertical direction .
根据本公开的一个或多个实施例,本公开提供的手势识别方法中,所述预设平面为水平平面或竖直平面。According to one or more embodiments of the present disclosure, in the gesture recognition method provided in the present disclosure, the preset plane is a horizontal plane or a vertical plane.
根据本公开的一个或多个实施例,本公开提供的手势识别方法中,在对所述目标图像的所述目标手部进行手势识别之前,还包括:According to one or more embodiments of the present disclosure, the gesture recognition method provided in the present disclosure, before performing gesture recognition on the target hand of the target image, further includes:
判断所述目标手部与所述预设平面的垂直距离与预设距离的差值是否小于第二预设差值;judging whether the difference between the vertical distance between the target hand and the preset plane and the preset distance is smaller than a second preset difference;
当所述目标手部与所述预设平面的垂直距离与所述预设距离的差值小于所述第二预设差值,则执行所述对所述目标图像的所述目标手部进行手势识别。When the difference between the vertical distance between the target hand and the preset plane and the preset distance is smaller than the second preset difference, then performing the processing on the target hand of the target image Gesture Recognition.
根据本公开的一个或多个实施例,本公开提供的手势识别方法中,对所述目标图像的所述目标手部进行手势识别,包括:According to one or more embodiments of the present disclosure, in the gesture recognition method provided in the present disclosure, performing gesture recognition on the target hand of the target image includes:
对所述目标图像进行手势分割、特征提取之后,基于提取的特征采用预设手势识别算法进行手势识别。After gesture segmentation and feature extraction are performed on the target image, gesture recognition is performed using a preset gesture recognition algorithm based on the extracted features.
根据本公开的一个或多个实施例,本公开提供了一种手势识别装置,包括:According to one or more embodiments of the present disclosure, the present disclosure provides a gesture recognition device, including:
图像获取模块,用于获取目标图像;An image acquisition module, configured to acquire a target image;
水平数据模块,用于通过对所述目标图像进行运动识别,确定目标手部的水平运动稳定数据;The horizontal data module is used to determine the horizontal motion stability data of the target hand by performing motion recognition on the target image;
垂直数据模块,用于确定所述目标手部与预设平面的垂直距离;a vertical data module, configured to determine the vertical distance between the target hand and a preset plane;
手势识别模块,用于当基于所述水平运动稳定数据确定所述目标手部在水平方向稳定,并且基于所述目标手部与预设平面的垂直距离确定所述目标手部在垂直方向稳定,则对所述目标图像的所述目标手部进行手势识别。The gesture recognition module is used to determine that the target hand is stable in the horizontal direction based on the horizontal motion stability data, and determine that the target hand is stable in the vertical direction based on the vertical distance between the target hand and a preset plane, Then perform gesture recognition on the target hand of the target image.
根据本公开的一个或多个实施例,本公开提供的手势识别装置中,所述目标图像包括视频中提取的当前RGB图像帧和上一RGB图像帧,所述水平数据模块具体用于:According to one or more embodiments of the present disclosure, in the gesture recognition device provided by the present disclosure, the target image includes the current RGB image frame and the previous RGB image frame extracted from the video, and the horizontal data module is specifically used for:
基于所述当前RGB图像帧和所述上一RGB图像帧,采用光流算法计算所述当前RGB图像帧的光流场,并对所述光流场进行阈值分割,得到包括所述目标手部的前景区域和背景区域;Based on the current RGB image frame and the last RGB image frame, an optical flow algorithm is used to calculate the optical flow field of the current RGB image frame, and the optical flow field is thresholded to obtain the target hand the foreground and background regions of
将所述前景区域的速度矢量和所述背景区域的速度矢量确定为所述目标手部的水平运动稳定数据。The velocity vector of the foreground area and the velocity vector of the background area are determined as the horizontal motion stabilization data of the target hand.
根据本公开的一个或多个实施例,本公开提供的手势识别装置中,所述手势识别模块具体用于:According to one or more embodiments of the present disclosure, in the gesture recognition device provided by the present disclosure, the gesture recognition module is specifically used for:
如果所述前景区域的速度矢量和所述背景区域的速度矢量的差值小于预设阈值,则确定所述目标手部在水平方向稳定。If the difference between the velocity vector of the foreground area and the velocity vector of the background area is smaller than a preset threshold, it is determined that the target hand is stable in the horizontal direction.
根据本公开的一个或多个实施例,本公开提供的手势识别装置中,所述目标图像包括第一时刻的第一深度图像和第二时刻的第二深度图像,所述垂直数据模块具体用于:According to one or more embodiments of the present disclosure, in the gesture recognition device provided by the present disclosure, the target image includes a first depth image at a first moment and a second depth image at a second moment, and the vertical data module is specifically used At:
基于所述第一深度图像和所述第二深度图像,分别提取第一时刻和第二时刻所述目标手部与所述预设平面的第一垂直距离和第二垂直距离,所述第一深度图像和所述第二深度图像中均包括所述目标手部和所述预设平面。Based on the first depth image and the second depth image, respectively extract a first vertical distance and a second vertical distance between the target hand and the preset plane at the first moment and the second moment, the first Both the depth image and the second depth image include the target hand and the preset plane.
根据本公开的一个或多个实施例,本公开提供的手势识别装置中,所述垂直数据模块具体用于:According to one or more embodiments of the present disclosure, in the gesture recognition device provided by the present disclosure, the vertical data module is specifically used for:
采用距离传感器分别确定第一时刻和第二时刻所述目标手部与所述预设平面的第一垂直距离和第二垂直距离。A distance sensor is used to determine a first vertical distance and a second vertical distance between the target hand and the preset plane at the first moment and the second moment respectively.
根据本公开的一个或多个实施例,本公开提供的手势识别装置中,所述手势识别模块具体用于:According to one or more embodiments of the present disclosure, in the gesture recognition device provided by the present disclosure, the gesture recognition module is specifically used for:
如果所述目标手部与所述预设平面的所述第一垂直距离和所述第二垂直距离之间的差值小于第一预设差值,则确定所述目标手部在垂直方向稳定。If the difference between the first vertical distance and the second vertical distance between the target hand and the preset plane is smaller than a first preset difference, it is determined that the target hand is stable in the vertical direction .
根据本公开的一个或多个实施例,本公开提供的手势识别装置中,所述预设平面为水平平面或竖直平面。According to one or more embodiments of the present disclosure, in the gesture recognition device provided in the present disclosure, the preset plane is a horizontal plane or a vertical plane.
根据本公开的一个或多个实施例,本公开提供的手势识别装置中,所述装置还包括垂直判断模块,用于:在对所述目标图像的所述目标手部进行手势识别之前,According to one or more embodiments of the present disclosure, in the gesture recognition device provided in the present disclosure, the device further includes a vertical judgment module, configured to: before performing gesture recognition on the target hand of the target image,
判断所述目标手部与所述预设平面的垂直距离与预设距离的差值是否小于第二预设差值;judging whether the difference between the vertical distance between the target hand and the preset plane and the preset distance is smaller than a second preset difference;
当所述目标手部与所述预设平面的垂直距离与所述预设距离的差值小于所述第二预设差值,则执行所述对所述目标图像的所述目标手部进行手势识别。When the difference between the vertical distance between the target hand and the preset plane and the preset distance is smaller than the second preset difference, then performing the processing of the target hand in the target image Gesture Recognition.
根据本公开的一个或多个实施例,本公开提供的手势识别装置中,所述手势识别模块具体用于:According to one or more embodiments of the present disclosure, in the gesture recognition device provided by the present disclosure, the gesture recognition module is specifically used for:
对所述目标图像进行手势分割、特征提取之后,基于提取的特征采用预设手势识别算法进行手势识别。After gesture segmentation and feature extraction are performed on the target image, gesture recognition is performed using a preset gesture recognition algorithm based on the extracted features.
根据本公开的一个或多个实施例,本公开提供了一种电子设备,包括:According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:
处理器;processor;
用于存储所述处理器可执行指令的存储器;memory for storing said processor-executable instructions;
所述处理器,用于从所述存储器中读取所述可执行指令,并执行所述指令以实现如本公开提供的任一所述的手势识别方法。The processor is configured to read the executable instructions from the memory, and execute the instructions to implement any gesture recognition method provided in the present disclosure.
根据本公开的一个或多个实施例,本公开提供了一种计算机可读存储介质,所述存储介质存储有计算机程序,所述计算机程序用于执行如本公开提供的任一所述的手势识别方法。According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium, the storage medium stores a computer program, and the computer program is used to perform any of the gestures provided in the present disclosure recognition methods.
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解,本公开中所涉及的公开范围,并不限于上述技术特征的特定组合而成的技术方案,同时也应涵盖在不脱离上述公开构思的情况下,由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。The above description is only a preferred embodiment of the present disclosure and an illustration of the applied technical principle. Those skilled in the art should understand that the disclosure scope involved in this disclosure is not limited to the technical solution formed by the specific combination of the above-mentioned technical features, but also covers the technical solutions formed by the above-mentioned technical features or Other technical solutions formed by any combination of equivalent features. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.
此外,虽然采用特定次序描绘了各操作,但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下,多任务和并行处理可能是有利的。同样地,虽然在上面论述中包含了若干具体实现细节,但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地,在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。In addition, while operations are depicted in a particular order, this should not be understood as requiring that the operations be performed in the particular order shown or performed in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while the above discussion contains several specific implementation details, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.
尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了本主题,但是应当理解所附权利要求书中所限定的主题未必局限于上面描述的特定特征或动作。相反,上面所描述的特定特征和动作仅仅是实现权利要求书的示例形式。Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims (12)

  1. 一种手势识别方法,其特征在于,包括:A gesture recognition method, characterized in that, comprising:
    获取目标图像;Get the target image;
    通过对所述目标图像进行运动识别,确定目标手部的水平运动稳定数据;Determine the horizontal motion stability data of the target hand by performing motion recognition on the target image;
    确定所述目标手部与预设平面的距离;determining the distance between the target hand and a preset plane;
    当基于所述水平稳定数据确定所述目标手部在水平方向稳定,并且基于所述目标手部与预设平面的垂直距离确定所述目标手部在垂直方向稳定,则对所述目标图像的所述目标手部进行手势识别。When it is determined based on the horizontal stability data that the target hand is stable in the horizontal direction, and based on the vertical distance between the target hand and a preset plane, it is determined that the target hand is stable in the vertical direction, then the target image The target hand performs gesture recognition.
  2. 根据权利要求1所述的方法,其特征在于,所述目标图像包括视频中提取的当前RGB图像帧和上一RGB图像帧,通过对所述目标图像进行运动识别,确定目标手部的水平运动稳定数据,包括:The method according to claim 1, wherein the target image includes the current RGB image frame and the previous RGB image frame extracted from the video, and the horizontal motion of the target hand is determined by performing motion recognition on the target image Stable data, including:
    基于所述当前RGB图像帧和所述上一RGB图像帧,采用光流算法计算所述当前RGB图像帧的光流场,并对所述光流场进行阈值分割,得到包括所述目标手部的前景区域和背景区域;Based on the current RGB image frame and the last RGB image frame, an optical flow algorithm is used to calculate the optical flow field of the current RGB image frame, and the optical flow field is thresholded to obtain the target hand the foreground and background regions of
    将所述前景区域的速度矢量和所述背景区域的速度矢量确定为所述目标手部的水平运动稳定数据。The velocity vector of the foreground area and the velocity vector of the background area are determined as the horizontal motion stabilization data of the target hand.
  3. 根据权利要求2所述的方法,其特征在于,基于所述水平运动稳定数据确定所述目标手部在水平方向稳定,包括:The method according to claim 2, wherein determining that the target hand is stable in the horizontal direction based on the horizontal motion stability data comprises:
    如果所述前景区域的速度矢量和所述背景区域的速度矢量的差值小于预设阈值,则确定所述目标手部在水平方向稳定。If the difference between the velocity vector of the foreground area and the velocity vector of the background area is smaller than a preset threshold, it is determined that the target hand is stable in the horizontal direction.
  4. 根据权利要求1所述的方法,其特征在于,所述目标图像包括第一时刻的第一深度图像和第二时刻的第二深度图像,确定所述目标手部与预设平面的垂直距离,包括:The method according to claim 1, wherein the target image comprises a first depth image at a first moment and a second depth image at a second moment, and the vertical distance between the target hand and a preset plane is determined, include:
    基于所述第一深度图像和所述第二深度图像,分别提取第一时刻和第二时刻所述目标手部与所述预设平面的第一垂直距离和第二垂直距离,所述第一深度图像和所述第二深度图像中均包括所述目标手部和所述预设平面。Based on the first depth image and the second depth image, respectively extract a first vertical distance and a second vertical distance between the target hand and the preset plane at the first moment and the second moment, the first Both the depth image and the second depth image include the target hand and the preset plane.
  5. 根据权利要求1所述的方法,其特征在于,确定所述目标手部与预设平面的垂直距离,包括:The method according to claim 1, wherein determining the vertical distance between the target hand and a preset plane comprises:
    采用距离传感器分别确定第一时刻和第二时刻所述目标手部与所述预设平面的第一垂直距离和第二垂直距离。A distance sensor is used to determine a first vertical distance and a second vertical distance between the target hand and the preset plane at the first moment and the second moment respectively.
  6. 根据权利要求4或5所述的方法,其特征在于,基于所述目标手部与预设平面的垂直距离确定所述目标手部在垂直方向稳定,包括:The method according to claim 4 or 5, wherein determining that the target hand is stable in the vertical direction based on the vertical distance between the target hand and a preset plane includes:
    如果所述目标手部与所述预设平面的所述第一垂直距离和所述第二垂直距离之间的差值小于第一预设差值,则确定所述目标手部在垂直方向稳定。If the difference between the first vertical distance and the second vertical distance between the target hand and the preset plane is smaller than a first preset difference, it is determined that the target hand is stable in the vertical direction .
  7. 根据权利要求1所述的方法,其特征在于,所述预设平面为水平平面或竖直平面。The method according to claim 1, wherein the preset plane is a horizontal plane or a vertical plane.
  8. 根据权利要求1所述的方法,其特征在于,在对所述目标图像的所述目标手部进行手势识别之前,还包括:The method according to claim 1, wherein before performing gesture recognition on the target hand of the target image, further comprising:
    判断所述目标手部与所述预设平面的垂直距离与预设距离的差值是否小于第二预设差值;judging whether the difference between the vertical distance between the target hand and the preset plane and the preset distance is smaller than a second preset difference;
    当所述目标手部与所述预设平面的垂直距离与所述预设距离的差值小于所述第二预设差值,则执行所述对所述目标图像的所述目标手部进行手势识别。When the difference between the vertical distance between the target hand and the preset plane and the preset distance is smaller than the second preset difference, then performing the processing of the target hand in the target image Gesture Recognition.
  9. 根据权利要求1所述的方法,其特征在于,对所述目标图像的所述目标手部进行手势识别,包括:The method according to claim 1, wherein performing gesture recognition on the target hand of the target image comprises:
    对所述目标图像进行手势分割、特征提取之后,基于提取的特征采用预设手势识别算法进行手势识别。After gesture segmentation and feature extraction are performed on the target image, gesture recognition is performed using a preset gesture recognition algorithm based on the extracted features.
  10. 一种手势识别装置,其特征在于,包括:A gesture recognition device, characterized in that it comprises:
    图像获取模块,用于获取目标图像;An image acquisition module, configured to acquire a target image;
    水平数据模块,用于通过对所述目标图像进行运动识别,确定目标手部的水平运动稳定数据;The horizontal data module is used to determine the horizontal motion stability data of the target hand by performing motion recognition on the target image;
    垂直数据模块,用于确定所述目标手部与预设平面的垂直距离;a vertical data module, configured to determine the vertical distance between the target hand and a preset plane;
    手势识别模块,用于当基于所述水平运动稳定数据确定所述目标手部在水平方向稳定,并且基于所述目标手部与预设平面的垂直距离确定所述目标手部在垂直方向稳定,则对所述目标图像的所述目标手部进行手势识别。The gesture recognition module is used to determine that the target hand is stable in the horizontal direction based on the horizontal motion stability data, and determine that the target hand is stable in the vertical direction based on the vertical distance between the target hand and a preset plane, Then perform gesture recognition on the target hand of the target image.
  11. 一种电子设备,其特征在于,所述电子设备包括:An electronic device, characterized in that the electronic device comprises:
    处理器;processor;
    用于存储所述处理器可执行指令的存储器;memory for storing said processor-executable instructions;
    所述处理器,用于从所述存储器中读取所述可执行指令,并执行所述指令以实现上述权利要求1-9中任一所述的手势识别方法。The processor is configured to read the executable instructions from the memory, and execute the instructions to implement the gesture recognition method described in any one of claims 1-9 above.
  12. 一种计算机可读存储介质,其特征在于,所述存储介质存储有计算机程序,所述计算机程序用于执行上述权利要求1-9中任一所述的手势识别方法。A computer-readable storage medium, characterized in that the storage medium stores a computer program, and the computer program is used to execute the gesture recognition method according to any one of claims 1-9.
PCT/CN2022/109467 2021-08-20 2022-08-01 Gesture recognition method and apparatus, and device and medium WO2023020268A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110962932.4A CN113642493B (en) 2021-08-20 2021-08-20 Gesture recognition method, device, equipment and medium
CN202110962932.4 2021-08-20

Publications (1)

Publication Number Publication Date
WO2023020268A1 true WO2023020268A1 (en) 2023-02-23

Family

ID=78423231

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/109467 WO2023020268A1 (en) 2021-08-20 2022-08-01 Gesture recognition method and apparatus, and device and medium

Country Status (2)

Country Link
CN (1) CN113642493B (en)
WO (1) WO2023020268A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113642493B (en) * 2021-08-20 2024-02-09 北京有竹居网络技术有限公司 Gesture recognition method, device, equipment and medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106970701A (en) * 2016-01-14 2017-07-21 芋头科技(杭州)有限公司 A kind of gesture changes recognition methods
CN107589850A (en) * 2017-09-26 2018-01-16 深圳睛灵科技有限公司 A kind of recognition methods of gesture moving direction and system
US20190034714A1 (en) * 2016-02-05 2019-01-31 Delphi Technologies, Llc System and method for detecting hand gestures in a 3d space
CN112306235A (en) * 2020-09-25 2021-02-02 北京字节跳动网络技术有限公司 Gesture operation method, device, equipment and storage medium
CN113642493A (en) * 2021-08-20 2021-11-12 北京有竹居网络技术有限公司 Gesture recognition method, device, equipment and medium

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100748174B1 (en) * 2005-12-27 2007-08-09 엠텍비젼 주식회사 Apparatus for detecting and correcting motion by hand in moving pictures
CN102779268B (en) * 2012-02-06 2015-04-22 西南科技大学 Hand swing motion direction judging method based on direction motion historigram and competition mechanism
KR101593950B1 (en) * 2014-05-28 2016-02-15 숭실대학교산학협력단 Apparatus for hand gesture based interface and pointing method using the same
CN112464833A (en) * 2020-12-01 2021-03-09 平安科技(深圳)有限公司 Dynamic gesture recognition method, device, equipment and storage medium based on optical flow

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106970701A (en) * 2016-01-14 2017-07-21 芋头科技(杭州)有限公司 A kind of gesture changes recognition methods
US20190034714A1 (en) * 2016-02-05 2019-01-31 Delphi Technologies, Llc System and method for detecting hand gestures in a 3d space
CN107589850A (en) * 2017-09-26 2018-01-16 深圳睛灵科技有限公司 A kind of recognition methods of gesture moving direction and system
CN112306235A (en) * 2020-09-25 2021-02-02 北京字节跳动网络技术有限公司 Gesture operation method, device, equipment and storage medium
CN113642493A (en) * 2021-08-20 2021-11-12 北京有竹居网络技术有限公司 Gesture recognition method, device, equipment and medium

Also Published As

Publication number Publication date
CN113642493A (en) 2021-11-12
CN113642493B (en) 2024-02-09

Similar Documents

Publication Publication Date Title
US11321593B2 (en) Method and apparatus for detecting object, method and apparatus for training neural network, and electronic device
CN109584276B (en) Key point detection method, device, equipment and readable medium
WO2018177379A1 (en) Gesture recognition, gesture control and neural network training methods and apparatuses, and electronic device
CN108198044B (en) Commodity information display method, commodity information display device, commodity information display medium and electronic equipment
US10204423B2 (en) Visual odometry using object priors
US10891473B2 (en) Method and device for use in hand gesture recognition
WO2022237811A1 (en) Image processing method and apparatus, and device
US9224064B2 (en) Electronic device, electronic device operating method, and computer readable recording medium recording the method
WO2022116947A1 (en) Video cropping method and apparatus, storage medium and electronic device
US10846565B2 (en) Apparatus, method and computer program product for distance estimation between samples
CN110163171B (en) Method and device for recognizing human face attributes
CN110660102B (en) Speaker recognition method, device and system based on artificial intelligence
KR20120044484A (en) Apparatus and method for tracking object in image processing system
WO2022105622A1 (en) Image segmentation method and apparatus, readable medium, and electronic device
CN111914756A (en) Video data processing method and device
WO2023020268A1 (en) Gesture recognition method and apparatus, and device and medium
CN112306235A (en) Gesture operation method, device, equipment and storage medium
CN112488095A (en) Seal image identification method and device and electronic equipment
EP2727047A1 (en) Method and apparatus for face tracking utilizing integral gradient projections
CN118097157A (en) Image segmentation method and system based on fuzzy clustering algorithm
WO2024120223A1 (en) Image processing method and apparatus, and device, storage medium and computer program product
WO2022194145A1 (en) Photographing position determination method and apparatus, device, and medium
WO2022194158A1 (en) Target tracking method and apparatus, device, and medium
CN116659646A (en) Fan blade vibration detection method and device based on machine vision
CN111310595A (en) Method and apparatus for generating information

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE