WO2020038108A1 - 动态动作检测方法、动态动作控制方法及装置 - Google Patents

动态动作检测方法、动态动作控制方法及装置 Download PDF

Info

Publication number
WO2020038108A1
WO2020038108A1 PCT/CN2019/093583 CN2019093583W WO2020038108A1 WO 2020038108 A1 WO2020038108 A1 WO 2020038108A1 CN 2019093583 W CN2019093583 W CN 2019093583W WO 2020038108 A1 WO2020038108 A1 WO 2020038108A1
Authority
WO
WIPO (PCT)
Prior art keywords
motion
queue
frame image
dynamic
static
Prior art date
Application number
PCT/CN2019/093583
Other languages
English (en)
French (fr)
Inventor
杜天元
于晨笛
杨雅迪
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Priority to KR1020217005497A priority Critical patent/KR102321562B1/ko
Priority to JP2021502886A priority patent/JP7127202B2/ja
Priority to SG11202013098TA priority patent/SG11202013098TA/en
Priority to US17/257,136 priority patent/US11455836B2/en
Publication of WO2020038108A1 publication Critical patent/WO2020038108A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm
    • G06V40/113—Recognition of static hand signs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm
    • G06V40/117—Biometrics derived from hands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person

Definitions

  • the present disclosure relates to the field of image processing technology, and in particular, to a dynamic motion detection method, a dynamic motion control method, and a device.
  • the present disclosure proposes a technical solution for dynamic motion detection.
  • a dynamic motion detection method including: adding frame images in a video stream to a first queue to obtain the first queue in which frame images are partially updated; Dynamic actions in frame images of a queue; if the dynamic actions do not match the result of the motion detection, update the motion detection result according to the dynamic motion, the motion detection result including a motion reference result or a previous detection Motion detection results.
  • a dynamic motion control method includes: acquiring a video stream; determining a motion detection result in the video stream by using the method according to any one of the foregoing; The corresponding operation of the motion detection result will be described.
  • a dynamic motion detection device includes a first queue update module configured to add a frame image in a video stream to a first queue to obtain the first updated frame image.
  • a queue configured to add a frame image in a video stream to a first queue to obtain the first updated frame image.
  • a queue a dynamic motion detection module for detecting a dynamic motion in a partially updated frame image of the first queue; a motion detection result update module for a case where the dynamic motion does not match the motion detection result And updating the motion detection result according to the dynamic motion, where the motion detection result includes a motion reference result or a previously detected motion detection result.
  • a dynamic motion control device includes: a video stream acquisition module for acquiring a video stream; and a motion detection result acquisition module for determining by using the device described in any one of the foregoing.
  • a motion detection result in the video stream an operation execution module, configured to control a device to perform an operation corresponding to the motion detection result.
  • an electronic device including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: execute the above-mentioned dynamic motion detection method and / or dynamic Motion control method.
  • a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions execute the above-mentioned dynamic motion detection method and / or dynamic motion control method when executed by a processor.
  • a computer program includes computer-readable code, and when the computer-readable code runs in an electronic device, a processor in the electronic device performs the dynamic action described above. Detection method and / or dynamic motion control method.
  • a frame image in a video stream may be added to the first queue, the first queue in which the frame image is partially updated is obtained, and a dynamic action in the frame image in the first queue after the partial update is detected;
  • the motion detection result is updated according to the dynamic motion.
  • dynamic motion detection is performed based on the frame images in the partially updated first queue, and the dynamic action can also be quickly determined based on the start or end frame of the dynamic action. It makes the detection result of dynamic action more accurate and the detection efficiency is higher.
  • the embodiments of the present disclosure can implement real-time tracking of dynamic actions in a video stream.
  • FIG. 1 shows a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure
  • FIG. 2 shows a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure
  • FIG. 3 shows a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure
  • FIG. 4 shows a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure
  • FIG. 5 illustrates a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure
  • FIG. 6 illustrates a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure
  • FIG. 7 illustrates a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure
  • FIG. 8 illustrates a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure
  • FIG. 9 illustrates a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure
  • FIG. 10 illustrates a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure
  • FIG. 11 illustrates a flowchart of a dynamic motion control method according to an embodiment of the present disclosure
  • FIG. 12 illustrates a flowchart of an application example of a motion recognition method according to an embodiment of the present disclosure
  • FIG. 13 illustrates a block diagram of a dynamic motion detection device according to an embodiment of the present disclosure
  • FIG. 14 illustrates a block diagram of a dynamic motion control device according to an embodiment of the present disclosure
  • Fig. 15 is a block diagram of an electronic device according to an exemplary embodiment.
  • the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, and C, may mean including from A, Any one or more elements selected from the set consisting of B and C.
  • numerous specific details are given in the detailed description below. Those skilled in the art should understand that the present disclosure can be implemented without certain specific details. In some examples, methods, means, components, and circuits that are well known to those skilled in the art have not been described in detail in order to highlight the gist of the present disclosure.
  • FIG. 1 illustrates a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure.
  • the dynamic motion detection method may be executed by an electronic device such as a terminal device or a server, where the terminal device may be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, or personal digital processing ( Personal Digital Assistant (PDA), handheld devices, computing devices, in-vehicle devices, wearable devices, etc.
  • the dynamic motion detection method may be implemented by a processor invoking computer-readable instructions stored in a memory.
  • the dynamic motion detection method includes:
  • step S10 the frame images in the video stream are added to the first queue, and the first queue in which the frame images are partially updated is obtained.
  • a set number of frame images need to be intercepted in the video stream for dynamic motion detection. After the detection is completed, the set number of frame images need to be intercepted. Used to detect the next dynamic action. The real-time performance of dynamic motion detection is poor.
  • the minimum number of frame images used for dynamic motion detection may be determined as the length of the first queue. For example, for the dynamic motion of forefinger extending and sliding to the right, the static motion of the hand in multiple frames of images is the forefinger extending.
  • the dynamic motion detection is performed based on less than N frame images, because the number of frame images is too small, the change in spatial position of static motion is too small. It is impossible to determine the change in spatial position of static motion, and dynamic motion detection cannot be performed. .
  • N is the minimum number of frame images that can be used for dynamic motion detection. .
  • N can be determined as the length of the first queue, where N is a positive integer.
  • the length of the first queue may be determined according to the content in the video stream, the accuracy requirement for dynamic motion detection, the speed requirement for dynamic motion detection, or previous experience values.
  • the present disclosure does not limit the length of the first queue.
  • the value range of N can be: 2 ⁇ N ⁇ 100, or 10 ⁇ N ⁇ 200, etc.
  • the value of N can be determined according to requirements.
  • the frame images in the video stream may be sequentially added to the first queue, and the first queue may be partially updated.
  • the frame image can be added from the rear end of the first queue, and the frontmost frame image of the first queue can be moved out, so that the frame image in the first queue is kept preset quantity.
  • a frame image in the video stream may be added to the first queue, and the first queue in which the frame image is partially updated is obtained.
  • Multiple frame images in the video stream that are shorter than the length of the first queue may also be added to the first queue to obtain the first queue in which the frame image is partially updated.
  • Step S20 Detect a dynamic action in the frame image of the first queue after being partially updated.
  • the frame images in the partially updated first queue may be used for detection to determine a dynamic action in each frame image.
  • the frame images in the partially updated first queue can be input to a neural network, and the neural network can detect dynamic actions in the frame images in the first queue.
  • dynamic motion detection may not be performed on the frame images in the first queue to save system resources.
  • the execution object of the dynamic action may include various movable objects such as a person, an animal, or a robot.
  • the dynamic action may include a change in a relative position of a set part of the execution object with respect to its environment.
  • the dynamic motion may include a motion made by a person using a forefinger slide, a motion made by a person using four limbs, or a motion made by a robot using a configured movable device to move forward and backward.
  • This disclosure does not limit the execution target of a dynamic action, nor does it limit the type of dynamic action.
  • step S30 if the dynamic motion does not match the motion detection result, the motion detection result is updated according to the dynamic motion, and the motion detection result includes a motion reference result or a previously detected motion detection result.
  • a frame image of a video stream may include multiple dynamic actions.
  • the motion detection result of the previous detection may be used as the motion detection result of the video stream. For example, three actions can be identified in the frame image of video stream 1, which are action 1, action 2, and action 3 in this order.
  • the action 1 may be determined as the action detection result of the previous detection as the action detection result.
  • the action detection result can be updated to action 2; when action 3 is recognized, the action detection result can be updated to action 3.
  • the motion detection result also includes a motion reference result. The given action can be set as the action reference result, or the action reference result can be set to null.
  • the dynamic actions in the frame images of multiple video streams can be analyzed, and the motion detection results are updated according to the multiple dynamic actions in the frame images of multiple video streams.
  • dynamic motion detection may be performed on video stream 1 and video stream 2 in sequence.
  • Action 1 and Action 2 can be identified in the frame image of Video Stream 1
  • Action 3 and Action 4 can be identified in the frame image of Video Stream 2.
  • motion 1 is identified from the frame image of video stream 1
  • motion 1 may be used as the motion detection result.
  • the action detection result may be updated to the action 2.
  • Motion recognition is performed based on the frame image of video stream 2.
  • motion detection result can be updated to motion 3.
  • the action detection result may be updated to the action 4.
  • One or more video streams for dynamic motion detection can be obtained according to requirements.
  • the motion detection result of the video stream may be obtained after performing dynamic motion detection based on a frame image of the video stream itself, or may be obtained after performing dynamic motion detection based on a frame image in another video stream.
  • the dynamic motion when the dynamic motion does not match the motion detection result, it may indicate that a new motion is determined based on the frame image in the video stream, and the motion detection result may be updated with the new motion to The action detection result can be updated in time.
  • the motion detection result in the embodiment of the present disclosure may be used to track the motion of the execution object in real time.
  • the video stream of the operator of the device can be captured, and the motion detection result of the captured video stream can be obtained using the dynamic motion detection method in the embodiment of the present disclosure, and the device can be controlled according to the motion detection result.
  • the real-time nature of the motion detection results can enable equipment operators to accurately control the equipment.
  • a frame image in a video stream may be added to a first queue to obtain the first queue in which the frame image is partially updated, and a dynamic action in the frame image in the first queue after the partial update is detected;
  • the motion detection result is updated according to the dynamic motion.
  • dynamic motion detection is performed based on the frame images in the partially updated first queue, and the dynamic action can also be quickly determined based on the start or end frame of the dynamic action. It makes the detection result of dynamic action more accurate and the detection efficiency is higher.
  • the embodiments of the present disclosure can implement real-time tracking of dynamic actions in a video stream.
  • FIG. 2 shows a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure.
  • a method for acquiring a motion detection result of a previous detection includes step S1, When there is a dynamic action in the frame images of the first queue, the frame images in the first queue are moved to the second queue.
  • Step S2 Determine the motion detection result according to the frame images in the second queue.
  • the motion detection result of the previous detection may be acquired in an initial state.
  • the first queue and the second queue may be empty, and the action detection result may be no action.
  • the frame images in the video stream may be input into the first queue, and the motion detection result determined according to the frame images in the first queue may be used as the motion detection result of the previous detection.
  • dynamic motion detection can be performed from the first frame image in the video stream. It is also possible to start dynamic motion detection on the frame image including the execution object detected in the video stream. This disclosure does not limit this.
  • the frame images of the video stream may be sequentially added to the first queue according to the time sequence, until the first queue can perform dynamic motion detection.
  • the length of the second queue may be equal to the length of the first queue, or may be different from the length of the first queue.
  • the method in the embodiment shown in FIG. 1 can be used to sequentially update the first queue according to each subsequent frame image to be identified, and according to the detected partially updated first queue, The motion in the frame image is updated with the motion detection result.
  • the method for acquiring the motion detection result of the previous detection may include sequentially adding frame images in the video stream to the first queue.
  • the frame images in the first queue are moved to the second queue, and a motion detection result is obtained according to the actions in the frame images of the second queue.
  • the previously detected motion detection result is a dynamic motion in the video stream.
  • FIG. 3 shows a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure.
  • the motion detection result includes a static motion.
  • step S10 in the dynamic motion detection method includes: Step S11, obtaining The static action in the frame image to be identified in the video stream is described.
  • the first queue is empty, and the second queue has frame images.
  • a frame image that has not been subjected to dynamic motion detection in the video stream may be used as a frame image to be identified.
  • the dynamic action may include a static action and a motion track of the static action.
  • dynamic action 1 is "extended index finger sliding right"
  • static action included in dynamic action 1 is index finger extension
  • motion trajectory of static motion is sliding to the right.
  • the dynamic action may include static information and switching information of the static action.
  • the switching information of the static actions may include switching information between at least two static actions.
  • dynamic action 2 is "change from fist to palm"
  • dynamic action 2 includes static action 1 palm and static action 2 punch
  • the switching information of dynamic action 2 is the switch from punch to palm.
  • the dynamic action may include a static action, switching information of the static action, and a motion track of the static action.
  • dynamic action 3 is a "long jump action”.
  • Dynamic action 3 includes static action 1 running and static action 2 jumping.
  • the motion track of static action 1 is to the right, the motion track of static action 2 is to the right.
  • the switching information between static movements is switched from running to jumping.
  • a static motion may be detected in each frame image to be identified.
  • the static motion in a plurality of frame images corresponding to one dynamic motion matches the static motion in the dynamic motion.
  • the static actions in at least two frame images can be compared to obtain the motion trajectory and / or switching information of the static actions in the frame images.
  • the motion trajectory and / or switching information of a static motion in a plurality of frame images corresponding to one dynamic motion matches the motion trajectory and / or switching information in a dynamic motion.
  • the static motion in each frame image to be identified in the video stream may be sequentially detected, and the static motion in each frame image to be identified and the static motion in the motion detection result are matched. It is also possible to match the static motion in each frame image to be identified with the static motion in the frame image in the second queue.
  • the motion detection result is a swipe of the index finger extended to the right, which can detect whether the static motion of the index finger in the frame image to be recognized matches the static motion of the index finger in the motion detection result.
  • the frame image to be identified may be added to the first queue.
  • the frame image to be identified may be added to the back end of the first queue, and the frame image at the front end of the first queue may be removed, and the first queue may be partially updated.
  • the dynamic actions in the frame images of the partially updated first queue can be detected, and if the detected dynamic actions do not match the motion detection results, the motion detection results are updated according to the dynamic actions.
  • a static motion in a frame image to be identified in the video stream may be obtained.
  • the frame image to be identified is added to the first queue.
  • the process of detecting a static motion in a frame image is simple and reliable.
  • the frame image to be identified can be quickly added to the first queue, and the first queue can be partially updated quickly to improve the detection efficiency of dynamic motion.
  • FIG. 4 shows a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure.
  • the motion detection result further includes a motion trajectory and / or switching information of the static motion.
  • the dynamic motion detection method Step S10 includes:
  • Step S13 when the static motion in the frame image to be identified matches the static motion in the motion detection result, determine according to the frame image to be identified and the frame image in the second queue The motion trajectory and / or switching information of the static motion in the frame image to be identified.
  • Step S14 when the motion trajectory and / or switching information of the static motion in the frame image to be identified does not match the motion detection result, add the frame image to be identified to the first queue to obtain The first image of the frame image is partially updated.
  • the dynamic action further includes a motion track and / or switching information of the static action.
  • the frame to be identified may be determined based on the frame image to be identified and the last frame image or multiple frame images in the second queue.
  • the motion trajectory and / or switching information of the static motion in the frame image can be obtained according to the position difference between static motion 1 in the frame image to be identified and static motion 2 in the last frame image in the second queue. .
  • the motion detection result is dynamic motion 1 "extended index finger slides to the right"
  • the static motion in the frame image to be recognized is the index finger extended, which matches the motion detection result.
  • the position difference between the static motion in the frame image to be recognized and the static motion in the frame image in the second queue it can be determined that the index finger in the frame image to be recognized is sliding to the left, and then the static motion in the frame image to be recognized is static.
  • the motion trajectory of the motion does not match the motion detection result.
  • the dynamic state to which the static motion in the frame image to be identified belongs can be considered.
  • the motion does not match the motion detection result.
  • the frame images to be identified may be added to the first queue, and the first queue may be partially updated.
  • the dynamic actions in the frame images of the partially updated first queue can be detected, and if the detected dynamic actions do not match the motion detection results, the motion detection results are updated according to the dynamic actions.
  • the frame image to be identified is added to the first queue to obtain a frame image portion. Updated first queue. According to the motion trajectory and / or switching information, determining whether the dynamic motion to which the static motion in the frame image to be identified matches the motion detection result can improve the accuracy of the motion detection result.
  • FIG. 5 shows a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure. As shown in FIG. 5, the dynamic motion detection method further includes:
  • step S40 when the dynamic motion detected based on the partially updated frame image of the first queue does not match the motion detection result, the second queue is cleared, and the frames in the first queue are cleared. Images are moved into the second queue.
  • Step S50 Update the motion detection result according to a dynamic motion in a frame image in the second queue.
  • a dynamic action in a frame image in the partially updated first queue may be detected. It can detect the motion trajectory and / or switching information of the static motion and the static motion in the frame images in the partially updated first queue, and determine the dynamic motion based on the detected static motion, the motion trajectory and / or the switching information of the static motion. When the dynamic motion in the frame images in the partially updated first queue does not match the motion detection result, it can be considered that a new dynamic motion appears in the frame images of the partially updated first queue.
  • the second queue may be emptied, and all the frame images in the first queue are moved to the second queue. And update the motion detection result according to the dynamic motion in the frame image in the updated second queue.
  • real-time dynamic motion detection may be performed on the video stream obtained by shooting, and electronic device control may be performed according to the dynamic motion obtained by real-time detection.
  • the second queue can be emptied, all the frame images in the first queue can be moved to the second queue, and according to the updated second queue,
  • the motion detection results are updated in the dynamic motion of the frame image in.
  • Real-time analysis of each frame image in the video stream can improve the real-time and accuracy of dynamic motion detection.
  • FIG. 6 shows a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure. As shown in FIG. 6, the dynamic motion detection method further includes:
  • Step S15 Add the frame image to be identified to the second frame when the static motion, the motion trajectory and / or switching information of the static motion in the frame image to be identified match the motion detection result. queue.
  • the static motion in the frame image to be identified matches the static motion in the motion detection result
  • the motion trajectory and / or switching information of the static motion in the frame image to be identified is also related to the motion.
  • the motion trajectory and / or switching information of the static motion in the detection result match, it can be considered that the dynamic motion to which the static motion belongs in the frame image to be identified matches the motion detection result.
  • the frame image to be identified may be added to the first frame image. Two queues.
  • the frame images to be identified may be added to the rear end of the second queue, and the frame images at the forefront of the second queue may be moved out, so that the preset number of frame images in the second queue is maintained.
  • the dynamic action has continuity. Among multiple consecutive frame images corresponding to one dynamic action, the static action, the motion trajectory, and / or the switching information of the static action in the two frame images spaced apart may be quite different. Adding to-be-recognized frame images that match the motion detection results to the second queue can keep the real-time nature of the dynamic motion in the frame images of the second queue, making the subsequent motion detection results of the frame images more accurate.
  • the first queue may be used to detect a dynamic action in a subsequent frame image to be identified in the video stream.
  • the second queue may be used to save a frame image used to determine a motion detection result.
  • a determination can be made together with the frame images in the second queue to determine whether the frame images to be identified have a dynamic motion that matches the motion detection result. If the frame image to be identified has a dynamic motion that matches the motion detection result, it is added to the second queue, so that the continuity of the dynamic motion in the frame image of the second queue is guaranteed, and the accuracy of the dynamic motion detection result is improved. rate.
  • the frame image to be identified does not have a dynamic motion that matches the motion detection result, it is added to the first queue, so that the frame image of the first queue can be used to detect a new dynamic motion, improving the timeliness of dynamic motion detection .
  • the embodiments of the present disclosure can realize accurate and efficient real-time tracking of dynamic actions in a video stream.
  • the frame image to be identified is added to the second queue.
  • the frame images matching the motion detection result can be added to the second queue, and the real-time nature of the dynamic motion in the frame images of the second queue is maintained, so that the subsequent motion detection results of the frame image are more accurate.
  • FIG. 7 shows a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure.
  • the dynamic motion includes at least one static motion and a motion trajectory of the static motion.
  • Step S20 includes: Step S21, obtaining the partially updated frames of the first queue according to the static motion in the frames of the first queue of the partially updated image and the motion trajectory of the static motions in the frames of the image. Dynamic action in the image.
  • the dynamic action may include a static action and a motion track of the static action. It is possible to detect the static motion in each frame of the first queue of the partially updated image, and if the static motion in each frame of the image matches, calculate the motion trajectory of the static motion in each frame of image, and in each frame of image In the case where the motion trajectory of the static motion matches, the dynamic motion in the frame images of the first queue after partial update can be obtained according to the motion trajectory of the static motion and the static motion. In this embodiment, the dynamic motion in the partially updated frame image can be obtained according to the static motion in each frame image and the motion trajectory of the static motion in each frame image of the partially updated first queue. Detecting dynamic motion according to the motion trajectory of the static motion and the static motion can improve the detection accuracy of the dynamic motion.
  • FIG. 8 shows a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure.
  • the dynamic motion includes: at least two static motions and switching information between the static motions.
  • Step S20 includes: Step S22, obtaining the partially updated dynamic motion in the frame image of the first queue according to the static motion and the static motion switching information in the frames of the first queue after the partial update. .
  • the dynamic action may include static information and switching information of the static action. It is possible to detect the static motion in each frame of the first queue of the partially updated image. If the static motion in each frame of the image matches, the switching information of the static motion in each frame of image is calculated. In the case where the switching information of the static motion matches, the dynamic motion in the frame image of the partially updated first queue can be obtained according to the switching information of the static motion and the static motion. In this embodiment, the dynamic motion in the partially updated frame image may be obtained according to the static motion in each frame image and the switching of the static motion in each frame image in the partially updated first queue. Detecting a dynamic action according to the switching information of the static action and the static action can improve the accuracy of detecting the dynamic action.
  • FIG. 9 shows a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure.
  • the dynamic motion includes at least two static motions, motion trajectories of each static motion, and switching information between the static motions, as shown in FIG. 9.
  • Step S20 in the dynamic motion detection method includes: Step S23, obtaining part of the static motion in each frame of the first queue image, the motion trajectory of each static motion, and switching information between the static motions according to the partially updated image.
  • the updated dynamic actions in the frame images of the first queue includes: Step S23, obtaining part of the static motion in each frame of the first queue image, the motion trajectory of each static motion, and switching information between the static motions according to the partially updated image.
  • the updated dynamic actions in the frame images of the first queue includes: Step S23, obtaining part of the static motion in each frame of the first queue image, the motion trajectory of each static motion, and switching information between the static motions according to the partially updated image.
  • the dynamic action may include a static action, a motion track of each static action, and switching information of the static action. It is possible to detect the static motion in each frame of the first queue of the partially updated image. When the static motion in each frame of the image matches, calculate the motion trajectory of each static motion in each frame of image and switch between static motions. Information, in the case that the motion trajectory of each static motion and the switching information between static motions in each frame of the image also match, according to the static motion, the motion trajectory of each static motion, and the switching information of the static motion, a partially updated first Dynamic actions in a queue of frame images.
  • the partially updated frame image of the first queue is obtained according to the static motion, the motion trajectory of each static motion, and the switching information between the static motions in the frames of the first queue after partial update Dynamic action in. Detecting dynamic motions based on static motions, motion trajectories of each static motion, and switching information between static motions can improve the accuracy of detecting dynamic motions.
  • FIG. 10 shows a flowchart of a dynamic motion detection method according to an embodiment of the present disclosure.
  • the dynamic motion includes a dynamic motion of a hand, and the static motion includes a gesture.
  • step S11 in the dynamic motion detection method. Including: step S111, detecting whether a hand is included in the frame image to be identified.
  • step S112 when a hand is included in the image to be identified, a gesture in the image to be identified is obtained according to a finger and / or a palm on the hand.
  • the dynamic actions may include various types of dynamic actions such as hand dynamic actions and limb dynamic actions.
  • the performers of the dynamic movements of the hand may include humans, animals, robots, and the like.
  • the hand dynamic motion may include a dynamic motion of a human hand or a dynamic motion of a robot's hand. This disclosure does not limit this.
  • the frame image to be identified may be input to a neural network, and the frame image to be identified is subjected to convolution processing and other operations to detect whether the frame image to be identified includes a hand. When the detection result does not include a hand, subsequent analysis of the frame image to be recognized may not be performed to save system resources.
  • a gesture may be further determined. Gestures can be obtained from fingers on the hand. For example, the gesture can be determined to be "five" based on all five fingers being extended. The gesture can also be obtained based on the palm of the hand, for example, the gesture can be determined to be "up” or “down” according to palm up or palm down. You can also get gestures based on the fingers and palms of your hands. For example, you can determine that the gesture is "press down” based on the five fingers extended with the palms down. According to requirements, it can be determined that a gesture is obtained based on a finger and / or a palm. This disclosure does not limit this.
  • the image in the to-be-recognized image may be obtained according to the fingers and / or palm of the hand. gesture.
  • step S112 includes: identifying a gesture in the image to be identified according to a position of a first key point of a finger on the hand and / or a position of a second key point of the palm,
  • the first key point of the finger includes a fingertip or a knuckle
  • the second key point of the palm includes a point on the palm, the back of the palm, or the edge of the palm.
  • the first key point may be determined on a finger.
  • One or more first key points can be identified on one finger.
  • the point of the knuckle or fingertip position may be determined as the first key point.
  • a second key point can be identified on the palm.
  • the palm can include one or more key points.
  • a point on the palm, the back of the palm, or the edge of the palm may be determined as the second key point.
  • the gesture can be determined according to the position of the first key point of the finger, the gesture can also be determined according to the position of the second key point of the palm, and the position of the first key point of the finger and the position of the palm can also be determined. After the positions of the second key points are combined, a gesture is determined.
  • a gesture in a frame image to be recognized may be determined according to a position of a first key point of a finger and / or a position of a second key point of a palm. Gestures determined based on key points of the hand are accurate and reliable.
  • the dynamic motion of the hand further includes a motion trajectory and / or switching information of the gesture
  • step S13 in the dynamic motion detection method includes: according to the first position in the frame image to be identified. A key point and a first key point in a frame image of the second queue corresponding to the key point, determining a motion trajectory and / or switching information in the frame image to be identified, and / or, according to the to-be-recognized
  • the second key point in the frame image and the second key point in the frame image of the second queue corresponding to the second key point determine the motion trajectory and / or switching information in the frame image to be identified.
  • the first key point in the frame image to be identified and the first key point in the frame image of the second queue corresponding to the first key point can be determined.
  • the position difference between them determines the motion trajectory and / or switching information in the frame image to be identified.
  • the second key point in the frame image to be identified and the second key point in the frame image of the second queue corresponding to the second key point determines the motion trajectory and / or switching information in the frame image to be identified.
  • the first key point in the frame image to be recognized and the first in the frame image of the second queue corresponding to the first key point can be determined.
  • the position difference between the key points and the position difference between the second key point in the frame image to be identified and the second key point in the frame image of the second queue corresponding thereto may be jointly determined.
  • the second key point in the image and the second key point in the frame image of the second queue corresponding to the second key point determine the motion trajectory and / or switching information in the frame image to be identified. According to the first key point and / or the second key point, accurate motion trajectory and / or switching information can be obtained in the frame image to be identified, so that the detection result of the dynamic motion is accurate and reliable.
  • the dynamic motion includes a static motion and a motion trajectory of the static motion.
  • Step S20 includes: detecting a static motion in a frame image of the first queue; and calculating a static motion in a target frame image. Relative to the cumulative movement distance of the static motion in the first frame image of the first queue in the X-axis direction and the Y-axis direction, the target frame image includes the first frame except the first frame Any frame image outside the image; and obtaining a dynamic motion in the frame image of the first queue according to the static motion in the target frame image and the accumulated moving distance.
  • an image coordinate system can be established for the frame images, the coordinate position of the static motion in each frame image in the image coordinate system is determined, and the coordinate values of the static motion on the X axis and the Y axis are obtained.
  • the static motion is a gesture
  • the coordinate values of the first key point of the hand on the X and Y axes, and / or the coordinate values of the second key point of the hand on the X and Y axes can be used.
  • the coordinate values of the gesture on the X and Y axes can be used.
  • the cumulative movement distances of the static motions in each frame image in the X-axis direction and the Y-axis direction relative to the static motions in the first frame image of the first queue may be calculated.
  • the cumulative moving distance may include an array composed of the cumulative moving distance of the X axis and the cumulative moving distance of the Y axis.
  • the moving direction and distance of the static motion in each frame image relative to the static motion in the first frame image can be determined.
  • the dynamic motion in the first queue may be determined according to the static motion in each frame of images in the first queue and the cumulative moving distance of the static motion.
  • the cumulative moving distance of static motion in each frame of image can be determined according to the position change of each frame of image in the image coordinate system. According to the image coordinate system, the cumulative moving distance of static motion in each frame of image can be accurately obtained. The result of motion recognition is accurate and reliable, and the error rate of motion recognition is reduced.
  • the obtaining the dynamic motion in the frame image of the first queue according to the static motion in the target frame image and the cumulative moving distance includes: statically in the target frame image If the cumulative movement distance of the movement is less than the movement threshold, it is determined that the dynamic movement of the target object in the frame image of the first queue is an intermittent movement.
  • the dynamic action has continuity.
  • the static action moves to a certain distance in the spatial position, it can be determined that the dynamic action is generated.
  • the moving distance of the static action in the spatial position is too small, the performer can be considered to have stopped performing the dynamic action.
  • the movement threshold can be determined according to requirements. When the cumulative movement distance of the static motion in the frame image is less than the movement threshold, it can be considered that the dynamic motion in the frame image has a discontinuity.
  • the movement threshold can be used to accurately determine whether the motion in the first queue frame image is an intermittent motion, so that the result of motion recognition is accurate and reliable.
  • the obtaining the dynamic motion in the frame image of the first queue according to the static motion in the target frame image and the cumulative moving distance includes: the cumulative moving distance is greater than or equal to In the case of the movement threshold, calculate a ratio between the cumulative movement distance of the static motion in each of the frame images with respect to the width of the target object to which the static motion belongs; when the ratio is less than the ratio threshold, determine The motion trajectory of the static motion in the target frame image is a rotational motion trajectory; according to the static motion in the target frame image and the rotational motion trajectory, a dynamic motion in the frame image of the first queue is obtained.
  • the dynamic action may include a rotation action.
  • the dynamic movement of the hand can be taken as an example.
  • a ratio between the cumulative movement distance of the gesture in each of the frame images to the width of the hand may be calculated.
  • the cumulative movement distance of a gesture can be represented by a vector in the image coordinate system.
  • the ratio between the size of the vector and the width of the hand can be calculated.
  • the ratio is less than the ratio threshold, it can be considered that the motion trajectory of a gesture in each frame of image is a rotational motion trajectory.
  • a ratio between a cumulative movement distance of a static motion in each frame image and a width of a target object to which the static motion belongs may be calculated; and in a case where the ratio is less than a threshold value of the ratio, determining a target frame image
  • the motion trajectory of the static motion is the rotation motion trajectory. According to the accumulated moving distance and the ratio threshold, the detection result of the dynamic action can be made more accurate.
  • the method further includes: in a case where the ratio is greater than or equal to the ratio threshold, determining a motion trajectory of a static motion in a target frame image as a trajectory of translation; according to the target frame image The static motion and the horizontal movement are used as trajectories to obtain the dynamic motion in the frame image of the first queue.
  • the motion trajectory of the static motion may include a flat movement as a trajectory.
  • the ratio is greater than or equal to the ratio threshold, it can be considered that the motion trajectory of the gesture in each frame of image is a trajectory of translation.
  • the motion trajectory of the static motion is the trajectory of the horizontal motion, so that the detection result of the dynamic motion is more accurate.
  • FIG. 11 illustrates a flowchart of a dynamic motion control method according to an embodiment of the present disclosure.
  • the dynamic motion control method may be performed by an electronic device such as a terminal device or a server, where the terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, or a personal digital processing ( Personal Digital Assistant (PDA), handheld devices, computing devices, in-vehicle devices, wearable devices, etc.
  • the dynamic motion control method may be implemented by a processor invoking computer-readable instructions stored in a memory.
  • the dynamic motion control method includes: Step S100, acquiring a video stream.
  • Step S110 Determine any motion detection result in the video stream by using any one of the foregoing dynamic motion detection methods.
  • Step S120 The control device performs an operation corresponding to the motion detection result.
  • a shooting device may be set on a device that needs to perform a control operation, and a video stream may be obtained in real time by using the shooting device.
  • the video stream captured by the shooting device can also be obtained.
  • Video streams can be captured with different cameras.
  • the photographing device may include a binocular camera, a depth camera, or an ordinary camera. According to the needs of dynamic motion detection, different types of cameras can be used to shoot video streams.
  • step S120 includes: obtaining an operation instruction corresponding to the motion detection result according to a predetermined relationship between a predetermined motion detection result and an operation instruction; and controlling the device to execute the operation instruction according to the operation instruction. Operate accordingly.
  • a correspondence relationship between a motion detection result and an operation instruction may be preset.
  • one action detection result corresponds to one operation instruction, or multiple action detection results correspond to one operation instruction.
  • the type and content of the operation instruction can be determined according to the type of the operated device and the operation requirements. The present disclosure does not limit the form and specific content of the operation instruction.
  • an operation instruction may be output to control the operated device. Since the motion detection result can track the motion in the video stream in real time, the output operation instructions can also track the motion of the execution object in the video stream in real time, so that the operator can precisely control the operated device.
  • an operation instruction corresponding to the motion detection result may be determined and output according to the correspondence relationship and the motion detection result. Based on the real-time and accuracy of the motion detection results, the operation instructions can also track the motion of the execution object in the video stream in real time, so that the operator can operate the device more accurately.
  • the controlling the device to perform a corresponding operation according to the operation instruction includes controlling a vehicle window, a door, a vehicle-mounted system, or a vehicle-mounted device of the vehicle according to the operation instruction.
  • a monitoring device may be provided in the vehicle, and a monitoring video of a driver or an occupant in the vehicle may be taken as a video stream.
  • the captured video stream can be dynamically detected in real time.
  • an operation instruction corresponding to the motion detection result may be obtained; and then the vehicle window, door, vehicle-mounted system, or vehicle-mounted device (for example, vehicle Multimedia devices, on-board robots, etc.).
  • control device performs an operation corresponding to the motion detection result, including:
  • the vehicle In response to the detection result being a predefined dynamic action, the vehicle is controlled to perform an operation corresponding to the predefined dynamic action.
  • the predefined dynamic motion includes a dynamic gesture
  • the dynamic gesture includes at least one of the following: single-finger clockwise / counterclockwise rotation, palm left / right wave, two-finger forward poking, thumb and Little finger extended, palm pressed down, thumb extended left / right movement, palm left / right slide, palm up fist changed to palm, palm up fist changed, palm down fist changed, palm down changed Fist, single-finger swipe, multi-finger pinch, single-finger double-click, single-finger double-click, multi-finger double-click, multi-finger click; operations corresponding to the predefined dynamic actions include at least one of the following: Small, song switching, song pause / resume, answer or start a call, hang up or reject a call, increase or decrease the air conditioner temperature, multi-screen interaction, open the sunroof, close the sunroof, lock the door lock, unlock the door lock, drag Navigate, zoom out, zoom in.
  • a single-finger dynamic gesture of clockwise / counterclockwise rotation can be used to adjust the volume of the vehicle's audio equipment.
  • a dynamic gesture of pressing down with the palm of the hand can be used to hang up or reject the communication device of the vehicle.
  • the dynamic gesture of left / right movement with the thumb extended can be used to increase or decrease the temperature of the air conditioner of the vehicle.
  • Multi-screen interactive operation can be performed on the display screen of the vehicle by using the dynamic gesture of left / right long sliding of the palm of the hand.
  • the dynamic gesture of turning the palm upwards into a palm can be used to open the sunroof of the vehicle (the set length can be opened each time, for example, 10 cm each time). You can use the dynamic gesture of turning your palm upwards into a fist to close the sunroof on the vehicle.
  • the dynamic gesture of turning the palm downwards into a punch can be used to lock the door of the vehicle.
  • the dynamic gesture of turning the palm down into the palm can be used to unlock the door lock of the vehicle.
  • a dynamic gesture of one-finger sliding can be used to perform drag navigation operations on the vehicle's navigation device.
  • a single-finger double-tap dynamic gesture can be used to zoom in on the vehicle's navigation device.
  • an operation instruction may be used to control a vehicle window, a door, a vehicle-mounted system, or a vehicle-mounted device.
  • the dynamic actions identified in the video stream can be used to perform different operations on the vehicle itself or the on-board system on the vehicle. Based on the dynamic motion detection method in the embodiment of the present disclosure, the operator can accurately control the vehicle or the vehicle-mounted system.
  • FIG. 12 shows a flowchart of an application example of a motion recognition method according to an embodiment of the present disclosure, as shown in FIG. 12:
  • a depth camera can be configured in the vehicle to use the depth camera to obtain the driver's monitoring image as a video stream.
  • motion recognition is performed on a driver's hand motion.
  • step S210 the first queue and the second queue are set to empty, and the motion detection result is also set to empty. Add the frame images in the video stream to the first queue in order of the time period.
  • step S220 it is detected whether there is a dynamic action in the frame images of the first queue. If there is no dynamic action, proceed to step S230; if there is dynamic action, proceed to step S240.
  • the length of the first queue may be ten frames of images.
  • the dynamic motion is a dynamic motion of the hand.
  • whether there is a hand in the frame image can be sequentially identified, and when there is a hand in the frame image of the first queue, a gesture in the image to be identified can be identified according to a finger on the hand and / or a palm.
  • the motion trajectory and / or switching information of the gestures in the frames of images can be determined.
  • the dynamic motion in the first queue can be detected.
  • Step S230 Continue to add each frame image in the video stream to the first queue in the order of time, and then skip to step S220.
  • the eleventh frame image may be added to the back end of the first queue, and the first frame image at the front of the first queue may be moved out of the first queue.
  • the first queue includes frame images from the second frame to the eleventh frame.
  • it can be determined whether there is an action in the first queue at this time. If there is no action, the twelfth frame image can be added to the first queue, and the second frame image can be removed, until it is determined that there is motion in the frame image of the first queue according to step S220.
  • Step S240 Move the frame images in the first queue to the second queue.
  • the first queue is emptied, and the motion detection result is determined according to the frame images in the second queue to obtain the previously detected motion detection result.
  • the first queue is empty, the second queue has ten frames of images, and is the twenty-third to thirty-second frames in the video stream. Frame by frame image.
  • the motion detection result may be determined according to the motion in the frame image of the second queue.
  • Frame images in the video stream without motion recognition can be used for subsequent analysis as frame images to be identified. That is, the frame image starting from the frame image of the thirty-third frame can be used as the frame image to be identified, and enters step S250 of subsequent dynamic motion detection.
  • Step S250 determining in turn whether each frame image to be identified in the video stream matches the motion detection result. If yes, go to step S260; if no, go to step S270.
  • the 33rd frame image and the motion detection result can be determined according to the 33rd frame image and the last frame image (the 32nd frame image) in the second queue. Whether it matches.
  • step S260 if there is a match, the frame image to be identified is added to a second queue.
  • the frame image in the second queue is updated to the twenty-fourth frame to the thirty-third frame.
  • step S270 if there is no match, the frame image to be identified is added to the first queue.
  • Step S280 Determine whether the dynamic motion in the frame image of the first queue matches the motion detection result. If they do not match, proceed to step S290.
  • step S290 if the dynamic motion in the frame images of the first queue does not match the motion detection result, the second queue is cleared, and the frame images in the first queue are moved to the second queue.
  • the motion detection result is updated according to the motion in the frame image of the updated second queue.
  • the present disclosure also provides a dynamic motion detection device, a dynamic motion control device, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any of the dynamic motion detection methods and dynamic motion control methods provided by the disclosure,
  • a dynamic motion detection device a dynamic motion control device
  • an electronic device a computer-readable storage medium
  • a program all of which can be used to implement any of the dynamic motion detection methods and dynamic motion control methods provided by the disclosure, The corresponding technical solutions and descriptions and the corresponding records in the method section are not repeated here.
  • FIG. 13 shows a block diagram of a dynamic motion detection device according to an embodiment of the present disclosure.
  • the dynamic motion detection device includes a first queue update module 10 for adding a frame image in a video stream to a first A queue to obtain the first queue in which the frame image is partially updated; a dynamic action detection module 20 for detecting a dynamic action in the frame image of the first queue after being partially updated; an action detection result update module 30 for When the dynamic motion does not match the motion detection result, the motion detection result is updated according to the dynamic motion, and the motion detection result includes a motion reference result or a previously detected motion detection result.
  • the apparatus further includes: a previously detected motion detection result acquisition module, configured to, if there is a dynamic motion in a frame image of the first queue, move the first queue The frame images in are moved to the second queue; the motion detection result is determined according to the frame images in the second queue.
  • a previously detected motion detection result acquisition module configured to, if there is a dynamic motion in a frame image of the first queue, move the first queue The frame images in are moved to the second queue; the motion detection result is determined according to the frame images in the second queue.
  • the motion detection result includes a static motion
  • the first queue update module 10 includes a first acquisition sub-module for acquiring a frame image to be identified in the video stream.
  • a static motion a first update submodule, configured to add the frame image to be identified to the condition that the static motion in the frame image to be identified does not match the static motion in the motion detection result
  • the first queue obtains the first queue in which the frame image is partially updated.
  • the motion detection result further includes a motion trajectory and / or switching information of the static motion.
  • the first queue update module 10 includes: a second acquisition submodule, configured to When the static motion in the frame image to be identified matches the static motion in the motion detection result, the to-be-recognized image is determined according to the frame image to be identified and the frame image in the second queue.
  • a motion trajectory and / or switching information of a static motion in a frame image a second update submodule, configured to: In the case, the frame image to be identified is added to the first queue, and the first queue in which the frame image is partially updated is obtained.
  • the apparatus further includes: a queue transfer module, configured to: when the dynamic motion detected according to the partially updated frame image of the first queue does not match a motion detection result Next, the second queue is emptied, and the frame images in the first queue are moved to the second queue.
  • the motion detection result update module is configured to perform dynamic actions based on the frame images in the second queue. To update the motion detection result.
  • real-time dynamic motion detection may be performed on the video stream obtained by shooting, and electronic device control may be performed according to the dynamic motion obtained by real-time detection.
  • the second queue can be emptied, all the frame images in the first queue can be moved to the second queue, and according to the updated second queue,
  • the motion detection results are updated in the dynamic motion of the frame image in.
  • Real-time analysis of each frame image in the video stream can improve the real-time and accuracy of dynamic motion detection.
  • the apparatus further includes: a second queue update module, configured to perform static motion in the frame image to be identified, motion trajectory and / or switching information of the static motion and the motion. When the detection results match, the frame image to be identified is added to the second queue.
  • the dynamic motion includes: at least one static motion and a motion trajectory of the static motion
  • the dynamic motion detection module 20 includes: a first dynamic motion detection sub-module for The motion trajectories of the static motion in each frame image of the first queue and the static motion in each frame image after the update are obtained to obtain the partially updated dynamic motion in the frame image of the first queue.
  • the dynamic action includes: at least two static actions and switching information between the static actions
  • the dynamic action detection module 20 includes: a second dynamic action detection sub-module for The updated information of the static motion and the static motion in each frame image of the first queue is updated to obtain a partially updated dynamic motion in the frame image of the first queue.
  • the dynamic motion includes: at least two static motions, motion trajectories of each static motion, and switching information between the static motions
  • the dynamic motion detection module 20 includes: a third dynamic motion detection A sub-module, configured to obtain the partially updated frames of the first queue according to the static movements in each frame of the first queue image, the motion trajectory of each static movement, and the switching information between the static movements Dynamic action in the image.
  • the dynamic motion includes a dynamic motion of a hand
  • the static motion includes a gesture
  • the first acquisition submodule includes a hand detection unit for detecting the frame to be identified.
  • a gesture determination unit configured to obtain a gesture in the to-be-recognized image according to a finger and / or a palm of the hand when the to-be-recognized image includes a hand.
  • the gesture determination unit is configured to identify the image to be identified according to a position of a first key point of a finger on the hand and / or a position of a second key point of the palm.
  • the first key point of the finger includes a fingertip or a knuckle
  • the second key point of the palm includes a point on the palm, the back of the palm, or the edge of the palm.
  • the dynamic motion of the hand further includes a motion trajectory and / or switching information of the gesture
  • the second acquisition submodule includes:
  • a first obtaining unit configured to determine the first key point in the frame image to be identified and the first key point in the frame image of the second queue corresponding to the first key point in the frame image to be identified; Movement track and / or switching information, and / or
  • a second obtaining unit configured to determine the second key point in the frame image to be identified according to the second key point in the frame image to be identified and the second key point in the frame image of the second queue corresponding to the second key point And / or switch information.
  • the dynamic motion includes a static motion and a motion trajectory of the static motion.
  • the dynamic motion detection module 20 includes a static motion detection sub-module for detecting frame images of the first queue.
  • the static motion in; the cumulative movement distance determination sub-module for calculating the accumulation of the static motion in the target frame image in the X-axis direction and the Y-axis direction relative to the static motion in the first frame image of the first queue
  • the target frame image includes any frame image in the first queue except the first frame image;
  • a fourth dynamic motion detection sub-module is configured to The cumulative moving distance obtains a dynamic action in a frame image of the first queue.
  • the fourth dynamic motion detection submodule is configured to determine a frame of the first queue when a cumulative movement distance of a static motion in the target frame image is less than a movement threshold.
  • the dynamic motion of the target object in the image is an intermittent motion.
  • the fourth dynamic motion detection sub-module is configured to calculate a cumulative motion of a static motion in each frame image when the cumulative motion distance is greater than or equal to the motion threshold.
  • the fourth dynamic motion detection sub-module is further configured to: when the ratio is greater than or equal to the ratio threshold, determine the motion trajectory of the static motion in the target frame image as a translation movement operation. Trajectory; obtaining a dynamic motion in the frame image of the first queue according to the static motion in the target frame image and the translation movement.
  • the dynamic motion control device includes a video stream acquisition module 100 for acquiring a video stream, and a motion detection result acquisition module 200. It is used to determine the motion detection result in the video stream by using the dynamic motion detection device described above; and an operation execution module 300 is configured to control the device to perform an operation corresponding to the motion detection result.
  • a shooting device may be set on a device that needs to perform a control operation, and a video stream may be obtained in real time by using the shooting device.
  • the video stream captured by the shooting device can also be obtained.
  • Video streams can be captured with different cameras.
  • the photographing device may include a binocular camera, a depth camera, or an ordinary camera. According to the needs of dynamic motion detection, different types of cameras can be used to shoot video streams.
  • the operation execution module 300 includes: an operation instruction acquisition submodule, configured to acquire an operation corresponding to the motion detection result according to a correspondence between a predetermined motion detection result and the operation instruction. Instruction; an operation execution submodule, configured to control the device to perform a corresponding operation according to the operation instruction.
  • the operation execution sub-module is configured to control a vehicle window, a door, a vehicle-mounted system, or a vehicle-mounted device according to the operation instruction.
  • a monitoring device may be provided in the vehicle, and a monitoring video of a driver or an occupant in the vehicle may be taken as a video stream.
  • the captured video stream can be dynamically detected in real time.
  • an operation instruction corresponding to the motion detection result may be obtained; and then the vehicle window, door, vehicle-mounted system, or vehicle-mounted device (for example, vehicle Multimedia devices, on-board robots, etc.).
  • the operation execution module 300 is further configured to: in response to the detection result being a predefined dynamic action, control the vehicle to perform an operation corresponding to the predefined dynamic action.
  • the predefined dynamic motion includes a dynamic gesture
  • the dynamic gesture includes at least one of the following: single-finger clockwise / counterclockwise rotation, palm left / right wave, two-finger forward poking, thumb and Little finger extended, palm pressed down, thumb extended left / right movement, palm left / right slide, palm up fist changed palm, palm up fist changed, palm down fist changed, palm down fist changed Palm, single-finger swipe, multi-finger pinch, single-finger double-click, single-finger double-click, multi-finger double-click, multi-finger click; operations corresponding to the predefined dynamic actions include at least one of the following: Small, song switching, song pause / resume, answer or start a call, hang up or reject a call, increase or decrease the air conditioner temperature, multi-screen interaction, open the sunroof, close the sunroof, lock the door lock, unlock the door lock, drag Navigate, zoom out, zoom in.
  • the functions provided by the device provided in the embodiments of the present disclosure or the modules included may be used to execute the method described in the foregoing method embodiment.
  • the description of the method embodiment above The technical effects of the device may be Refer to the description of the corresponding method embodiment. For brevity, it will not be repeated here.
  • An embodiment of the present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, which are implemented by the processor when the computer program instructions are executed by the processor.
  • the computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium.
  • An embodiment of the present disclosure further provides an electronic device including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute any one of the foregoing methods.
  • the electronic device may be provided as a terminal, a server, or other forms of devices.
  • An embodiment of the present disclosure also provides a computer program, where the computer program includes computer-readable code, and when the computer-readable code runs in an electronic device, a processor in the electronic device executes any method implementation of the present disclosure. example.
  • Fig. 15 is a block diagram of an electronic device 800 according to an exemplary embodiment.
  • the electronic device 800 may be a terminal such as a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.
  • the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, and a sensor component 814 , And communication component 816.
  • the processing component 802 generally controls overall operations of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations.
  • the processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the method described above.
  • the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components.
  • the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.
  • the memory 804 is configured to store various types of data to support operation at the electronic device 800. Examples of such data include instructions for any application or method for operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, and the like.
  • the memory 804 may be implemented by any type of volatile or non-volatile storage devices, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), Programming read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read-only memory
  • EPROM Programming read-only memory
  • PROM programmable read-only memory
  • ROM read-only memory
  • magnetic memory flash memory
  • flash memory magnetic disk or optical disk.
  • the power component 806 provides power to various components of the electronic device 800.
  • the power component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 800.
  • the multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and a user.
  • the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user.
  • the touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or slide action, but also detect duration and pressure related to the touch or slide operation.
  • the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.
  • the audio component 810 is configured to output and / or input audio signals.
  • the audio component 810 includes a microphone (MIC).
  • the microphone is configured to receive an external audio signal.
  • the received audio signal may be further stored in the memory 804 or transmitted via the communication component 816.
  • the audio component 810 further includes a speaker for outputting audio signals.
  • the I / O interface 812 provides an interface between the processing component 802 and a peripheral interface module.
  • the peripheral interface module may be a keyboard, a click wheel, a button, or the like. These buttons can include, but are not limited to: a home button, a volume button, a start button, and a lock button.
  • the sensor component 814 includes one or more sensors for providing various aspects of the state evaluation of the electronic device 800.
  • the sensor component 814 may detect the on / off state of the electronic device 800, and the relative positioning of the components, such as the display and keypad of the electronic device 800.
  • the sensor component 814 may also detect the electronic device 800 or an electronic device 800.
  • the position of the component changes, the presence or absence of the user's contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800.
  • the sensor component 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact.
  • the sensor component 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications.
  • the sensor component 814 may further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
  • the communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices.
  • the electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof.
  • the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel.
  • the communication component 816 further includes a near field communication (NFC) module to facilitate short-range communication.
  • the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
  • RFID radio frequency identification
  • IrDA infrared data association
  • UWB ultra wideband
  • Bluetooth Bluetooth
  • the electronic device 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), Implementation of a programming gate array (FPGA), controller, microcontroller, microprocessor, or other electronic component to perform the above method.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DSPDs digital signal processing devices
  • PLDs programmable logic devices
  • FPGA programming gate array
  • controller microcontroller, microprocessor, or other electronic component to perform the above method.
  • a non-volatile computer-readable storage medium such as a memory 804 including computer program instructions, and the computer program instructions may be executed by the processor 820 of the electronic device 800 to complete the above method.
  • the present disclosure may be a system, method, and / or computer program product.
  • the computer program product may include a computer-readable storage medium having computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.
  • the computer-readable storage medium may be a tangible device that can hold and store instructions used by the instruction execution device.
  • the computer-readable storage medium may be, for example, but not limited to, an electric storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Non-exhaustive list of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) Or flash memory), static random access memory (SRAM), portable compact disc read only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical encoding device, such as a printer with instructions stored thereon A protruding structure in the hole card or groove, and any suitable combination of the above.
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • flash memory flash memory
  • SRAM static random access memory
  • CD-ROM compact disc read only memory
  • DVD digital versatile disc
  • memory stick floppy disk
  • mechanical encoding device such as a printer with instructions stored thereon A protruding structure in the hole card or groove, and any suitable combination of the above.
  • Computer-readable storage media used herein are not to be interpreted as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (for example, light pulses through fiber optic cables), or via electrical wires Electrical signal transmitted.
  • the computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network.
  • the network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers.
  • the network adapter card or network interface in each computing / processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device .
  • Computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or in one or more programming languages.
  • the programming languages include object-oriented programming languages—such as Smalltalk, C ++, and the like—and conventional procedural programming languages—such as the "C" language or similar programming languages.
  • Computer-readable program instructions may be executed entirely on a user's computer, partly on a user's computer, as a stand-alone software package, partly on a user's computer, partly on a remote computer, or entirely on a remote computer or server carried out.
  • the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through the Internet using an Internet service provider) connection).
  • LAN local area network
  • WAN wide area network
  • an electronic circuit such as a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA), can be personalized by using state information of computer-readable program instructions.
  • FPGA field-programmable gate array
  • PDA programmable logic array
  • the electronic circuit can Computer-readable program instructions are executed to implement various aspects of the present disclosure.
  • These computer-readable program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing device, thereby producing a machine such that when executed by a processor of a computer or other programmable data processing device , Means for implementing the functions / actions specified in one or more blocks in the flowcharts and / or block diagrams.
  • These computer-readable program instructions may also be stored in a computer-readable storage medium, and these instructions cause a computer, a programmable data processing apparatus, and / or other devices to work in a specific manner. Therefore, a computer-readable medium storing instructions includes: An article of manufacture that includes instructions to implement various aspects of the functions / acts specified in one or more blocks in the flowcharts and / or block diagrams.
  • Computer-readable program instructions can also be loaded onto a computer, other programmable data processing device, or other device, so that a series of operating steps can be performed on the computer, other programmable data processing device, or other device to produce a computer-implemented process , So that the instructions executed on the computer, other programmable data processing apparatus, or other equipment can implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.
  • each block in the flowchart or block diagram may represent a module, a program segment, or a part of an instruction, which contains one or more components for implementing a specified logical function.
  • Executable instructions may also occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved.
  • each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts can be implemented in a dedicated hardware-based system that performs the specified function or action. , Or it can be implemented with a combination of dedicated hardware and computer instructions. Without violating the logic, different embodiments of the present application may be combined with each other. The description of the different embodiments is emphasized. For the description of the emphasis, refer to the records of other embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Social Psychology (AREA)
  • Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Image Analysis (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本公开涉及一种动态动作检测方法、动态动作控制方法及装置，所述动态动作检测方法包括：将视频流中的帧图像加入第一队列，获得帧图像部分更新的所述第一队列；检测部分更新后的所述第一队列的帧图像中的动态动作；在所述动态动作与动作检测结果不匹配的情况下，根据所述动态动作更新所述动作检测结果，所述动作检测结果包括动作参考结果或在先检测的动作检测结果。本公开实施例可以根据部分更新的第一队列中的帧图像进行动态动作的检测，使得视频流中的动态动作能够被更加及时地检测出来、还可以迅速的根据动态动作的起始帧或结束帧确定动态动作，使得动态动作的检测结果更加准确，检测效率更高。

Description

动态动作检测方法、动态动作控制方法及装置
本申请要求在2018年8月24日提交中国专利局、申请号为201810974244.8、发明名称为“动态动作检测方法、动态动作控制方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本公开涉及图像处理技术领域，尤其涉及一种动态动作检测方法、动态动作控制方法及装置。
背景技术
随着人机交互需求的增多，通过在图像中检测操作者的动态动作，并利用检测出的动态动作对设备进行控制的应用需求越来越多。
发明内容
本公开提出了一种动态动作检测技术方案。
根据本公开的一方面，提供了一种动态动作检测方法，包括：将视频流中的帧图像加入第一队列，获得帧图像部分更新的所述第一队列；检测部分更新后的所述第一队列的帧图像中的动态动作；在所述动态动作与动作检测结果不匹配的情况下，根据所述动态动作更新所述动作检测结果，所述动作检测结果包括动作参考结果或在先检测的动作检测结果。
根据本公开的一方面，提供了一种动态动作控制方法，所述方法包括：获取视频流；采用上述任一项所述的方法确定所述视频流中的动作检测结果；控制设备执行与所述动作检测结果相应的操作。
根据本公开的一方面，提供了一种动态动作检测装置，所述装置包括：第一队列更新模块，用于将视频流中的帧图像加入第一队列，获得帧图像部分更新的所述第一队列；动态动作检测模块，用于检测部分更新后的所述第一队列的帧图像中的动态动作；动作检测结果更新模块，用于在所述动态动作与动作检测结果不匹配的情况下，根据所述动态动作更新所述动作检测结果，所述动作检测结果包括动作参考结果或在先检测的动作检测结果。
根据本公开的一方面，提供了一种动态动作控制装置，所述装置包括：视频流获取模块，用于获取视频流；动作检测结果获取模块，用于采用上述任一项所述的装置确定所述视频流中的动作检测结果；操作执行模块，用于控制设备执行与所述动作检测结果相应的操作。
根据本公开的一方面，提供了一种电子设备，包括：处理器；用于存储处理器可执行指令的存储器；其中，所述处理器被配置为：执行上述动态动作检测方法和/或动态动作控制方法。
根据本公开的一方面，提供了一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时执行上述动态动作检测方法和/或动态动作控制方法。
根据本公开的一方面，提供了一种计算机程序，所述计算机程序包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行上述动态动作检测方法和/或动态动作控制方法。
在本公开实施例中，可以将视频流中的帧图像加入第一队列，获得帧图像部分更新的所述第一队列，检测部分更新后的所述第一队列的帧图像中的动态动作；在所述动态动作与动作检测结果不匹配的情况下，根据所述动态动作更新所述动作检测结果。在本公开实施例中，不需要等到第一队列中的帧图像全部更新，便可以根据部分更新的第一队列中的帧图像进行动态动作的检测，使得视频流中的动态动作能够被更加及时地检测出来。当动态动作的起始帧或结束帧加入第一队列时，根据部分更新的第一队列中的帧图像进行动态动作检测，还可以迅速的根据动态动作的起始帧或结束帧确定动态动作，使得动态动作的检测结果更加准确，检测效率更高。本公开实施例可以实现对视频流中动态动作的实时追踪。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，而非限制本公开。根据下面参考附图对示例性实施例的详细说明，本公开的其它特征及方面将变得清楚。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。
图1示出根据本公开实施例的动态动作检测方法的流程图；
图2示出根据本公开实施例的动态动作检测方法的流程图；
图3示出根据本公开实施例的动态动作检测方法的流程图；
图4示出根据本公开实施例的动态动作检测方法的流程图；
图5示出根据本公开实施例的动态动作检测方法的流程图；
图6示出根据本公开实施例的动态动作检测方法的流程图；
图7示出根据本公开实施例的动态动作检测方法的流程图；
图8示出根据本公开实施例的动态动作检测方法的流程图；
图9示出根据本公开实施例的动态动作检测方法的流程图；
图10示出根据本公开实施例的动态动作检测方法的流程图；
图11示出根据本公开实施例的动态动作控制方法的流程图；
图12示出根据本公开实施例的动作识别方法应用示例的流程图；
图13示出根据本公开实施例的动态动作检测装置的框图；
图14示出根据本公开实施例的动态动作控制装置的框图；
图15是根据一示例性实施例示出的一种电子设备的框图。
具体实施方式
以下将参考附图详细说明本公开的各种示例性实施例、特征和方面。附图中相同的附图标记表示功能相同或相似的元件。尽管在附图中示出了实施例的各种方面，但是除非特别指出，不必按比例绘制附图。在这里专用的词“示例性”意为“用作例子、实施例或说明性”。这里作为“示例性”所说明的任何实施例不必解释为优于或好于其它实施例。本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中术语“至少一种”表示多种中的任意一种或多种中的至少两种的任意组合，例如，包括A、B、C中的至少一种，可以表示包括从A、B和C构成的集合中选择的任意一个或多个元素。另外，为了更好地说明本公开，在下文的具体实施方式中给出了众多的具体细节。本领域技术人员应当理解，没有某些具体细节，本公开同样可以实施。在一些实例中，对于本领域技术人员熟知的方法、手段、元件和电路未作详细描述，以便于凸显本公开的主旨。
图1示出根据本公开实施例的动态动作检测方法的流程图。所述动态动作检测方法可以由终端设备或服务器等电子设备执行，其中，终端设备可以为用户设备(User Equipment，UE)、移动设备、用户终端、终端、蜂窝电话、无绳电话、个人数字处理(Personal Digital Assistant，PDA)、手持设备、计算设备、车载设备、可穿戴设备等。在一些可能的实现方式中，所述动态动作检测方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。
如图1所示，所述动态动作检测方法包括：
步骤S10，将视频流中的帧图像加入第一队列，获得帧图像部分更新的所述第一队列。
在一种可能的实现方式中，在传统的动态动作检测方法中，需要在视频流中截取设定数量的帧图像用于动态动作的检测，检测完成后，需要再截取设定数量的帧图像用于下一次动态动作的检测。动态动作检测的实时性较差。
在一种可能的实现方式中，可以将用于动态动作检测的帧图像的最小数量，确定为第一队列的长度。例如，对于食指伸出向右滑动的动态动作，多个帧图像中的手部的静态动作均为食指伸出。当根据小于N个帧图像进行动态动作检测时，由于帧图像数量过少，静态动作在空间位置上的变化也过小，无法确定静态动作在空间位置上的变化，也就无法进行动态动作检测。当根据大于或等于N个的帧图像进行动态动作检测时，可以确定食指伸出的静态动作在空间位置上的变化为向右滑动，则N为可以用于动态动作检测的帧图像的最小数量。可以将N确定为第一队列的长度，其中，N为正整数。在一种可能的实现方式中，可以根据视频流中的内容、动态动作检测的精度需求、动态动作检测的速度需求或在先的经验值，确定第一队列的长度。本公开不限定第一队列的长度。例如，N的取值范围可以为：2≤N≤100、或10≤N≤200等，可以根据需求确定N的取值。
在一种可能的实现方式中，可以将视频流中的帧图像依次加入第一队列，部分更新第一队列。当有新的帧图像加入第一队列中时，可以将帧图像自第一队列的后端加入，并将第一队列最前端的帧图像移出，以使第一队列中的帧图像保持预设的数量。在一种可能的实现方式中，可以将视频流中的一个帧图像加入第一队列，获得帧图像部分更新的所述第一队列。也可以将视频流中的少于第一队列长度的多个帧图像加入第一队列，获得帧图像部分更新的所述第一队列。
步骤S20，检测部分更新后的所述第一队列的帧图像中的动态动作。
在一种可能的实现方式中，可以利用部分更新的第一队列中的帧图像进行检测，确定各帧图像中的动态动作。可 以将部分更新的第一队列中的帧图像输入神经网络，经神经网络检测第一队列中的帧图像中的动态动作。在一种可能的实现方式中，在第一队列未满的情况下，可以不对第一队列中的帧图像进行动态动作检测，以节省***资源。
在一种可能的实现方式中，动态动作的执行对象可以包括人、动物或机器人等各种可以活动的对象。动态动作可以包括执行对象的设定部位相对于其所处环境产生的相对位置的变化。例如，动态动作可以包括人利用食指的滑动做出的动作，可以包括人利用四肢做出的奔跑的动作，也可以包括机器人利用所配置的可运动装置做出的前后移动的动作。本公开不限定动态动作的执行对象，也不限定动态动作的类型。
步骤S30，在所述动态动作与动作检测结果不匹配的情况下，根据所述动态动作更新所述动作检测结果，所述动作检测结果包括动作参考结果或在先检测的动作检测结果。
在一种可能的实现方式中，一个视频流的帧图像中可以包括多个动态动作。可以将在先检测的动作检测结果，作为视频流的动作检测结果。例如，可以在视频流1的帧图像中识别出三个动作，依次为动作1、动作2和动作3。当识别出动作1时，可以将动作1确定为在先检测的动作检测结果，作为动作检测结果。当识别出动作2时，可以将动作检测结果更新为动作2；当识别出动作3时，可以将动作检测结果更新为动作3。在一种可能的实现方式中，所述动作检测结果也包括动作参考结果。可以将给定动作设定为动作参考结果，也可以将动作参考结果设置为空。
在一种可能的实现方式中，可以分析多个视频流的帧图像中的动态动作，并根据多个视频流的帧图像中的多个动态动作，更新动作检测结果。例如，可以依次对视频流1和视频流2进行动态动作检测。视频流1的帧图像中可以识别出动作1和动作2，视频流2的帧图像中可以识别出动作3和动作4。当根据视频流1的帧图像识别出动作1时，可以将动作1作为动作检测结果。当根据视频流1的帧图像识别出动作2时，可以将动作检测结果更新为动作2。再根据视频流2的帧图像进行动作识别，当识别出动作3时，可以将动作检测结果更新为动作3。当识别出动作4时，可以将动作检测结果更新为动作4。可以根据需求获取用于动态动作检测的一个或多个视频流。视频流的动作检测结果，可以是根据视频流自身的帧图像进行动态动作检测后获得的，也可以是根据其它视频流中的帧图像进行动态动作检测后获得的。
在一种可能的实现方式中，在所述动态动作与动作检测结果不匹配的情况下，可以表示根据视频流中的帧图像判断出了新的动作，可以以新动作更新动作检测结果，以使动作检测结果能够及时地更新。
在一种可能的实现方式中，本公开实施例中的动作检测结果可以用于实时追踪执行对象的动作。可以拍摄设备操作者的视频流，使用本公开实施例中的动态动作检测方法，获得所拍摄的视频流的动作检测结果，并根据动作检测结果对设备进行控制。动作检测结果的实时性，可以使得设备操作者能够精确地控制设备。
在本实施例中，可以将视频流中的帧图像加入第一队列，获得帧图像部分更新的所述第一队列，检测部分更新后的所述第一队列的帧图像中的动态动作；在所述动态动作与动作检测结果不匹配的情况下，根据所述动态动作更新所述动作检测结果。在本公开实施例中，不需要等到第一队列中的帧图像全部更新，便可以根据部分更新的第一队列中的帧图像进行动态动作的检测，使得视频流中的动态动作能够被更加及时地检测出来。当动态动作的起始帧或结束帧加入第一队列时，根据部分更新的第一队列中的帧图像进行动态动作检测，还可以迅速的根据动态动作的起始帧或结束帧确定动态动作，使得动态动作的检测结果更加准确，检测效率更高。本公开实施例可以实现对视频流中动态动作的实时追踪。
图2示出根据本公开实施例的动态动作检测方法的流程图，如图2所示，所述动态动作检测方法中，所述在先检测的动作检测结果的获取方法，包括：步骤S1，在所述第一队列的帧图像中有动态动作的情况下，将所述第一队列中的帧图像移至第二队列中。步骤S2，根据所述第二队列中的帧图像确定所述动作检测结果。
在一种可能的实现方式中，可以在初始状态下，获取在先检测的动作检测结果。在初始状态下，所述第一队列和所述第二队列可以为空，所述动作检测结果可以为无动作。在初始状态下，可以将视频流中的帧图像输入第一队列中，根据第一队列中的帧图像确定的动作检测结果，作为在先检测的动作检测结果。在一种可能的实现方式中，在初始状态下，可以自视频流中第一个帧图像开始进行动态动作检测。也可以对视频流中检测到的包括执行对象的帧图像开始进行动态动作检测。本公开对此不作限定。在一种可能的实现方式中，可以将视频流的帧图像按照时间的前后顺序，依次加入第一队列，直至第一队列可以进行动态动作检测。
在一种可能的实现方式中，第二队列的长度可以与第一队列的长度相等，也可以与第一队列的长度不等。在判断第一队列的帧图像中有动态动作的情况下，可以将第一队列中的帧图像全部移至第二队列中，第一队列被清空。可以根据第二队列中的帧图像中的动态动作确定动作检测结果。
在一种可能的实现方式中，在第一队列中的帧图像被移至第二队列中后，第一队列中的帧图像的数量为0，得到在 先检测的动作检测结果。初始状态结束后，可以利用如图1所示的实施例中的方法，依次根据后续的每一待识别的帧图像，部分更新第一队列，并根据检测到的部分更新后的第一队列中的帧图像中的动态动作，更新动作检测结果。
在本实施例中，在先检测的动作检测结果的获取方法，可以包括将视频流中的帧图像依次加入第一队列。在所述第一队列的帧图像中有动态动作的情况下，将所述第一队列中的帧图像移至第二队列中，并根据第二队列的帧图像中的动作得到动作检测结果。在先检测的动作检测结果为视频流中的动态动作，利用第二队列保留获得动作检测结果的帧图像，可以提高动作识别的实时性和准确性。
图3示出根据本公开实施例的动态动作检测方法的流程图，所述动作检测结果包括静态动作，如图3所示，所述动态动作检测方法中步骤S10，包括：步骤S11，获取所述视频流中待识别的帧图像中的静态动作。步骤S12，在所述待识别的帧图像中的静态动作与所述动作检测结果中的静态动作不匹配的情况下，将所述待识别的帧图像加入所述第一队列，获得帧图像部分更新的所述第一队列。
在一种可能的实现方式中，在得到在先检测的动作检测结果后，第一队列为空，第二队列中有帧图像。可以将视频流中没有进行过动态动作检测的帧图像作为待识别的帧图像。
在一种可能的实现方式中，动态动作可以包括静态动作和静态动作的动作轨迹。例如，动态动作1为“伸出的食指向右滑动”，动态动作1所包含的静态动作为食指伸出，静态动作的动作轨迹为向右滑动。在一种可能的实现方式中，动态动作可以包括静态动作和静态动作的切换信息。静态动作的切换信息可以包括至少两个静态动作间的切换信息。例如，动态动作2为“由拳变掌”，动态动作2包括静态动作1掌和静态动作2拳，动态动作2的切换信息为拳切换至掌。
在一种可能的实现方式中，动态动作可以包括静态动作、静态动作的切换信息和静态动作的动作轨迹。例如，动态动作3为“跳远动作”，动态动作3包括静态动作1跑步和静态动作2跳起，静态动作1的动作轨迹为向右，静态动作2的动作轨迹为向右，动态动作3的静态动作间的切换信息为由跑步切换至跳起。
在一种可能的实现方式中，可以在各待识别的帧图像中检测到静态动作。对应于一个动态动作的多个帧图像中的静态动作，与动态动作中的静态动作匹配。可以将至少两个帧图像中的静态动作进行比对，得到帧图像中的静态动作的动作轨迹和/或切换信息。对应于一个动态动作的多个帧图像中的静态动作的动作轨迹和/或切换信息，与动态动作中的动作轨迹和/或切换信息匹配。
在一种可能的实现方式中，可以依次检测视频流中每一个待识别的帧图像中的静态动作，将每一个待识别的帧图像中的静态动作和动作检测结果中的静态动作进行匹配。也可以将每一个待识别的帧图像中的静态动作，与第二队列中的帧图像中的静态动作进行匹配。例如，动作检测结果为伸出的食指向右滑动，可以检测待识别的帧图像中的食指的静态动作，是否与动作检测结果中食指的静态动作匹配。
在一种可能的实现方式中，如果待识别的帧图像中的静态动作与动作检测结果中的静态动作不匹配，可以将待识别的帧图像加入第一队列。可以将待识别的帧图像加入第一队列的后端，并将第一队列最前端的帧图像移出，部分更新第一队列。可以检测部分更新后的第一队列的帧图像中的动态动作，在检测到的动态动作与动作检测结果不匹配的情况下，根据动态动作更新动作检测结果。
在本实施例中，可以获取视频流中待识别的帧图像中的静态动作，在所述待识别的帧图像中的静态动作与所述动作检测结果中的静态动作不匹配的情况下，将所述待识别的帧图像加入所述第一队列。在帧图像中检测静态动作的过程简单可靠，可以将待识别的帧图像快速地加入第一队列中，快速地部分更新第一队列，提高动态动作的检测效率。
图4示出根据本公开实施例的动态动作检测方法的流程图，所述动作检测结果还包括所述静态动作的动作轨迹和/或切换信息，如图4所示，所述动态动作检测方法中步骤S10，包括：
步骤S13，在所述待识别的帧图像中的静态动作与所述动作检测结果中的静态动作匹配的情况下，根据所述待识别的帧图像和所述第二队列中的帧图像，确定所述待识别的帧图像中静态动作的动作轨迹和/或切换信息。
步骤S14，在所述待识别的帧图像中静态动作的动作轨迹和/或切换信息与所述动作检测结果不匹配的情况下，将所述待识别的帧图像加入所述第一队列，获得帧图像部分更新的所述第一队列。
在一种可能的实现方式中，动态动作还包括静态动作的动作轨迹和/或切换信息。在待识别的帧图像中的静态动作与动作检测结果中的静态动作匹配的情况下，可以根据待识别的帧图像和第二队列中的最后一个帧图像或多个帧图像，确定待识别的帧图像中静态动作的动作轨迹和/或切换信息。可以根据待识别的帧图像中的静态动作1和第二队列中的最后一个帧图像中的静态动作2之间的位置差，得到静态动作1相对于静态动作2的动作轨迹和/或切换信息。例如，动作检测结果为动态动作1“伸出的食指向右滑动”，待识别的帧图像中的静态动作为食指伸出，与动作检测结果匹配。 可以根据待识别的帧图像中的静态动作和第二队列中的帧图像中的静态动作的位置差，确定待识别的帧图像中的食指为向左滑动，则待识别的帧图像中的静态动作的动作轨迹与动作检测结果不匹配。
在一种可能的实现方式中，在待识别的帧图像中静态动作的动作轨迹和/或切换信息与动作检测结果不匹配的情况下，可以认为待识别的帧图像中的静态动作所属的动态动作与动作检测结果不匹配。可以将待识别的帧图像加入第一队列，部分更新第一队列。可以检测部分更新后的第一队列的帧图像中的动态动作，在检测到的动态动作与动作检测结果不匹配的情况下，根据动态动作更新动作检测结果。
在本实施例中，在待识别的帧图像中静态动作的动作轨迹和/或切换信息与动作检测结果不匹配的情况下，将待识别的帧图像加入所述第一队列，获得帧图像部分更新的第一队列。根据动作轨迹和/或切换信息，确定待识别的帧图像中的静态动作所属的动态动作与动作检测结果是否匹配，可以提高动作检测结果的准确率。
图5示出根据本公开实施例的动态动作检测方法的流程图，如图5所示，所述动态动作检测方法，还包括：
步骤S40，在根据部分更新后的所述第一队列的帧图像检测到的所述动态动作与动作检测结果不匹配的情况下，清空所述第二队列，将所述第一队列中的帧图像移至所述第二队列中。
步骤S50，根据所述第二队列中的帧图像中的动态动作，更新所述动作检测结果。
在一种可能的实现方式中，可以检测部分更新后的第一队列中的帧图像中的动态动作。可以检测部分更新后的第一队列中的帧图像中的静态动作和静态动作的动作轨迹和/或切换信息，并根据检测到的静态动作、静态动作的动作轨迹和/或切换信息确定动态动作，当部分更新后的第一队列中的帧图像中动态动作与动作检测结果不匹配时，可以认为部分更新后的第一队列的帧图像中出现了新的动态动作。
在一种可能的实现方式中，当部分更新后的第一队列的帧图像中出现了新的动态动作时，可以清空第二队列，将第一队列中的帧图像全部移至第二队列中，并根据更新后的第二队列中的帧图像中的动态动作，更新动作检测结果。
在一种可能的实现方式中，可以将拍摄得到的视频流进行实时的动态动作检测，并根据实时检测得到的动态动作进行电子设备的控制。可以检测视频流中的待识别的帧图像中的静态动作、静态动作的动作轨迹和/或切换信息，是否与动作检测结果匹配，当不匹配时，将待识别的帧图像加入第一队列部分更新第一队列。当检测部分更新的第一队列中的帧图像中有新的动态动作时，可以清空第二队列，将第一队列中的帧图像全部移至第二队列中，并根据更新后的第二队列中的帧图像中的动态动作更新动作检测结果。可以对视频流中的各帧图像进行实时的分析，提高动态动作检测的实时性和准确率。
图6示出根据本公开实施例的动态动作检测方法的流程图，如图6所示，所述动态动作检测方法，还包括：
步骤S15，在所述待识别的帧图像中的静态动作、静态动作的动作轨迹和/或切换信息与所述动作检测结果匹配的情况下，将所述待识别的帧图像加入所述第二队列。
在一种可能的实现方式中，在待识别的帧图像中的静态动作与动作检测结果中的静态动作匹配，待识别的帧图像中的静态动作的动作轨迹和/或切换信息，也与动作检测结果中的静态动作的动作轨迹和/或切换信息匹配的情况下，可以认为待识别的帧图像中的静态动作所属的动态动作与动作检测结果匹配。在一种可能的实现方式中，在待识别的帧图像中的静态动作、静态动作的动作轨迹和/或切换信息与动作检测结果匹配的情况下，可以将待识别的帧图像加入所述第二队列。可以将待识别的帧图像加入第二队列的后端，并将第二队列最前端的帧图像移出，以使第二队列中的帧图像保持预设的数量。动态动作具有连续性，对应于一个动态动作的多个连续的帧图像中，相隔较远的两个帧图像中的静态动作、静态动作的动作轨迹和/或切换信息可能差别较大。将与动作检测结果匹配的待识别的帧图像加入第二队列中，可以保持第二队列的帧图像中的动态动作的实时性，使得后续的帧图像的动态动作检测结果更加准确。
在一种可能的实现方式中，可以利用第一队列检测视频流中后续的待识别的帧图像中的动态动作。可以利用第二队列保存用于确定动作检测结果的帧图像。视频流中待识别的帧图像加入第一队列之前，可以与第二队列中的帧图像一起进行判断，确定待识别的帧图像是否具有与动作检测结果匹配的动态动作。如果待识别的帧图像中具有与动作检测结果匹配的动态动作，则将其加入第二队列中，使得第二队列的帧图像中的动态动作的连续性得到保证，提高动态动作检测结果的准确率。如果待识别的帧图像中不具有与动作检测结果匹配的动态动作，则将其加入第一队列中，使得第一队列的帧图像可以用于检测新的动态动作，提高动态动作检测的及时性。通过第一队列和第二队列的设置，本公开实施例可以实现视频流中的动态动作的准确、高效的实时追踪。
在本实施例中，在待识别的帧图像中的静态动作、静态动作的动作轨迹和/或切换信息与动作检测结果匹配的情况下，将待识别的帧图像加入所述第二队列。可以将与动作检测结果匹配的帧图像加入第二队列，保持第二队列的帧图 像中的动态动作的实时性，使得后续的帧图像的动态动作检测结果更加准确。
图7示出根据本公开实施例的动态动作检测方法的流程图，所述动态动作包括：至少一个静态动作和所述静态动作的动作轨迹，如图7所示，所述动态动作检测方法中步骤S20，包括：步骤S21，根据部分更新后的所述第一队列的各帧图像中的静态动作和各帧图像中的静态动作的动作轨迹，获得部分更新后的所述第一队列的帧图像中的动态动作。
在一种可能的实现方式中，动态动作可以包括静态动作和静态动作的动作轨迹。可以检测部分更新后的所述第一队列的各帧图像中的静态动作，在各帧图像中的静态动作匹配的情况下，计算各帧图像中的静态动作的动作轨迹，在各帧图像中的静态动作的动作轨迹匹配的情况下，可以根据静态动作和静态动作的动作轨迹，获得部分更新后的第一队列的帧图像中的动态动作。在本实施例中，可以根据部分更新后的第一队列的各帧图像中的静态动作和各帧图像中的静态动作的动作轨迹，获得部分更新后的帧图像中的动态动作。根据静态动作和静态动作的动作轨迹检测动态动作，可以提高动态动作的检测准确率。
图8示出根据本公开实施例的动态动作检测方法的流程图，所述动态动作包括：至少两个静态动作和静态动作间的切换信息，如图8所示，所述动态动作检测方法中步骤S20，包括：步骤S22，根据部分更新后的所述第一队列的各帧图像中的静态动作和静态动作的切换信息，获得部分更新后的所述第一队列的帧图像中的动态动作。
在一种可能的实现方式中，动态动作可以包括静态动作和静态动作的切换信息。可以检测部分更新后的所述第一队列的各帧图像中的静态动作，在各帧图像中的静态动作匹配的情况下，计算各帧图像中的静态动作的切换信息，在各帧图像中的静态动作的切换信息匹配的情况下，可以根据静态动作和静态动作的切换信息，获得部分更新后的第一队列的帧图像中的动态动作。在本实施例中，可以根据部分更新后的第一队列的各帧图像中的静态动作和各帧图像中的静态动作的切换信息，获得部分更新后的帧图像中的动态动作。根据静态动作和静态动作的切换信息检测动态动作，可以提高动态动作的检测准确率。
图9示出根据本公开实施例的动态动作检测方法的流程图，所述动态动作包括：至少两个静态动作、各静态动作的动作轨迹和静态动作间的切换信息，如图9所示，所述动态动作检测方法中步骤S20，包括：步骤S23，根据部分更新后的所述第一队列的各帧图像中的静态动作、各静态动作的动作轨迹和静态动作间的切换信息，获得部分更新后的所述第一队列的帧图像中的动态动作。
在一种可能的实现方式中，动态动作可以包括静态动作、各静态动作的动作轨迹和静态动作的切换信息。可以检测部分更新后的所述第一队列的各帧图像中的静态动作，在各帧图像中的静态动作匹配的情况下，计算各帧图像中各静态动作的动作轨迹和静态动作间的切换信息，在各帧图像中各静态动作的动作轨迹和静态动作间的切换信息也匹配的情况下，可以根据静态动作、各静态动作的动作轨迹和静态动作的切换信息，获得部分更新后的第一队列的帧图像中的动态动作。在本实施例中，根据部分更新后的所述第一队列的各帧图像中的静态动作、各静态动作的动作轨迹和静态动作间的切换信息，获得部分更新后的第一队列的帧图像中的动态动作。根据静态动作、各静态动作的动作轨迹和静态动作间的切换信息检测动态动作，可以提高动态动作的检测准确率。
图10示出根据本公开实施例的动态动作检测方法的流程图，所述动态动作包括手部动态动作，所述静态动作包括手势，如图10所示，所述动态动作检测方法中步骤S11，包括：步骤S111，检测所述待识别的帧图像中是否包括手部。步骤S112，在所述待识别图像中包括手部的情况下，根据所述手部上的手指和/或手掌获得所述待识别图像中的手势。
在一种可能的实现方式中，动态动作可以包括手部动态动作、肢体动态动作等各种类型的动态动作。手部动态动作的执行者可以包括人、动物、机器人等。手部动态动作可以包括人的手部的动态动作或机器人的手部的动态动作。本公开对此不做限定。可以首先检测待识别的帧图像中是否包括手部。可以将待识别的帧图像输入神经网络，经神经网络对待识别的帧图像进行卷积处理等操作，检测待识别的帧图像中是否包括手部。当检测结果为不包括手部时，可以不对待识别的帧图像进行后续的分析，以节省***资源。
在一种可能的实现方式中，在检测结果为待识别的帧图像中包括手部时，可以进一步确定手势。可以根据手部上的手指获得手势。例如，可以根据五个手指都伸出，确定手势为“五”。也可以根据手部的手掌获得手势，例如，可以根据掌心向上或掌心向下，确定手势为“向上”或“向下”。还可以根据手部的手指和手掌获得手势。例如，可以根据五指伸出，掌心向下，确定手势为“向下按”。可以根据需求，确定根据手指和/或手掌获得手势。本公开对此不做限定。在本实施例中，可以检测待识别的帧图像中是否包括手部，可以在待识别图像中包括手部的情况下，根据手部上的手指和/或手掌获得所述待识别图像中的手势。通过检测待识别的帧图像中是否包括手部，可以提高动态动作的检测效率。
在一种可能的实现方式中，步骤S112，包括：根据所述手部上的手指的第一关键点的位置和/或手掌的第二关键点的位置识别所述待识别图像中的手势，所述手指的第一关键点包括指尖或指关节，所述手掌的第二关键点包括掌心、手掌背部或手掌边缘上的点。
在一种可能的实现方式中，传统的手势识别方法中，利用手势模板进行手势识别时，需要将手势事先输入模板，导致手势识别的效率低，并且对模板的依赖性大。在一种可能的实现方式中，可以在手指上确定第一关键点。一个手指上可以确定一个或多个第一关键点。例如，可以将指关节、指尖位置的点确定为第一关键点。可以在手掌上确定第二关键点。手掌上可以包括一个或多个关键点。例如，可以将掌心、手掌背部或手掌边缘位置上的点确定为第二关键点。本公开实施例可以根据手部的关键点的位置，灵活准确地识别出不同的手势，免去对手势识别模板的依赖。
在一种可能的实现方式中，可以根据手指的第一关键点的位置确定手势，也可以根据手掌的第二关键点的位置确定手势，还可以将手指的第一关键点的位置和手掌的第二关键点的位置进行组合后确定手势。在本实施例中，可以根据手指的第一关键点的位置和/或手掌的第二关键点的位置，确定待识别的帧图像中的手势。根据手部的关键点确定出的手势，精确可靠。
在一种可能的实现方式中，所述手部动态动作还包括手势的动作轨迹和/或切换信息，所述动态动作检测方法中步骤S13，包括：根据所述待识别的帧图像中的第一关键点和与之对应的所述第二队列的帧图像中的第一关键点，确定所述待识别的帧图像中的动作轨迹和/或切换信息，和/或，根据所述待识别的帧图像中的第二关键点和与之对应的所述第二队列的帧图像中的第二关键点，确定所述待识别的帧图像中的动作轨迹和/或切换信息。
在一种可能的实现方式中，在根据手指可以确定手势的情况下，可以根据待识别的帧图像中的第一关键点，和与之对应的第二队列的帧图像中的第一关键点之间的位置差，确定所述待识别的帧图像中的动作轨迹和/或切换信息。
在一种可能的实现方式中，在根据手掌可以确定手势的情况下，可以根据待识别的帧图像中的第二关键点，和与之对应的第二队列的帧图像中的第二关键点之间的位置差，确定所述待识别的帧图像中的动作轨迹和/或切换信息。
在一种可能的实现方式中，在根据手指和手掌可以确定手势的情况下，可以根据待识别的帧图像中的第一关键点，和与之对应的第二队列的帧图像中的第一关键点之间的位置差，以及可以根据待识别的帧图像中的第二关键点，和与之对应的第二队列的帧图像中的第二关键点之间的位置差，共同确定所述待识别的帧图像中的动作轨迹和/或切换信息。在本实施例中，根据所述待识别的帧图像中的第一关键点和与之对应的所述第二队列的帧图像中的第一关键点，和/或根据所述待识别的帧图像中的第二关键点和与之对应的所述第二队列的帧图像中的第二关键点，确定所述待识别的帧图像中的动作轨迹和/或切换信息。可以根据第一关键点和/或第二关键点，在待识别的帧图像中得到准确的动作轨迹和/或切换信息，使得动态动作的检测结果精准可靠。
在一种可能的实现方式中，所述动态动作包括静态动作和静态动作的动作轨迹，步骤S20，包括：检测所述第一队列的帧图像中的静态动作；计算目标帧图像中的静态动作相对于所述第一队列的第一个帧图像中的静态动作在X轴方向和Y轴方向上的累积移动距离，所述目标帧图像包括所述第一队列中除所述第一个帧图像外的任一帧图像；根据所述目标帧图像中静态动作和所述累积移动距离获得所述第一队列的帧图像中的动态动作。
在一种可能的实现方式中，可以为帧图像建立图像坐标系，确定各帧图像中的静态动作在图像坐标系中的坐标位置，得到静态动作在X轴和Y轴上的坐标值。例如，当静态动作为手势时，可以根据手部的第一关键点在X轴和Y轴上的坐标值、和/或根据手部的第二关键点在X轴和Y轴上的坐标值，得到手势在X轴和Y轴上的坐标值。在一种可能的实现方式中，可以计算各帧图像中的静态动作，相对于所述第一队列的第一个帧图像中的静态动作在X轴方向和Y轴方向上的累积移动距离。累积移动距离可以包括由X轴的累积移动距离，和Y轴的累积移动距离组成的数组。根据累积移动距离，可以确定出各帧图像中的静态动作相对于第一个帧图像中的静态动作的移动方向和所移动的距离。在本实施例中，可以根据第一队列中的各帧图像中的静态动作和静态动作的累积移动距离，确定第一队列中的动态动作。可以根据各帧图像在图像坐标系中的位置变化，确定各帧图像中静态动作的累积移动距离。根据图像坐标系，可以精确地得到各帧图像中的静态动作的累积移动距离。使得动作识别的结果精准、可靠，降低动作识别的错误率。
在一种可能的实现方式中，所述根据所述目标帧图像中静态动作和所述累积移动距离获得所述第一队列的帧图像中的动态动作，包括：在所述目标帧图像中静态动作的累积移动距离小于移动阈值的情况下，确定所述第一队列的帧图像中目标对象的动态动作为间断动作。
在一种可能的实现方式中，动态动作具有连续性，当静态动作在空间位置上的移动达到一定距离时，可以确定产生了动态动作。当静态动作在空间位置上的移动距离过小时，可以认为执行者停止了执行动态动作。可以根据需求确 定移动阈值，在帧图像中的静态动作的累积移动距离小于移动阈值的情况下，可以认为帧图像中的动态动作产生了间断。在本实施例中，可以利用移动阈值，精确确定第一队列帧图像中的动作是否为间断动作，使得动作识别的结果精准、可靠。
在一种可能的实现方式中，所述根据所述目标帧图像中静态动作和所述累积移动距离获得所述第一队列的帧图像中的动态动作，包括：在所述累积移动距离大于等于所述移动阈值的情况下，计算各所述帧图像中静态动作的累积移动距离相对于所述静态动作所属的目标对象的宽度之间的比值；在所述比值小于比值阈值的情况下，确定目标帧图像中静态动作的动作轨迹为旋转动作轨迹；根据所述目标帧图像中的静态动作和所述旋转动作轨迹，获得所述第一队列的帧图像中的动态动作。
在一种可能的实现方式中，动态动作可以包括旋转动作。例如可以以手部动态动作为例。当动态动作为手部动态动作时，可以计算各所述帧图像中的手势的累积移动距离相对于手部的宽度之间的比值。可以将手势的累积移动距离，在图像坐标系中利用矢量来表示。可以计算所述矢量的大小和手部的宽度之间的比值。在所述比值小于比值阈值的情况下，可以认为各帧图像中的手势的动作轨迹为旋转动作轨迹。
在本实施例中，可以计算各帧图像中的静态动作的累积移动距离相对于所述静态动作所属的目标对象的宽度之间的比值；在比值小于比值阈值的情况下，确定目标帧图像中的静态动作的动作轨迹为旋转动作轨迹。根据累积移动距离和比值阈值，可以使得动态动作的检测结果更加精准。
在一种可能的实现方式中，所述方法还包括：在所述比值大于等于所述比值阈值的情况下，确定目标帧图像中静态动作的动作轨迹为平移动作轨迹；根据所述目标帧图像中的静态动作和所述平移动作轨迹，获得所述第一队列的帧图像中的动态动作。
在一种可能的实现方式中，静态动作的动作轨迹可以包括平移动作轨迹。在上述比值大于等于比值阈值的情况下，可以认为各帧图像中的手势的动作轨迹为平移动作轨迹。在本实施例中根据累积移动距离和比值阈值，可以判断静态动作的动作轨迹为平移动作轨迹，使得动态动作的检测结果更加精准。
图11示出根据本公开实施例的动态动作控制方法的流程图。所述动态动作控制方法可以由终端设备或服务器等电子设备执行，其中，终端设备可以为用户设备(User Equipment，UE)、移动设备、用户终端、终端、蜂窝电话、无绳电话、个人数字处理(Personal Digital Assistant，PDA)、手持设备、计算设备、车载设备、可穿戴设备等。在一些可能的实现方式中，所述动态动作控制方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。
如图11所示，所述动态动作控制方法，包括：步骤S100，获取视频流。步骤S110，采用上述任一项动态动作检测方法确定所述视频流中的动作检测结果。步骤S120，控制设备执行与所述动作检测结果相应的操作。
在一种可能的实现方式中，可以在需要进行控制操作的设备上设置拍摄装置，可以利用拍摄装置实时获取视频流。也可以获取拍摄装置拍摄完成的视频流。可以利用不同的拍摄装置拍摄视频流。拍摄装置可以包括双目摄像头、深度摄像头或普通摄像头。可以根据动态动作检测的需求，选用不同类型的摄像头拍摄视频流。
在一种可能的实现方式中，步骤S120，包括：根据预先确定的动作检测结果与操作指令之间的对应关系，获取与所述动作检测结果对应的操作指令；根据所述操作指令控制设备执行相应操作。
在一种可能的实现方式中，可以预设动作检测结果和操作指令之间的对应关系。在对应关系中，可以是一个动作检测结果对应一个操作指令，也可以是多个动作检测结果对应一个操作指令。可以根据***作的设备的类型和操作需求，确定操作指令的类型和内容。本公开不限定操作指令的形式和具体内容。
在一种可能的实现方式中，可以输出操作指令，对***作的设备进行控制。由于动作检测结果可以实时追踪视频流中的动作，输出的操作指令也可以实时追踪视频流中执行对象的动作，使得操作者可以精准的控制***作的设备。
在本实施例中，可以根据对应关系和动作检测结果，确定与动作检测结果对应的操作指令并输出。基于动作检测结果的实时性和准确性，操作指令也能够实时追踪视频流中执行对象的动作，使得操作者能够更加精确的操作设备。
在一种可能的实现方式中，所述根据所述操作指令控制设备执行相应操作，包括：根据所述操作指令对车辆的车窗、车门、车载***或车载设备进行控制。
在一种可能的实现方式中，可以在车辆中设置监控设备，拍摄车辆中的驾驶者或乘坐者的监控视频作为视频流。可以将拍摄到的视频流进行实时地动态动作检测。根据预先确定的动作检测结果与操作指令之间的对应关系，可获取与所述动作检测结果对应的操作指令；然后可根据操作指令对车辆的车窗、车门、车载***或车载设备(例如车载多媒体设备、车载机器人等)进行控制。
在一种可能的实现方式中，控制设备执行与所述动作检测结果相应的操作，包括：
响应于所述检测结果为预定义动态动作，控制车辆执行与所述预定义动态动作对应的操作。
在一种可能的实现方式中，所述预定义动态动作包括动态手势，所述动态手势包括以下至少之一：单指顺/逆时针旋转、手掌左/右挥动、两指前戳、拇指和小指伸出、手掌朝下向下压、拇指伸出左/右移动、手掌左/右长滑、掌心向上拳变掌、掌心向上掌变拳、掌心向下掌变拳、掌心向下掌变拳、单指滑动、多指向内捏住、单指双击、单指单击、多指双击、多指单击；与所述预定义动态动作对应的操作包括以下至少之一：调节音量大/小，歌曲切换，歌曲暂停/继续，接听或启动电话、挂断或拒接电话、空调温度升高或降低、多屏互动、打开天窗、关闭天窗、锁紧门锁、解锁门锁、拖动导航、缩小地图、放大地图。
在一种可能的实现方式中，可以利用单指顺/逆时针旋转的动态手势，对车辆的音响设备进行调节音量大/小的操作。可以利用手掌左/右挥动的动态手势，对车辆的音响设备进行歌曲切换的操作。可以利用两指前戳的动态手势，对车辆的音响设备进行歌曲暂停/继续的操作。可以利用拇指和小指伸出的动态手势，对车辆的通信设备进行接听或启动电话的操作。可以利用手掌朝下向下压的动态手势，对车辆的通信设备进行挂断或拒接电话的操作。可以利用拇指伸出左/右移动的动态手势，对车辆的空调设备进行空调温度升高或降低的操作。可以利用手掌左/右长滑的动态手势，对车辆的显示屏幕进行多屏互动的操作。可以利用掌心向上拳变掌的动态手势，对车辆进行打开天窗的操作(可以每次打开设定的长度，例如每次打开10厘米)。可以利用掌心向上掌变拳的动态手势，对车辆进行关闭天窗的操作。可以利用掌心向下掌变拳的动态手势，对车辆进行锁紧门锁的操作。可以利用掌心向下拳变掌的动态手势，对车辆进行解锁门锁的操作。可以利用单指滑动的动态手势，对车辆的导航设备进行拖动导航的操作。可以利用多指向内捏住的动态手势，对车辆的导航设备进行缩小地图的操作。可以利用单指双击的动态手势，对车辆的导航设备进行放大地图的操作。在本实施例中，可以利用操作指令对车辆的车窗、车门、车载***或车载设备进行控制。可以利用在视频流中识别出的动态动作，对车辆自身或车辆上的车载***进行不同的操作。基于本公开实施例中的动态动作检测方法，可以使得操作者对车辆自身或车载***进行精准地控制。
应用示例：图12示出根据本公开实施例的动作识别方法应用示例的流程图，如图12所示：
可以在车辆中配置深度摄像头，利用深度摄像头获取驾驶员的监控图像作为视频流。可以将拍摄到的监控图像进行实时的动作识别。在本实施实例中，对驾驶员的手部动作进行动作识别。
一、获取在先检测的动作检测结果：步骤S210，将第一队列和第二队列设置为空，动作检测结果也设置为空。将视频流中的帧图像按照时间段前后顺序依次加入第一队列。步骤S220，检测第一队列的帧图像中是否有动态动作。如无动态动作，接步骤S230，如有动态动作，接步骤S240。
在一种可能的实现方式中，第一队列的长度可以为十帧帧图像。在视频流中的第一帧至第十帧帧图像加入第一队列后，可以判断第一队列中的十帧帧图像中是否有动态动作。例如，动态动作为手部动态动作。具体的，可以依次识别帧图像中是否有手部，当第一队列的帧图像中有手部的情况下，可以根据手部上的手指和/或手掌识别待识别图像中的手势。当各帧图像中的手势匹配时，可以确定各帧图像中的手势的动作轨迹和/或切换信息。在各帧图像中的手势的动作轨迹和/或切换信息也匹配的情况下，可以检测第一队列中的动态动作。
步骤S230，继续将视频流中的各帧图像按时间前后顺序依次加入第一队列中，跳转至步骤S220。在一种可能的实现方式中，可以将第十一帧帧图像加入第一队列的后端，并将第一队列中最前端的第一帧帧图像移出第一队列。此时第一队列中包括第二帧至第十一帧帧图像，跳转至步骤S220后可以判断此时的第一队列中是否有动作。如无动作，可继续将第十二帧帧图像加入第一队列，并移出第二帧帧图像，直至根据步骤S220判断第一队列的帧图像中有动作。
步骤S240，将第一队列中的帧图像移至第二队列中。第一队列被清空，根据第二队列中的帧图像确定动作检测结果，得到在先检测的动作检测结果。在一种可能的实现方式中，得到在先检测的动作检测结果后，第一队列为空，第二队列中有十帧帧图像，为视频流中的第二十三帧至第三十二帧帧图像。可以根据第二队列的帧图像中的动作确定动作检测结果。视频流中未进行动作识别的帧图像，可以作为待识别的帧图像进行后续分析。即自第三十三帧帧图像开始的帧图像，可以作为待识别的帧图像，进入后续动态动作检测的步骤S250。
二、动态动作检测步骤：步骤S250，依次判断视频流中每一待识别的帧图像与动作检测结果是否匹配。如果匹配，接步骤S260，如果不匹配，接步骤S270。
在一种可能的实现方式中，可以根据第三十三帧帧图像与第二队列中的最后端的帧图像(第三十二帧帧图像)，确定第三十三帧帧图像与动作检测结果是否匹配。可以先确定第三十三帧帧图像中的手势，与第二队列中的第三十二帧 帧图像的手势是否一致。在手势一致的情况下，确定第三十三帧帧图像中的手势的动作轨迹和/或切换信息，与动作检测结果中手势的动作轨迹和/或切换信息是否匹配。在手势的动作轨迹和/或切换信息也匹配的情况下，可以确定第三十三帧帧图像与动作检测结果匹配。
步骤S260，如果匹配，将所述待识别的帧图像加入第二队列中。在一种可能的实现方式中，将第三十三帧帧图像加入第二队列中后，第二队列中的帧图像更新为第二十四帧至第三十三帧。
步骤S270，如果不匹配，将所述待识别的帧图像加入第一队列中。
步骤S280，确定所述第一队列的帧图像中的动态动作是否与动作检测结果匹配。如果不匹配，接步骤S290。
步骤S290，在第一队列的帧图像中的动态动作与动作检测结果不匹配的情况下，清空第二队列，将第一队列中的帧图像移至第二队列中。并根据更新后的第二队列的帧图像中的动作更新所述动作检测结果。
可以理解，本公开提及的上述各个方法实施例，在不违背原理逻辑的情况下，均可以彼此相互结合形成结合后的实施例，限于篇幅，本公开不再赘述。本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的撰写顺序并不意味着严格的执行顺序而对实施过程构成任何限定，各步骤的具体执行顺序应当以其功能和可能的内在逻辑确定。
此外，本公开还提供了动态动作检测装置、动态动作控制装置、电子设备、计算机可读存储介质、程序，上述均可用来实现本公开提供的任一种动态动作检测方法、动态动作控制方法，相应技术方案和描述和参见方法部分的相应记载，不再赘述。
图13示出根据本公开实施例的动态动作检测装置的框图，如图13所示，所述动态动作检测装置，包括：第一队列更新模块10，用于将视频流中的帧图像加入第一队列，获得帧图像部分更新的所述第一队列；动态动作检测模块20，用于检测部分更新后的所述第一队列的帧图像中的动态动作；动作检测结果更新模块30，用于在所述动态动作与动作检测结果不匹配的情况下，根据所述动态动作更新所述动作检测结果，所述动作检测结果包括动作参考结果或在先检测的动作检测结果。
在一种可能的实现方式中，所述装置还包括：在先检测的动作检测结果获取模块，用于在所述第一队列的帧图像中有动态动作的情况下，将所述第一队列中的帧图像移至第二队列中；根据所述第二队列中的帧图像确定所述动作检测结果。
在一种可能的实现方式中，所述动作检测结果包括静态动作，所述第一队列更新模块10，包括：第一获取子模块，用于获取所述视频流中待识别的帧图像中的静态动作；第一更新子模块，用于在所述待识别的帧图像中的静态动作与所述动作检测结果中的静态动作不匹配的情况下，将所述待识别的帧图像加入所述第一队列，获得帧图像部分更新的所述第一队列。
在一种可能的实现方式中，所述动作检测结果还包括所述静态动作的动作轨迹和/或切换信息，所述第一队列更新模块10，包括：第二获取子模块，用于在所述待识别的帧图像中的静态动作与所述动作检测结果中的静态动作匹配的情况下，根据所述待识别的帧图像和所述第二队列中的帧图像，确定所述待识别的帧图像中静态动作的动作轨迹和/或切换信息；第二更新子模块，用于在所述待识别的帧图像中静态动作的动作轨迹和/或切换信息与所述动作检测结果不匹配的情况下，将所述待识别的帧图像加入所述第一队列，获得帧图像部分更新的所述第一队列。
在一种可能的实现方式中，所述装置还包括：队列转移模块，用于在根据部分更新后的所述第一队列的帧图像检测到的所述动态动作与动作检测结果不匹配的情况下，清空所述第二队列，将所述第一队列中的帧图像移至所述第二队列中；动作检测结果更新模块，用于根据所述第二队列中的帧图像中的动态动作，更新所述动作检测结果。
在一种可能的实现方式中，可以将拍摄得到的视频流进行实时的动态动作检测，并根据实时检测得到的动态动作进行电子设备的控制。可以检测视频流中的待识别的帧图像中的静态动作、静态动作的动作轨迹和/或切换信息，是否与动作检测结果匹配，当不匹配时，将待识别的帧图像加入第一队列部分更新第一队列。当检测部分更新的第一队列中的帧图像中有新的动态动作时，可以清空第二队列，将第一队列中的帧图像全部移至第二队列中，并根据更新后的第二队列中的帧图像中的动态动作更新动作检测结果。可以对视频流中的各帧图像进行实时的分析，提高动态动作检测的实时性和准确率。
在一种可能的实现方式中，所述装置还包括：第二队列更新模块，用于在所述待识别的帧图像中的静态动作、静态动作的动作轨迹和/或切换信息与所述动作检测结果匹配的情况下，将所述待识别的帧图像加入所述第二队列。
在一种可能的实现方式中，所述动态动作包括：至少一个静态动作和所述静态动作的动作轨迹，所述动态动作检测模块20，包括：第一动态动作检测子模块，用于根据部分更新后的所述第一队列的各帧图像中的静态动作和各帧图 像中的静态动作的动作轨迹，获得部分更新后的所述第一队列的帧图像中的动态动作。
在一种可能的实现方式中，所述动态动作包括：至少两个静态动作和静态动作间的切换信息，所述动态动作检测模块20，包括：第二动态动作检测子模块，用于根据部分更新后的所述第一队列的各帧图像中的静态动作和静态动作的切换信息，获得部分更新后的所述第一队列的帧图像中的动态动作。
在一种可能的实现方式中，所述动态动作包括：至少两个静态动作、各静态动作的动作轨迹和静态动作间的切换信息，所述动态动作检测模块20，包括：第三动态动作检测子模块，用于根据部分更新后的所述第一队列的各帧图像中的静态动作、各静态动作的动作轨迹和静态动作间的切换信息，获得部分更新后的所述第一队列的帧图像中的动态动作。在一种可能的实现方式中，所述动态动作包括手部动态动作，所述静态动作包括手势，所述第一获取子模块，包括：手部检测单元，用于检测所述待识别的帧图像中是否包括手部；手势确定单元，用于在所述待识别图像中包括手部的情况下，根据所述手部上的手指和/或手掌获得所述待识别图像中的手势。
在一种可能的实现方式中，所述手势确定单元，用于：根据所述手部上的手指的第一关键点的位置和/或手掌的第二关键点的位置识别所述待识别图像中的手势，所述手指的第一关键点包括指尖或指关节，所述手掌的第二关键点包括掌心、手掌背部或手掌边缘上的点。
在一种可能的实现方式中，所述手部动态动作还包括手势的动作轨迹和/或切换信息，所述第二获取子模块，包括：
第一获取单元，用于根据所述待识别的帧图像中的第一关键点和与之对应的所述第二队列的帧图像中的第一关键点，确定所述待识别的帧图像中的动作轨迹和/或切换信息，和/或
第二获取单元，用于根据所述待识别的帧图像中的第二关键点和与之对应的所述第二队列的帧图像中的第二关键点，确定所述待识别的帧图像中的动作轨迹和/或切换信息。
在一种可能的实现方式中，所述动态动作包括静态动作和静态动作的动作轨迹，所述动态动作检测模块20，包括：静态动作检测子模块，用于检测所述第一队列的帧图像中的静态动作；累积移动距离确定子模块，用于计算目标帧图像中的静态动作相对于所述第一队列的第一个帧图像中的静态动作在X轴方向和Y轴方向上的累积移动距离，所述目标帧图像包括所述第一队列中除所述第一个帧图像外的任一帧图像；第四动态动作检测子模块，用于根据所述目标帧图像中静态动作和所述累积移动距离获得所述第一队列的帧图像中的动态动作。
在一种可能的实现方式中，所述第四动态动作检测子模块，用于：在所述目标帧图像中静态动作的累积移动距离小于移动阈值的情况下，确定所述第一队列的帧图像中目标对象的动态动作为间断动作。
在一种可能的实现方式中，所述第四动态动作检测子模块，用于：在所述累积移动距离大于等于所述移动阈值的情况下，计算各所述帧图像中静态动作的累积移动距离相对于所述静态动作所属的目标对象的宽度之间的比值；在所述比值小于比值阈值的情况下，确定目标帧图像中静态动作的动作轨迹为旋转动作轨迹；根据所述目标帧图像中的静态动作和所述旋转动作轨迹，获得所述第一队列的帧图像中的动态动作。
在一种可能的实现方式中，所述第四动态动作检测子模块，还用于：在所述比值大于等于所述比值阈值的情况下，确定目标帧图像中静态动作的动作轨迹为平移动作轨迹；根据所述目标帧图像中的静态动作和所述平移动作轨迹，获得所述第一队列的帧图像中的动态动作。
图14示出根据本公开实施例的动态动作控制装置的框图，如图14所示，所述动态动作控制装置包括：视频流获取模块100，用于获取视频流；动作检测结果获取模块200，用于采用上述的动态动作检测装置确定所述视频流中的动作检测结果；操作执行模块300，用于控制设备执行与所述动作检测结果相应的操作。
在一种可能的实现方式中，可以在需要进行控制操作的设备上设置拍摄装置，可以利用拍摄装置实时获取视频流。也可以获取拍摄装置拍摄完成的视频流。可以利用不同的拍摄装置拍摄视频流。拍摄装置可以包括双目摄像头、深度摄像头或普通摄像头。可以根据动态动作检测的需求，选用不同类型的摄像头拍摄视频流。
在一种可能的实现方式中，操作执行模块300，包括：操作指令获取子模块，用于根据预先确定的动作检测结果与操作指令之间的对应关系，获取与所述动作检测结果对应的操作指令；操作执行子模块，用于根据所述操作指令控制设备执行相应操作。
在一种可能的实现方式中，所述操作执行子模块，用于：根据所述操作指令对车辆的车窗、车门、车载***或设车载设备进行控制。
在一种可能的实现方式中，可以在车辆中设置监控设备，拍摄车辆中的驾驶者或乘坐者的监控视频作为视频流。可以将拍摄到的视频流进行实时地动态动作检测。根据预先确定的动作检测结果与操作指令之间的对应关系，可获取 与所述动作检测结果对应的操作指令；然后可根据操作指令对车辆的车窗、车门、车载***或车载设备(例如车载多媒体设备、车载机器人等)进行控制。
在一种可能的实现方式中，所述操作执行模块300，还用于：响应于所述检测结果为预定义动态动作，控制车辆执行与所述预定义动态动作对应的操作。
在一种可能的实现方式中，所述预定义动态动作包括动态手势，所述动态手势包括以下至少之一：单指顺/逆时针旋转、手掌左/右挥动、两指前戳、拇指和小指伸出、手掌朝下向下压、拇指伸出左/右移动、手掌左/右长滑、掌心向上拳变掌、掌心向上掌变拳、掌心向下掌变拳、掌心向下拳变掌、单指滑动、多指向内捏住、单指双击、单指单击、多指双击、多指单击；与所述预定义动态动作对应的操作包括以下至少之一：调节音量大/小，歌曲切换，歌曲暂停/继续，接听或启动电话、挂断或拒接电话、空调温度升高或降低、多屏互动、打开天窗、关闭天窗、锁紧门锁、解锁门锁、拖动导航、缩小地图、放大地图。
在一些实施例中，本公开实施例提供的装置具有的功能或包含的模块可以用于执行上文方法实施例描述的方法，其具体实现可以参照上文方法实施例的描述，装置技术效果可参见相应方法实施例的记载，为了简洁，这里不再赘述。
本公开实施例还提出一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现上述方法。计算机可读存储介质可以是非易失性计算机可读存储介质或易失性计算机可读存储介质。
本公开实施例还提出一种电子设备，包括：处理器；用于存储处理器可执行指令的存储器；其中，所述处理器被配置为执行上述任一方法。电子设备可以被提供为终端、服务器或其它形态的设备。
本公开实施例还提出一种计算机程序，所述计算机程序包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行本公开任一方法实施例。
图15是根据一示例性实施例示出的一种电子设备800的框图。例如，电子设备800可以是移动电话，计算机，数字广播终端，消息收发设备，游戏控制台，平板设备，医疗设备，健身设备，个人数字助理等终端。
参照图15，电子设备800可以包括以下一个或多个组件：处理组件802，存储器804，电源组件806，多媒体组件808，音频组件810，输入/输出(I/O)的接口812，传感器组件814，以及通信组件816。
处理组件802通常控制电子设备800的整体操作，诸如与显示，电话呼叫，数据通信，相机操作和记录操作相关联的操作。处理组件802可以包括一个或多个处理器820来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件802可以包括一个或多个模块，便于处理组件802和其他组件之间的交互。例如，处理组件802可以包括多媒体模块，以方便多媒体组件808和处理组件802之间的交互。
存储器804被配置为存储各种类型的数据以支持在电子设备800的操作。这些数据的示例包括用于在电子设备800上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。存储器804可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。
电源组件806为电子设备800的各种组件提供电力。电源组件806可以包括电源管理***，一个或多个电源，及其他与为电子设备800生成、管理和分配电力相关联的组件。
多媒体组件808包括在所述电子设备800和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。所述触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与所述触摸或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件808包括一个前置摄像头和/或后置摄像头。当电子设备800处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜***或具有焦距和光学变焦能力。
音频组件810被配置为输出和/或输入音频信号。例如，音频组件810包括一个麦克风(MIC)，当电子设备800处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器804或经由通信组件816发送。在一些实施例中，音频组件810还包括一个扬声器，用于输出音频信号。
I/O接口812为处理组件802和***接口模块之间提供接口，上述***接口模块可以是键盘，点击轮，按钮等。这 些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。
传感器组件814包括一个或多个传感器，用于为电子设备800提供各个方面的状态评估。例如，传感器组件814可以检测到电子设备800的打开/关闭状态，组件的相对定位，例如所述组件为电子设备800的显示器和小键盘，传感器组件814还可以检测电子设备800或电子设备800一个组件的位置改变，用户与电子设备800接触的存在或不存在，电子设备800方位或加速/减速和电子设备800的温度变化。传感器组件814可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件814还可以包括光传感器，如CMOS或CCD图像传感器，用于在成像应用中使用。在一些实施例中，该传感器组件814还可以包括加速度传感器，陀螺仪传感器，磁传感器，压力传感器或温度传感器。
通信组件816被配置为便于电子设备800和其他设备之间有线或无线方式的通信。电子设备800可以接入基于通信标准的无线网络，如WiFi，2G或3G，或它们的组合。在一个示例性实施例中，通信组件816经由广播信道接收来自外部广播管理***的广播信号或广播相关信息。在一个示例性实施例中，所述通信组件816还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。
在示例性实施例中，电子设备800可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述方法。
在示例性实施例中，还提供了一种非易失性计算机可读存储介质，例如包括计算机程序指令的存储器804，上述计算机程序指令可由电子设备800的处理器820执行以完成上述方法。
本公开可以是***、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本公开的各个方面的计算机可读程序指令。计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是――但不限于――电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。用于执行本公开操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本公开的各个方面。
这里参照根据本公开实施例的方法、装置(***)和计算机程序产品的流程图和/或框图描述了本公开的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指 令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本公开的多个实施例的***、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的***来实现，或者可以用专用硬件与计算机指令的组合来实现。在不违背逻辑的情况下，本申请不同实施例之间可以相互结合，不同实施例描述有所侧重，为侧重描述的部分可以参见其他实施例的记载。
以上已经描述了本公开的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。

Claims (45)

  1. 一种动态动作检测方法，其特征在于，所述方法包括：
    将视频流中的帧图像加入第一队列，获得帧图像部分更新的所述第一队列；
    检测部分更新后的所述第一队列的帧图像中的动态动作；
    在所述动态动作与动作检测结果不匹配的情况下，根据所述动态动作更新所述动作检测结果，所述动作检测结果包括动作参考结果或在先检测的动作检测结果。
  2. 根据权利要求1所述的方法，其特征在于，所述在先检测的动作检测结果的获取方法，包括：
    在所述第一队列的帧图像中有动态动作的情况下，将所述第一队列中的帧图像移至第二队列中；
    根据所述第二队列中的帧图像确定所述动作检测结果。
  3. 根据权利要求1或2所述的方法，其特征在于，所述动作检测结果包括静态动作，所述将视频流中的帧图像加入第一队列，获得帧图像部分更新的所述第一队列，包括：
    获取所述视频流中待识别的帧图像中的静态动作；
    在所述待识别的帧图像中的静态动作与所述动作检测结果中的静态动作不匹配的情况下，将所述待识别的帧图像加入所述第一队列，获得帧图像部分更新的所述第一队列。
  4. 根据权利要求3所述的方法，其特征在于，所述动作检测结果还包括所述静态动作的动作轨迹和/或切换信息，所述将视频流中的帧图像加入第一队列，获得帧图像部分更新的所述第一队列，包括：
    在所述待识别的帧图像中的静态动作与所述动作检测结果中的静态动作匹配的情况下，根据所述待识别的帧图像和所述第二队列中的帧图像，确定所述待识别的帧图像中静态动作的动作轨迹和/或切换信息；
    在所述待识别的帧图像中静态动作的动作轨迹和/或切换信息与所述动作检测结果不匹配的情况下，将所述待识别的帧图像加入所述第一队列，获得帧图像部分更新的所述第一队列。
  5. 根据权利要求2至4中任一项所述的方法，其特征在于，所述方法还包括：
    在根据部分更新后的所述第一队列的帧图像检测到的所述动态动作与动作检测结果不匹配的情况下，清空所述第二队列，将所述第一队列中的帧图像移至所述第二队列中；
    根据所述第二队列中的帧图像中的动态动作，更新所述动作检测结果。
  6. 根据权利要求4或5所述的方法，其特征在于，所述方法还包括：
    在所述待识别的帧图像中的静态动作、静态动作的动作轨迹和/或切换信息与所述动作检测结果匹配的情况下，将所述待识别的帧图像加入所述第二队列。
  7. 根据权利要求1至6中任一项所述的方法，其特征在于，所述动态动作包括：至少一个静态动作和所述静态动作的动作轨迹，所述检测部分更新后的所述第一队列的帧图像中的动态动作，包括：
    根据部分更新后的所述第一队列的各帧图像中的静态动作和各帧图像中的静态动作的动作轨迹，获得部分更新后的所述第一队列的帧图像中的动态动作。
  8. 根据权利要求1至6中任一项所述的方法，其特征在于，所述动态动作包括：至少两个静态动作和静态动作间的切换信息，所述检测部分更新后的所述第一队列的帧图像中的动态动作，包括：
    根据部分更新后的所述第一队列的各帧图像中的静态动作和静态动作的切换信息，获得部分更新后的所述第一队列的帧图像中的动态动作。
  9. 根据权利要求1至6中任一项所述的方法，其特征在于，所述动态动作包括：至少两个静态动作、各静态动作的动作轨迹和静态动作间的切换信息，所述检测部分更新后的所述第一队列的帧图像中的动态动作，包括：
    根据部分更新后的所述第一队列的各帧图像中的静态动作、各静态动作的动作轨迹和静态动作间的切换信息，获得部分更新后的所述第一队列的帧图像中的动态动作。
  10. 根据权利要求3至9中任一项所述的方法，其特征在于，所述动态动作包括手部动态动作，所述静态动作包括手势，所述获取所述视频流中待识别的帧图像中的静态动作，包括：
    检测所述待识别的帧图像中是否包括手部；
    在所述待识别图像中包括手部的情况下，根据所述手部上的手指和/或手掌获得所述待识别图像中的手势。
  11. 根据权利要求10所述的方法，其特征在于，所述根据所述手部上的手指和/或手掌判断所述待识别图像中的手势，包括：
    根据所述手部上的手指的第一关键点的位置和/或手掌的第二关键点的位置识别所述待识别图像中的手势，所述手 指的第一关键点包括指尖或指关节，所述手掌的第二关键点包括掌心、手掌背部或手掌边缘上的点。
  12. 根据权利要求11所述的方法，其特征在于，所述手部动态动作还包括手势的动作轨迹和/或切换信息，所述根据所述待识别的帧图像和所述第二队列中的帧图像，确定所述待识别的帧图像中静态动作的动作轨迹和/或切换信息，包括：
    根据所述待识别的帧图像中的第一关键点和与之对应的所述第二队列的帧图像中的第一关键点，确定所述待识别的帧图像中的动作轨迹和/或切换信息，和/或
    根据所述待识别的帧图像中的第二关键点和与之对应的所述第二队列的帧图像中的第二关键点，确定所述待识别的帧图像中的动作轨迹和/或切换信息。
  13. 根据权利要求1至12中任一项所述的方法，其特征在于，所述动态动作包括静态动作和静态动作的动作轨迹，所述检测部分更新后的所述第一队列的帧图像中的动态动作，包括：
    检测所述第一队列的帧图像中的静态动作；
    计算目标帧图像中的静态动作相对于所述第一队列的第一个帧图像中的静态动作在X轴方向和Y轴方向上的累积移动距离，所述目标帧图像包括所述第一队列中除所述第一个帧图像外的任一帧图像；
    根据所述目标帧图像中静态动作和所述累积移动距离获得所述第一队列的帧图像中的动态动作。
  14. 根据权利要求13所述的方法，其特征在于，所述根据所述目标帧图像中静态动作和所述累积移动距离获得所述第一队列的帧图像中的动态动作，包括：
    在所述目标帧图像中静态动作的累积移动距离小于移动阈值的情况下，确定所述第一队列的帧图像中目标对象的动态动作为间断动作。
  15. 根据权利要求13所述的方法，其特征在于，所述根据所述目标帧图像中静态动作和所述累积移动距离获得所述第一队列的帧图像中的动态动作，包括：
    在所述累积移动距离大于等于所述移动阈值的情况下，计算各所述帧图像中静态动作的累积移动距离相对于所述静态动作所属的目标对象的宽度之间的比值；
    在所述比值小于比值阈值的情况下，确定目标帧图像中静态动作的动作轨迹为旋转动作轨迹；
    根据所述目标帧图像中的静态动作和所述旋转动作轨迹，获得所述第一队列的帧图像中的动态动作。
  16. 根据权利要求15所述的方法，其特征在于，所述方法还包括：
    在所述比值大于等于所述比值阈值的情况下，确定目标帧图像中静态动作的动作轨迹为平移动作轨迹；
    根据所述目标帧图像中的静态动作和所述平移动作轨迹，获得所述第一队列的帧图像中的动态动作。
  17. 一种动态动作控制方法，其特征在于，所述方法包括：
    获取视频流；
    采用如权利要求1至16中任一项所述的方法确定所述视频流中的动作检测结果；
    控制设备执行与所述动作检测结果相应的操作。
  18. 根据权利要求17所述的方法，其特征在于，控制设备执行与所述动作检测结果相应的操作，包括：
    根据预先确定的动作检测结果与操作指令之间的对应关系，获取与所述动作检测结果对应的操作指令；
    根据所述操作指令控制设备执行相应操作。
  19. 根据权利要求18所述的方法，其特征在于，所述根据所述操作指令控制设备执行相应操作，包括：
    根据所述操作指令对车辆的车窗、车门、车载***或车载设备进行控制。
  20. 根据权利要求17所述的方法，其特征在于，控制设备执行与所述动作检测结果相应的操作，包括：
    响应于所述检测结果为预定义动态动作，控制车辆执行与所述预定义动态动作对应的操作。
  21. 根据权利要求20所述的方法，其特征在于，所述预定义动态动作包括动态手势，所述动态手势包括以下至少之一：单指顺/逆时针旋转、手掌左/右挥动、两指前戳、拇指和小指伸出、手掌朝下向下压、拇指伸出左/右移动、手掌左/右长滑、掌心向上拳变掌、掌心向上掌变拳、掌心向下掌变拳、掌心向下拳变掌、单指滑动、多指向内捏住、单指双击、单指单击、多指双击、多指单击；
    与所述预定义动态动作对应的操作包括以下至少之一：调节音量大/小，歌曲切换，歌曲暂停/继续，接听或启动电话、挂断或拒接电话、空调温度升高或降低、多屏互动、打开天窗、关闭天窗、锁紧门锁、解锁门锁、拖动导航、缩小地图、放大地图。
  22. 一种动态动作检测装置，其特征在于，所述装置包括：
    第一队列更新模块，用于将视频流中的帧图像加入第一队列，获得帧图像部分更新的所述第一队列；
    动态动作检测模块，用于检测部分更新后的所述第一队列的帧图像中的动态动作；
    动作检测结果更新模块，用于在所述动态动作与动作检测结果不匹配的情况下，根据所述动态动作更新所述动作检测结果，所述动作检测结果包括动作参考结果或在先检测的动作检测结果。
  23. 根据权利要求22所述的装置，其特征在于，所述装置还包括：
    在先检测的动作检测结果获取模块，用于在所述第一队列的帧图像中有动态动作的情况下，将所述第一队列中的帧图像移至第二队列中；根据所述第二队列中的帧图像确定所述动作检测结果。
  24. 根据权利要求22或23所述的装置，其特征在于，所述动作检测结果包括静态动作，所述第一队列更新模块，包括：
    第一获取子模块，用于获取所述视频流中待识别的帧图像中的静态动作；
    第一更新子模块，用于在所述待识别的帧图像中的静态动作与所述动作检测结果中的静态动作不匹配的情况下，将所述待识别的帧图像加入所述第一队列，获得帧图像部分更新的所述第一队列。
  25. 根据权利要求24所述的装置，其特征在于，所述动作检测结果还包括所述静态动作的动作轨迹和/或切换信息，所述第一队列更新模块，包括：
    第二获取子模块，用于在所述待识别的帧图像中的静态动作与所述动作检测结果中的静态动作匹配的情况下，根据所述待识别的帧图像和所述第二队列中的帧图像，确定所述待识别的帧图像中静态动作的动作轨迹和/或切换信息；
    第二更新子模块，用于在所述待识别的帧图像中静态动作的动作轨迹和/或切换信息与所述动作检测结果不匹配的情况下，将所述待识别的帧图像加入所述第一队列，获得帧图像部分更新的所述第一队列。
  26. 根据权利要求23至25中任一项所述的装置，其特征在于，所述装置还包括：
    队列转移模块，用于在根据部分更新后的所述第一队列的帧图像检测到的所述动态动作与动作检测结果不匹配的情况下，清空所述第二队列，将所述第一队列中的帧图像移至所述第二队列中；
    动作检测结果更新模块，用于根据所述第二队列中的帧图像中的动态动作，更新所述动作检测结果。
  27. 根据权利要求25或26所述的装置，其特征在于，所述装置还包括：
    第二队列更新模块，用于在所述待识别的帧图像中的静态动作、静态动作的动作轨迹和/或切换信息与所述动作检测结果匹配的情况下，将所述待识别的帧图像加入所述第二队列。
  28. 根据权利要求22至27中任一项所述的装置，其特征在于，所述动态动作包括：至少一个静态动作和所述静态动作的动作轨迹，所述动态动作检测模块，包括：
    第一动态动作检测子模块，用于根据部分更新后的所述第一队列的各帧图像中的静态动作和各帧图像中的静态动作的动作轨迹，获得部分更新后的所述第一队列的帧图像中的动态动作。
  29. 根据权利要求22至27中任一项所述的装置，其特征在于，所述动态动作包括：至少两个静态动作和静态动作间的切换信息，所述动态动作检测模块，包括：
    第二动态动作检测子模块，用于根据部分更新后的所述第一队列的各帧图像中的静态动作和静态动作的切换信息，获得部分更新后的所述第一队列的帧图像中的动态动作。
  30. 根据权利要求22至27中任一项所述的装置，其特征在于，所述动态动作包括：至少两个静态动作、各静态动作的动作轨迹和静态动作间的切换信息，所述动态动作检测模块，包括：
    第三动态动作检测子模块，用于根据部分更新后的所述第一队列的各帧图像中的静态动作、各静态动作的动作轨迹和静态动作间的切换信息，获得部分更新后的所述第一队列的帧图像中的动态动作。
  31. 根据权利要求24至30中任一项所述的装置，其特征在于，所述动态动作包括手部动态动作，所述静态动作包括手势，所述第一获取子模块，包括：
    手部检测单元，用于检测所述待识别的帧图像中是否包括手部；
    手势确定单元，用于在所述待识别图像中包括手部的情况下，根据所述手部上的手指和/或手掌获得所述待识别图像中的手势。
  32. 根据权利要求31所述的装置，其特征在于，所述手势确定单元，用于：
    根据所述手部上的手指的第一关键点的位置和/或手掌的第二关键点的位置识别所述待识别图像中的手势，所述手 指的第一关键点包括指尖或指关节，所述手掌的第二关键点包括掌心、手掌背部或手掌边缘上的点。
  33. 根据权利要求32所述的装置，其特征在于，所述手部动态动作还包括手势的动作轨迹和/或切换信息，所述第二获取子模块，包括：
    第一获取单元，用于根据所述待识别的帧图像中的第一关键点和与之对应的所述第二队列的帧图像中的第一关键点，确定所述待识别的帧图像中的动作轨迹和/或切换信息，和/或
    第二获取单元，用于根据所述待识别的帧图像中的第二关键点和与之对应的所述第二队列的帧图像中的第二关键点，确定所述待识别的帧图像中的动作轨迹和/或切换信息。
  34. 根据权利要求22至33中任一项所述的装置，其特征在于，所述动态动作包括静态动作和静态动作的动作轨迹，所述动态动作检测模块，包括：
    静态动作检测子模块，用于检测所述第一队列的帧图像中的静态动作；
    累积移动距离确定子模块，用于计算目标帧图像中的静态动作相对于所述第一队列的第一个帧图像中的静态动作在X轴方向和Y轴方向上的累积移动距离，所述目标帧图像包括所述第一队列中除所述第一个帧图像外的任一帧图像；
    第四动态动作检测子模块，用于根据所述目标帧图像中静态动作和所述累积移动距离获得所述第一队列的帧图像中的动态动作。
  35. 根据权利要求34所述的装置，其特征在于，所述第四动态动作检测子模块，用于：
    在所述目标帧图像中静态动作的累积移动距离小于移动阈值的情况下，确定所述第一队列的帧图像中目标对象的动态动作为间断动作。
  36. 根据权利要求34所述的装置，其特征在于，所述第四动态动作检测子模块，用于：
    在所述累积移动距离大于等于所述移动阈值的情况下，计算各所述帧图像中静态动作的累积移动距离相对于所述静态动作所属的目标对象的宽度之间的比值；
    在所述比值小于比值阈值的情况下，确定目标帧图像中静态动作的动作轨迹为旋转动作轨迹；
    根据所述目标帧图像中的静态动作和所述旋转动作轨迹，获得所述第一队列的帧图像中的动态动作。
  37. 根据权利要求36所述的装置，其特征在于，所述第四动态动作检测子模块，还用于：
    在所述比值大于等于所述比值阈值的情况下，确定目标帧图像中静态动作的动作轨迹为平移动作轨迹；
    根据所述目标帧图像中的静态动作和所述平移动作轨迹，获得所述第一队列的帧图像中的动态动作。
  38. 一种动态动作控制装置，其特征在于，所述装置包括：
    视频流获取模块，用于获取视频流；
    动作检测结果获取模块，用于采用如权利要求22至37中任一项所述的装置确定所述视频流中的动作检测结果；
    操作执行模块，用于控制设备执行与所述动作检测结果相应的操作。
  39. 根据权利要求38所述的装置，其特征在于，操作执行模块，包括：
    操作指令获取子模块，用于根据预先确定的动作检测结果与操作指令之间的对应关系，获取与所述动作检测结果对应的操作指令；
    操作执行子模块，用于根据所述操作指令控制设备执行相应操作。
  40. 根据权利要求39所述的装置，其特征在于，所述操作执行子模块，用于：
    根据所述操作指令对车辆的车窗、车门、车载***或车载设备进行控制。
  41. 根据权利要求38所述的装置，其特征在于，所述操作执行模块，还用于：
    响应于所述检测结果为预定义动态动作，控制车辆执行与所述预定义动态动作对应的操作。
  42. 根据权利要求41所述的装置，其特征在于，所述预定义动态动作包括动态手势，所述动态手势包括以下至少之一：单指顺/逆时针旋转、手掌左/右挥动、两指前戳、拇指和小指伸出、手掌朝下向下压、拇指伸出左/右移动、手掌左/右长滑、掌心向上拳变掌、掌心向上掌变拳、掌心向下掌变拳、掌心向下拳变掌、单指滑动、多指向内捏住、单指双击、单指单击、多指双击、多指单击；
    与所述预定义动态动作对应的操作包括以下至少之一：调节音量大/小，歌曲切换，歌曲暂停/继续，接听或启动电话、挂断或拒接电话、空调温度升高或降低、多屏互动、打开天窗、关闭天窗、锁紧门锁、解锁门锁、拖动导航、缩小地图、放大地图。
  43. 一种电子设备，其特征在于，包括：
    处理器；
    用于存储处理器可执行指令的存储器；
    其中，所述处理器被配置为：执行权利要求1至21中任意一项所述的方法。
  44. 一种计算机可读存储介质，其上存储有计算机程序指令，其特征在于，所述计算机程序指令被处理器执行时实现权利要求1至21中任意一项所述的方法。
  45. 一种计算机程序，其特征在于，所述计算机程序包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行用于实现权利要求1至21中的任意一项所述的方法。
PCT/CN2019/093583 2018-08-24 2019-06-28 动态动作检测方法、动态动作控制方法及装置 WO2020038108A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
KR1020217005497A KR102321562B1 (ko) 2018-08-24 2019-06-28 동적 동작 검출 방법, 동적 동작 제어 방법 및 장치
JP2021502886A JP7127202B2 (ja) 2018-08-24 2019-06-28 動的動作検出方法、動的動作制御方法及び装置
SG11202013098TA SG11202013098TA (en) 2018-08-24 2019-06-28 Dynamic motion detection method and dynamic motion control method and device
US17/257,136 US11455836B2 (en) 2018-08-24 2019-06-28 Dynamic motion detection method and apparatus, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810974244.8A CN109144260B (zh) 2018-08-24 2018-08-24 动态动作检测方法、动态动作控制方法及装置
CN201810974244.8 2018-08-24

Publications (1)

Publication Number Publication Date
WO2020038108A1 true WO2020038108A1 (zh) 2020-02-27

Family

ID=64827872

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/093583 WO2020038108A1 (zh) 2018-08-24 2019-06-28 动态动作检测方法、动态动作控制方法及装置

Country Status (6)

Country Link
US (1) US11455836B2 (zh)
JP (1) JP7127202B2 (zh)
KR (1) KR102321562B1 (zh)
CN (1) CN109144260B (zh)
SG (1) SG11202013098TA (zh)
WO (1) WO2020038108A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11455836B2 (en) 2018-08-24 2022-09-27 Shanghai Sensetime Intelligent Technology Co., Ltd. Dynamic motion detection method and apparatus, and storage medium
CN115600171A (zh) * 2022-10-24 2023-01-13 星空映画（武汉）科技有限公司（Cn） 一种根据用户习惯判断的计算机用身份识别***
US11829443B2 (en) 2021-03-29 2023-11-28 International Business Machines Corporation Augmentation of video datasets for machine learning training

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11873000B2 (en) 2020-02-18 2024-01-16 Toyota Motor North America, Inc. Gesture detection for transport control
US20210253135A1 (en) * 2020-02-18 2021-08-19 Toyota Motor North America, Inc. Determining transport operation level for gesture control
EP3926544B1 (en) 2020-06-18 2024-03-13 Tata Consultancy Services Limited System and method of gesture recognition using a reservoir based convolutional spiking neural network

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102609093A (zh) * 2012-02-16 2012-07-25 中国农业大学 用手势控制视频播放的方法及装置
US20130300749A1 (en) * 2012-05-10 2013-11-14 Sony Computer Entertainment Inc. Image generating device, image generating method, and information storage medium
CN105894540A (zh) * 2016-04-11 2016-08-24 上海斐讯数据通信技术有限公司 一种基于移动终端的上下往复移动计数方法及***
CN109144260A (zh) * 2018-08-24 2019-01-04 上海商汤智能科技有限公司 动态动作检测方法、动态动作控制方法及装置

Family Cites Families (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004199623A (ja) 2002-12-20 2004-07-15 Minolta Co Ltd 移動体の計数方法
US7308112B2 (en) * 2004-05-14 2007-12-11 Honda Motor Co., Ltd. Sign based human-machine interaction
JP2006345081A (ja) 2005-06-07 2006-12-21 Sony Corp 検出装置および方法、記録媒体、プログラム、並びに画像表示装置
US20070057912A1 (en) 2005-09-14 2007-03-15 Romriell Joseph N Method and system for controlling an interface of a device through motion gestures
WO2009049314A2 (en) 2007-10-11 2009-04-16 Trustees Of Boston University Video processing system employing behavior subtraction between reference and observed video image sequences
CN101419499B (zh) * 2008-11-14 2010-06-02 东南大学 基于摄像头和话筒的多媒体人机交互方法
TWI476632B (zh) 2009-12-08 2015-03-11 Micro Star Int Co Ltd 運動物體辨識方法及基於運動物體辨識之指令輸入方法
US8751215B2 (en) * 2010-06-04 2014-06-10 Microsoft Corporation Machine based sign language interpreter
CN102053702A (zh) * 2010-10-26 2011-05-11 南京航空航天大学 动态手势控制***与方法
US20150193393A1 (en) * 2011-09-01 2015-07-09 Scott R. Violet Dynamic Display of Web Content
EP2650754A3 (en) * 2012-03-15 2014-09-24 Omron Corporation Gesture recognition apparatus, electronic device, gesture recognition method, control program, and recording medium
JP5906993B2 (ja) 2012-08-22 2016-04-20 富士通株式会社 符号化装置、符号化方法、及びプログラム
CN104239844A (zh) * 2013-06-18 2014-12-24 华硕电脑股份有限公司 图像识别***及图像识别方法
US9886094B2 (en) 2014-04-28 2018-02-06 Microsoft Technology Licensing, Llc Low-latency gesture detection
US20160162148A1 (en) * 2014-12-04 2016-06-09 Google Inc. Application launching and switching interface
US10097758B2 (en) * 2015-11-18 2018-10-09 Casio Computer Co., Ltd. Data processing apparatus, data processing method, and recording medium
JP2017191496A (ja) * 2016-04-14 2017-10-19 株式会社東海理化電機製作所 ジェスチャ判定装置
CN106648112A (zh) * 2017-01-07 2017-05-10 武克易 一种体感动作识别方法
CN106599882A (zh) * 2017-01-07 2017-04-26 武克易 一种体感动作识别装置
CN107169411B (zh) * 2017-04-07 2019-10-29 南京邮电大学 一种基于关键帧和边界约束dtw的实时动态手势识别方法
US10572072B2 (en) * 2017-09-29 2020-02-25 Apple Inc. Depth-based touch detection
CN108197596B (zh) * 2018-01-24 2021-04-06 京东方科技集团股份有限公司 一种手势识别方法和装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102609093A (zh) * 2012-02-16 2012-07-25 中国农业大学 用手势控制视频播放的方法及装置
US20130300749A1 (en) * 2012-05-10 2013-11-14 Sony Computer Entertainment Inc. Image generating device, image generating method, and information storage medium
CN105894540A (zh) * 2016-04-11 2016-08-24 上海斐讯数据通信技术有限公司 一种基于移动终端的上下往复移动计数方法及***
CN109144260A (zh) * 2018-08-24 2019-01-04 上海商汤智能科技有限公司 动态动作检测方法、动态动作控制方法及装置

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11455836B2 (en) 2018-08-24 2022-09-27 Shanghai Sensetime Intelligent Technology Co., Ltd. Dynamic motion detection method and apparatus, and storage medium
US11829443B2 (en) 2021-03-29 2023-11-28 International Business Machines Corporation Augmentation of video datasets for machine learning training
CN115600171A (zh) * 2022-10-24 2023-01-13 星空映画（武汉）科技有限公司（Cn） 一种根据用户习惯判断的计算机用身份识别***
CN115600171B (zh) * 2022-10-24 2023-08-04 星空映画(武汉)科技有限公司 一种根据用户习惯判断的计算机用户动作识别***

Also Published As

Publication number Publication date
KR20210030983A (ko) 2021-03-18
US20210248359A1 (en) 2021-08-12
US11455836B2 (en) 2022-09-27
CN109144260B (zh) 2020-08-18
JP2021524116A (ja) 2021-09-09
KR102321562B1 (ko) 2021-11-04
JP7127202B2 (ja) 2022-08-29
CN109144260A (zh) 2019-01-04
SG11202013098TA (en) 2021-02-25

Similar Documents

Publication Publication Date Title
WO2020038108A1 (zh) 动态动作检测方法、动态动作控制方法及装置
JP6328859B2 (ja) アプリケーションプログラム切替え方法、装置及び機器
CN104461232A (zh) 在屏幕显示过程中确定缩小比例的方法及装置
WO2022110614A1 (zh) 手势识别方法及装置、电子设备和存储介质
WO2020078105A1 (zh) 姿势检测方法、装置、设备及存储介质
CN104090721A (zh) 终端控制方法和装置
KR20170142839A (ko) 단말기의 작업모드 확정방법, 장치, 프로그램 및 기록매체
CN104536684A (zh) 界面显示方法及装置
CN104065883B (zh) 拍摄方法及装置
WO2017096958A1 (zh) 人机交互方法及装置、移动设备
CN106484138A (zh) 一种输入方法及装置
CN106168877A (zh) 进入单手模式的方法及装置
CN103927101A (zh) 操作控件的方法和装置
CN105373334A (zh) 交互式屏幕控制方法与装置
CN103955274A (zh) 应用控制方法和装置
KR20210005508A (ko) 터치신호의 처리방법, 장치 및 매체
CN108803892A (zh) 一种在输入法中调用第三方应用程序的方法和装置
CN108398127A (zh) 一种室内定位方法及装置
US20210158031A1 (en) Gesture Recognition Method, and Electronic Device and Storage Medium
CN113936697A (zh) 语音处理方法、装置以及用于语音处理的装置
EP4290338A1 (en) Method and apparatus for inputting information, and storage medium
CN108073291A (zh) 一种输入方法和装置、一种用于输入的装置
CN105607850A (zh) 持握方式确定方法及装置
CN107861683B (zh) 无人机无按钮操作方法及装置
CN111610921A (zh) 手势识别方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19852880

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021502886

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 20217005497

Country of ref document: KR

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19852880

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 06/08/2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19852880

Country of ref document: EP

Kind code of ref document: A1