CN112686169A - Gesture recognition control method and device, electronic equipment and storage medium - Google Patents
Gesture recognition control method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112686169A CN112686169A CN202011635321.0A CN202011635321A CN112686169A CN 112686169 A CN112686169 A CN 112686169A CN 202011635321 A CN202011635321 A CN 202011635321A CN 112686169 A CN112686169 A CN 112686169A
- Authority
- CN
- China
- Prior art keywords
- gesture
- target
- image
- target user
- starting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 230000006399 behavior Effects 0.000 claims description 38
- 238000012545 processing Methods 0.000 claims description 30
- 230000015654 memory Effects 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 14
- 230000000977 initiatory effect Effects 0.000 claims description 6
- 230000001960 triggered effect Effects 0.000 abstract description 10
- 210000003811 finger Anatomy 0.000 description 18
- 238000004891 communication Methods 0.000 description 17
- 230000006870 function Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 11
- 210000003813 thumb Anatomy 0.000 description 9
- 229920001621 AMOLED Polymers 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000037237 body shape Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 241000203475 Neopanax arboreus Species 0.000 description 1
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 239000002096 quantum dot Substances 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Landscapes
- User Interface Of Digital Computer (AREA)
Abstract
The embodiment of the application discloses a gesture recognition control method, a gesture recognition control device, electronic equipment and a storage medium, wherein an image containing a plurality of users is obtained; determining a target starting gesture which is closest to the central area of the image according to the image; the control mode of the business logic is triggered according to the target starting gesture, so that the control mode of the business logic can be triggered according to the target starting gesture closest to the central area of the image in a multi-person scene, and gesture control can be rapidly and accurately carried out in the multi-person scene.
Description
Technical Field
The application relates to the technical field of electronic equipment, in particular to a gesture recognition control method and device, electronic equipment and a storage medium.
Background
At present, many electronic devices adopt gesture recognition technology to realize device control, and in practical application, gestures of multiple people can be recognized in a scene of multiple people, so that the problem that the gesture control is difficult to accurately perform may exist, and thus user experience is influenced.
Disclosure of Invention
The embodiment of the application provides a gesture recognition control method and device, electronic equipment and a storage medium, which can start a gesture trigger service logic control mode according to a target closest to a central area of an image in a multi-user scene, so that gesture control can be rapidly and accurately performed in the multi-user scene.
In a first aspect, an embodiment of the present application provides a gesture recognition control method, where the method includes:
acquiring an image containing a plurality of users;
determining a target starting gesture closest to a central area of the image according to the image;
and triggering a control mode of the business logic according to the target starting gesture.
In a second aspect, an embodiment of the present application provides a gesture recognition control apparatus, which is applied to an electronic device, and the apparatus includes:
a processing unit for acquiring an image containing a plurality of users;
the processing unit is further used for determining a target starting gesture which is closest to the central area of the image according to the image;
and the processing unit is also used for triggering a control mode of the business logic according to the target starting gesture.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing the steps in the first aspect of the embodiment of the present application.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program for electronic data exchange, where the computer program enables a computer to perform some or all of the steps described in the first aspect of the embodiment of the present application.
In a fifth aspect, embodiments of the present application provide a computer program product, where the computer program product includes a non-transitory computer-readable storage medium storing a computer program, where the computer program is operable to cause a computer to perform some or all of the steps as described in the first aspect of the embodiments of the present application. The computer program product may be a software installation package.
The embodiment of the application has the following beneficial effects:
it can be seen that the gesture recognition control method, apparatus, electronic device and storage medium described in the embodiments of the present application acquire an image including a plurality of users; determining a target starting gesture which is closest to a central area of an image according to the image; the control mode of the business logic is triggered according to the target starting gesture, so that the control mode of the business logic can be triggered according to the target starting gesture closest to the central area of the image in a multi-person scene, and gesture control can be rapidly and accurately carried out in the multi-person scene.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1A is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure;
fig. 1B is a scene schematic diagram illustrating gesture control performed in a multi-person scene according to an embodiment of the present disclosure;
fig. 1C is a schematic flowchart of a gesture recognition control method according to an embodiment of the present disclosure;
FIG. 1D is a schematic diagram of a preset unlock gesture provided by an embodiment of the present application;
FIG. 1E is a schematic view of a scene for recognizing a target start gesture of a target user according to an embodiment of the present disclosure;
FIG. 2 is a schematic flow chart illustrating another gesture recognition control method provided in the embodiments of the present application;
fig. 3 is a schematic structural diagram of another electronic device provided in an embodiment of the present application;
fig. 4 is a block diagram illustrating functional units of a gesture recognition control device according to an embodiment of the present disclosure.
Detailed Description
The terms "first," "second," and the like in the description and claims of the present application and in the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
In order to make the technical solutions of the present application better understood, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The electronic device in the embodiments of the present application may refer to various forms of terminals, access terminals, subscriber units, subscriber stations, Mobile Stations (MSs), remote terminals, mobile devices, user terminals, terminal devices (terminal equipment), wireless communication devices, and the like. The electronic device may also be a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, a projection display device, a vehicle-mounted device, a wearable device, and the like, which are not limited in this embodiment.
As shown in fig. 1A, fig. 1A is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. The electronic device includes a camera module, a processor, a Memory, a signal processor, a signal transceiver, a speaker, a microphone, a Random Access Memory (RAM), a sensor, a network module, and the like. The camera module, the processor, the Memory, the signal processor, the speaker, the microphone, the Random Access Memory (RAM), the sensor and the network module are connected with the processor, and the signal transceiver is connected with the signal processor.
The Processor is a control center of the electronic device, connects various parts of the whole electronic device by using various interfaces and lines, executes various functions and processes data of the electronic device by running or executing software programs and/or modules stored in the memory and calling the data stored in the memory, thereby performing overall monitoring on the electronic device, and may be a Central Processing Unit (CPU), a Graphics Processing Unit (GPU) or a Network Processing Unit (NPU).
Further, the processor may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor.
The memory is used for storing software programs and/or modules, and the processor executes various functional applications and data processing of the electronic equipment by operating the software programs and/or modules stored in the memory. The memory mainly comprises a program storage area and a data storage area, wherein the program storage area can store an operating system, a software program required by at least one function and the like; the storage data area may store data created according to use of the electronic device, and the like. Further, the memory may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
Wherein the sensor comprises at least one of: light-sensitive sensors, gyroscopes, infrared proximity sensors, vibration detection sensors, pressure sensors, etc. Among them, the light sensor, also called an ambient light sensor, is used to detect the ambient light brightness. The light sensor may include a light sensitive element and an analog to digital converter. The photosensitive element is used for converting collected optical signals into electric signals, and the analog-to-digital converter is used for converting the electric signals into digital signals. Optionally, the light sensor may further include a signal amplifier, and the signal amplifier may amplify the electrical signal converted by the photosensitive element and output the amplified electrical signal to the analog-to-digital converter. The photosensitive element may include at least one of a photodiode, a phototransistor, a photoresistor, and a silicon photocell.
The camera module can include a camera, the camera can be a visible light camera (a general visual angle camera, a wide angle camera), an infrared camera, or a double-camera (having a distance measuring function), and is not limited herein.
The electronic device may further include a display unit, and the display function may be realized by the display. Alternatively, the display unit may be a display, which may include a display panel for displaying video or images. The display panel may be a Liquid Crystal Display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode (active-matrix organic light-emitting diode, AMOLED), a flexible light-emitting diode (FLED), a mini light-emitting diode (mini-light-emitting diode, mini), a Micro-o led, a quantum dot light-emitting diode (QLED), or the like.
Alternatively, if the electronic device is a projection display device, the display unit may include a curtain or a wall, and the projection display may be implemented through the curtain or the wall.
The network module may be at least one of: a bluetooth module, a wireless fidelity (Wi-Fi), etc., which are not limited herein.
As shown in fig. 1B, fig. 1B is a scene schematic diagram of performing gesture control in a multi-user scene according to an embodiment of the present disclosure, where the electronic device may include a display unit 110, a camera module 111, and a processor 120, and a plurality of users 130 perform gesture operations in front of the display, where the display unit 110 may be a display, a curtain, a wall, and the like, and the present disclosure is not limited thereto, and the electronic device may adopt an image that includes a plurality of users and is acquired by the camera module 111, and then the processor performs the following operations:
acquiring an image which is acquired by the camera module and contains a plurality of users;
determining a target starting gesture closest to a central area of the image according to the image;
and triggering a control mode of the business logic according to the target starting gesture.
It can be seen that the electronic device described in the embodiment of the present application is applied to an electronic device, where the electronic device includes a camera module, and obtains an image including a plurality of users by starting the electronic device; determining a target starting gesture which is closest to the central area of the image according to the image; and triggering a control mode of the business logic according to the target starting gesture, thereby realizing the gesture control on the electronic equipment quickly and accurately under the multi-person scene.
Optionally, the electronic device in this embodiment of the application may be a projection display device, and the gesture recognition control method in a multi-user scene may be implemented by the projection display device, specifically, the multiple users 130 execute gesture operations in front of a display, a curtain, or a wall, the projection display device may adopt images acquired by the camera module 111 and including the multiple users, and then the processor acquires the images acquired by the camera module and including the multiple users; determining a target starting gesture which is closest to the central area of the image according to the image; and triggering a control mode of the business logic according to the target starting gesture, thereby realizing the gesture control on the projection display equipment rapidly and accurately under the multi-person scene.
The following describes embodiments of the present application in detail.
Referring to fig. 1C, fig. 1C is a schematic flowchart of a gesture recognition control method according to an embodiment of the present application, where the gesture recognition control method includes:
101. an image containing a plurality of users is acquired.
Wherein, electronic equipment can include the module of making a video recording, and under many people's scenes, electronic equipment accessible module collection contains the image of a plurality of users.
In specific implementation, the electronic device may set an operating mode of the camera module to an image capturing mode or a video capturing mode, where in the image capturing mode, an image may be directly captured, and in the video capturing mode, the image may be a video image in captured video data.
Optionally, in this embodiment of the application, according to the position relationship of the plurality of users in the image, if the number of users is small, the object positions are concentrated, the number of users is smaller than the preset number of users, and the position relationship indicates that the plurality of users are distributed in the preset range of the central area of the image, the camera of the camera module may be adjusted to reduce the image; if the position relations of the users represent that the users are distributed at the edge of the image, the camera of the camera module can be adjusted to amplify the image, so that the camera module can be flexibly adjusted according to the position relations of the users in the image, the image can be shot to the full as much as possible, and people are prevented from being missed.
102. And determining a target starting gesture closest to the central area of the image according to the image.
The starting gesture refers to a preset unlocking gesture used for triggering the business logic in any business logic, and the target starting gesture refers to a starting gesture of a target user.
The preset unlocking gesture can be a preset static gesture or a preset dynamic gesture, and the preset unlocking gesture can include at least one of the following gestures: as shown in fig. 1D, fig. 1D is a schematic diagram of a preset unlocking gesture provided in an embodiment of the present application, where the preset unlocking gesture may be at least one of the following gestures or a combination of two or more of the following gestures: a gesture in which the index finger extends upward and the thumb, a gesture in which the thumb, the index finger and the middle finger extend, a gesture in which the thumb, the index finger, the middle finger and the ring finger extend, a gesture in which the index finger and the middle finger extend, a gesture in which the thumb and the index finger bend to connect the palms outward, a gesture in which the thumb sticks up and the index finger bends to connect the palms outward, a gesture in which the thumb sticks up, a gesture in which the thumb and the index finger bend to connect the palms to each other to form a hollow fist, a gesture in which the index finger extends laterally, a gesture in which the index finger extends downward, a gesture in which the thumb and the index finger bend to connect the palms inward, a gesture in which the thumb and the ring finger extend, a gesture in which the index finger, the index finger and the ring finger extend upward, a five-finger gesture extending downward, a thumb-holding gesture extending downward, a middle-finger holding gesture extending downward, and the like, which are not limited by the embodiments of the present application.
In the embodiment of the application, in a multi-user scene, there may be a plurality of users executing gesture operations, some gestures are start gestures for triggering the service logic, some gestures do not belong to the start gestures for triggering the service logic, and some people may not execute gesture operations, so that a target user among the plurality of users can be identified, the target user is determined as a user for controlling the service logic, and the service logic is triggered according to the target start gesture of the target user.
Optionally, in step 102, determining, from the image, a target start gesture closest to the central region of the image may include:
1021. performing gesture recognition on the image to obtain at least one starting gesture for triggering the service logic;
1022. and if the at least one starting gesture comes from more than two users, taking the starting gesture which is closest to the central area of the image in the at least one starting gesture as a target starting gesture.
In specific implementation, in a multi-user scene, one or more users may execute a start gesture, so that the image can be subjected to gesture recognition to obtain at least one start gesture for triggering a service logic, and if only one user executes the start gesture, the start gesture is determined as a target start gesture; if at least one starting gesture comes from more than two users, the starting gesture closest to the central area of the image can be determined as the target starting gesture by adopting a center position principle.
Specifically, determining a starting gesture closest to the central region of the image in the at least one starting gesture may specifically include: determining a center point coordinate of a rectangular frame corresponding to each starting gesture in at least one starting gesture to obtain at least one center point coordinate; determining the absolute value of the difference between each central point coordinate in the at least one central point coordinate and the 1/2 image width to obtain at least one absolute value of the difference; and determining the starting gesture corresponding to the minimum difference absolute value in the difference absolute values as a target starting gesture.
For example, assuming that the width of the image is w, subtracting w/2 from the coordinate x of the central point of the rectangular frame corresponding to each starting gesture to obtain a difference absolute value, taking the difference absolute value as a priority attribute of each starting gesture, wherein the smaller the difference absolute value is, the higher the priority is, when determining the target starting gesture, all recognized starting gestures can be traversed once, and the starting gesture with the highest priority is selected as the target starting gesture.
Optionally, in step 1021, performing gesture recognition on the image to obtain at least one starting gesture for triggering the service logic, where the gesture recognition may include:
2101. performing gesture recognition on the image to obtain at least one reference gesture;
2102. and matching the at least one reference gesture with a preset unlocking gesture for triggering the service logic to obtain at least one successfully matched reference gesture, and taking the at least one successfully matched reference gesture as the at least one starting gesture.
In specific implementation, a gesture in an image may be recognized first to obtain at least one reference gesture, where the gesture may be a static gesture and may be a dynamic gesture, and the embodiment of the present application is not limited to this, and then, the recognized reference gesture is matched with a preset unlocking gesture to obtain at least one reference gesture successfully matched with the preset unlocking gesture, and the at least one reference gesture successfully matched with the preset unlocking gesture is used as at least one start gesture.
For example, please refer to fig. 1E, where fig. 1E is a scene schematic diagram illustrating a target start gesture for identifying a target user in an embodiment of the present application, where a multi-user scene may include a user a, a user B, a user C, a user D, and a user E, and a gesture recognition may be performed on an image to obtain reference gestures corresponding to the user a, the user B, and the user D, respectively, to obtain a plurality of reference gestures, and then the plurality of reference gestures are matched with a preset unlock gesture for triggering a service logic to obtain at least one reference gesture that is successfully matched, and if the preset unlock gesture for triggering the service logic is a palm, the reference gesture that is successfully matched may be a start gesture corresponding to the user B and the user D, respectively; furthermore, the starting gesture closest to the central area of the image between the user B and the user D can be determined as a target starting gesture, that is, the starting gesture of the target user B is taken as a target starting gesture.
103. And triggering a control mode of the business logic according to the target starting gesture.
In a specific implementation, the service logic may be a service logic on an electronic device, for example, the electronic device may be a projection display device, and the service logic may be a service logic related to projection display, and a control mode of the service logic may be triggered according to a target start gesture determined by determining the target start gesture closest to a central area of an image, so as to perform gesture control on the projection display device in a multi-user scene.
Optionally, after the step 103, the following steps may be further included:
1031. carrying out face tracking on a target user corresponding to the target starting gesture;
1032. when the face of the target user is tracked, acquiring behavior information of the target user;
1033. and if the behavior information meets a preset condition, controlling the service logic according to the behavior information.
Wherein, the behavior information may include at least one of the following: the duration of the target start gesture, the location information of the target user, the location information of the target start gesture, the gesture change information of the target user, and the like, where the target user refers to a user who executes the target start gesture, and is not limited herein.
In specific implementation, the target user can be tracked, and behavior changes of the target user, such as position changes of the target user and gesture changes of the target user, can be identified, so that the service logic is controlled according to the behavior information of the target user.
Optionally, a face recognition technology can be adopted to track the target user, specifically, an image can be collected through the camera module, the face of the target user is recognized according to the image, and after the face is recognized, the business logic is controlled according to behavior information such as position change, behavior change and the like of the target user.
Optionally, in step 1032 above, the method further includes:
if the behavior information meets at least one of the following conditions, determining that the behavior information meets a preset condition:
the duration of the target starting gesture is longer than a first preset duration;
the position information of the target user and/or the position information of the target starting gesture are not changed within a second preset time length;
the gesture change information of the target user represents that the gesture of the target user changes according to a preset sequence.
In specific implementation, if the duration of the target starting gesture is longer than a first preset duration, which indicates that the target starting gesture of the target user lasts for a period of time, the implementation of the service logic can be controlled according to the target starting gesture; if the position information of the target user and/or the position information of the target starting gesture does not change within a second preset time length, the target user is indicated not to move within a period of time, or the target user does not move after the target user continues to move for a period of time, the implementation of the service logic can be controlled according to the target starting gesture; if the gesture of the target user changes and the gesture changes continuously according to the preset sequence, the fact that the user executes the gesture operation for controlling the specific service logic is indicated, and the implementation of the service logic can be controlled according to the gesture change information.
Optionally, in an embodiment of the present application, the method further includes:
if the face of the target user is not detected, acquiring the shape information of the target user;
and tracking the target user according to the appearance information.
Wherein, the shape information may include at least one of the following: body shape, clothing, hairstyle, head shape, etc.
In the embodiment of the application, if valid face information of a target user is not detected, for example, when the target user wears a mask, sunglasses, a mask, or the like, it is difficult to continuously track the face, the target user may be distinguished from other users of the plurality of users according to information such as a body shape, a clothing, a hairstyle, a head shape, and the like, and the target user is tracked, and when the target user is tracked, if the behavior information meets a preset condition, the service logic may be controlled according to the behavior information.
Optionally, after the step 103, the method further includes:
re-recognizing a new target start gesture when at least one of the following conditions is met;
the target user is not identified within a third preset time length;
the gesture of the target user is not recognized within a fourth preset time length;
the business logic is complete.
The third preset duration and the fourth preset duration may be set by default in the system or may be set by the user, which is not limited in the embodiment of the present application, and the third preset duration may be the same as the fourth preset duration or different from the fourth preset duration, for example, the third preset duration may be longer than the fourth preset duration, or the third preset duration may be shorter than the preset duration.
The new target starting gesture may be a starting gesture of the same user as the previous user, or may be a starting gesture of a different user from the previous user, which is not limited in the present application.
In the embodiment of the application, if the target user is not identified within the third preset time, it is indicated that the target user is lost, and specifically, if the face tracking loss or the shape tracking loss time of the target user exceeds the third preset time, a new target starting gesture of the target user can be re-identified; if the gesture of the target user is not recognized within the fourth preset time, indicating that the gesture tracking is lost, the target starting gesture of a new target user can be recognized again; if the service logic is finished, a new target starting gesture can be identified again, and the next round of service logic control is carried out.
For example, if the third preset time is longer than the fourth preset time, the gesture of the target user is not recognized within the fourth preset time, the target user may stop the gesture operation within the fourth preset time, the target user is not recognized within the third preset time, and the target user may leave the tracking picture within the third preset time; if the third preset time is shorter than the fourth preset time, the target user is not recognized within the third preset time, and the target user may walk out of the tracking picture, but the gesture is still in the tracking picture, the gesture of the target user is not recognized, and the gesture of the target user may leave the tracking picture.
Therefore, in the embodiment of the application, the behavior change of the target user is analyzed by tracking the target user, so that the service logic is controlled according to the behavior information of the target user.
It can be seen that, the gesture recognition control method described in the embodiment of the present application obtains an image including a plurality of users; determining a target starting gesture which is closest to the central area of the image according to the image; the control mode of the business logic is triggered according to the target starting gesture, so that the control mode of the business logic can be triggered according to the target starting gesture closest to the central area of the image in a multi-person scene, and gesture control can be rapidly and accurately carried out in the multi-person scene.
Referring to fig. 2, fig. 2 is a schematic flow chart of another gesture recognition control method according to an embodiment of the present application, consistent with the embodiment shown in fig. 1C, where the gesture recognition control method includes:
201. an image containing a plurality of users is acquired.
202. And performing gesture recognition on the image to obtain at least one starting gesture for triggering the service logic.
203. And if the at least one starting gesture comes from more than two users, taking the starting gesture which is closest to the central area of the image in the at least one starting gesture as a target starting gesture.
204. And triggering a control mode of the business logic according to the target starting gesture.
205. And carrying out face tracking on the target user corresponding to the target starting gesture.
206. And when the face of the target user is tracked, acquiring the behavior information of the target user.
207. And if the behavior information meets a preset condition, controlling the service logic according to the behavior information.
For the detailed description of the steps 201 to 207, reference may be made to the corresponding steps of the gesture recognition control method described in fig. 1C, and details are not repeated here.
It can be seen that, the gesture recognition control method described in the embodiment of the present application obtains an image including a plurality of users; performing gesture recognition on the image to obtain at least one starting gesture for triggering the service logic, if the at least one starting gesture comes from more than two users, taking the starting gesture which is closest to the central area of the image in the at least one starting gesture as a target starting gesture, triggering the control mode of the service logic according to the target starting gesture, performing face tracking on a target user corresponding to the target starting gesture, acquiring behavior information of the target user when the face of the target user is tracked, and if the behavior information meets a preset condition, the business logic is controlled according to the behavior information, so that the target user closest to the central area of the image can be determined in a multi-user scene, the face or the shape of the target user is tracked, and the continuous control right of the target user to the service logic is kept, so that the problem that the gesture control right is mutually robbed in a multi-user scene is solved.
Referring to fig. 3 in keeping with the above embodiments, fig. 3 is a schematic structural diagram of another electronic device provided in an embodiment of the present application, and as shown in the drawing, the electronic device includes a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, and in an embodiment of the present application, the programs include instructions for performing the following steps:
acquiring an image containing a plurality of users;
determining a target starting gesture closest to a central area of the image according to the image;
and triggering a control mode of the business logic according to the target starting gesture.
In one possible example, in connection with the determining from the image a target-initiating gesture that is closest to a center region of the image, the program includes instructions for:
performing gesture recognition on the image to obtain at least one starting gesture for triggering the service logic;
and if the at least one starting gesture comes from more than two users, taking the starting gesture which is closest to the central area of the image in the at least one starting gesture as a target starting gesture.
In one possible example, in the performing gesture recognition on the image to obtain at least one starting gesture for triggering the service logic, the program further includes instructions for:
performing gesture recognition on the image to obtain at least one reference gesture;
and matching the at least one reference gesture with a preset unlocking gesture for triggering the service logic to obtain at least one successfully matched reference gesture, and taking the at least one successfully matched reference gesture as the at least one starting gesture.
In one possible example, after said triggering a control mode of the business logic according to said target initiation gesture, the above program further comprises instructions for:
carrying out face tracking on a target user corresponding to the target starting gesture;
when the face of the target user is tracked, acquiring behavior information of the target user;
and if the behavior information meets a preset condition, controlling the service logic according to the behavior information.
In one possible example, the program further includes instructions for performing the steps of:
if the face of the target user is not detected, acquiring the shape information of the target user;
and tracking the target user according to the appearance information.
In one possible example, the behavior information includes at least one of: the duration of the target initiation gesture, the location information of the target user, the location information of the target initiation gesture, and the gesture change information of the target user, the program further comprising instructions for performing the steps of:
if the behavior information meets at least one of the following conditions, determining that the behavior information meets a preset condition:
the duration of the target starting gesture is longer than a first preset duration;
the position information of the target user and/or the position information of the target starting gesture are not changed within a second preset time length;
the gesture change information of the target user represents that the gesture of the target user changes according to a preset sequence.
In one possible example, the program further includes instructions for performing the steps of:
re-recognizing a new target start gesture when at least one of the following conditions is met;
the target user is not identified within a third preset time length;
the gesture of the target user is not recognized within a fourth preset time length;
the business logic is complete.
The above description has introduced the solution of the embodiment of the present application mainly from the perspective of the method-side implementation process. It is understood that the electronic device comprises corresponding hardware structures and/or software modules for performing the respective functions in order to realize the above-mentioned functions. Those of skill in the art will readily appreciate that the present application is capable of hardware or a combination of hardware and computer software implementing the various illustrative elements and algorithm steps described in connection with the embodiments provided herein. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiment of the present application, the electronic device may be divided into the functional units according to the method example, for example, each functional unit may be divided corresponding to each function, or two or more functions may be integrated into one processing unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit. It should be noted that the division of the unit in the embodiment of the present application is schematic, and is only a logic function division, and there may be another division manner in actual implementation.
Fig. 4 is a block diagram showing functional units of a gesture recognition control device 400 according to an embodiment of the present application. The gesture recognition control device 400, the gesture recognition control device 400 includes: a processing unit 402 and a communication unit 403. The processing unit 402 is used for controlling and managing the operation of the terminal. For example, the processing unit 402 is used to support the terminal to execute some steps in fig. 4 and other processes for the technical solutions described in this application. The communication unit 403 is used to support communication between the terminal and other devices in the wireless communication system. The wireless communication apparatus 400 may further include a storage unit 401 for storing program codes and data of the terminal.
The processing unit 402 may be a processor or a controller, and may be, for example, a Central Processing Unit (CPU), a general purpose processor, a Digital Signal Processor (DSP), an application-specific integrated circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. Processing unit 402 may also be a combination that performs computing functions, e.g., comprising one or more microprocessors, a combination of DSPs and microprocessors, and the like. The communication unit 403 may be a communication interface, a transceiver, a transceiving circuit, etc., and the storage unit 401 may be a memory. When the processing unit 402 is a processor, the communication unit 403 is a communication interface, and the storage unit 401 is a memory, the gesture recognition control apparatus 400 according to the embodiment of the present application may be an electronic device shown in fig. 3.
In a specific implementation, the processing unit 402 is configured to perform any one of the steps performed by the terminal in the above method embodiment, and when performing data transmission such as sending, optionally invokes the communication unit 403 to complete the corresponding operation. The details will be described below.
A processing unit 402 for acquiring an image containing a plurality of users;
the processing unit 402 is further configured to determine, according to the image, a target start gesture closest to a central region of the image;
the processing unit 402 is further configured to trigger a control mode of a service logic according to the target start gesture.
In one possible example, in terms of determining from the image a target start gesture that is closest to a center region of the image, the processing unit 402 is specifically configured to:
performing gesture recognition on the image to obtain at least one starting gesture for triggering the service logic;
and if the at least one starting gesture comes from more than two users, taking the starting gesture which is closest to the central area of the image in the at least one starting gesture as a target starting gesture.
In a possible example, in terms of performing gesture recognition on the image to obtain at least one starting gesture for triggering the service logic, the processing unit 402 is specifically configured to:
performing gesture recognition on the image to obtain at least one reference gesture;
and matching the at least one reference gesture with a preset unlocking gesture for triggering the service logic to obtain at least one successfully matched reference gesture, and taking the at least one successfully matched reference gesture as the at least one starting gesture.
In one possible example, after the triggering of the control mode of the business logic according to the target initiation gesture, the processing unit 402 is further configured to:
carrying out face tracking on a target user corresponding to the target starting gesture;
when the face of the target user is tracked, acquiring behavior information of the target user;
and if the behavior information meets a preset condition, controlling the service logic according to the behavior information.
In one possible example, the processing unit 402 is further configured to:
if the face of the target user is not detected, acquiring the shape information of the target user;
and tracking the target user according to the appearance information.
In one possible example, the behavior information includes at least one of: the duration of the target start gesture, the location information of the target user, the location information of the target start gesture, and the gesture change information of the target user, the processing unit 402 is further configured to:
if the behavior information meets at least one of the following conditions, determining that the behavior information meets a preset condition:
the duration of the target starting gesture is longer than a first preset duration;
the position information of the target user and/or the position information of the target starting gesture are not changed within a second preset time length;
the gesture change information of the target user represents that the gesture of the target user changes according to a preset sequence.
In one possible example, the processing unit 402 is further configured to:
re-recognizing a new target start gesture when at least one of the following conditions is met;
the target user is not identified within a third preset time length;
the gesture of the target user is not recognized within a fourth preset time length;
the business logic is complete.
It can be seen that, the gesture recognition control device described in the embodiment of the present application obtains an image including a plurality of users; determining a target starting gesture which is closest to the central area of the image according to the image; the control mode of the business logic is triggered according to the target starting gesture, so that the control mode of the business logic can be triggered according to the target starting gesture closest to the central area of the image in a multi-person scene, and gesture control can be rapidly and accurately carried out in the multi-person scene.
It can be understood that the functions of each program module of the gesture recognition control device in this embodiment may be specifically implemented according to the method in the foregoing method embodiment, and the specific implementation process may refer to the related description of the foregoing method embodiment, which is not described herein again.
Embodiments of the present application also provide a computer storage medium, where the computer storage medium stores a computer program for electronic data exchange, the computer program enabling a computer to execute part or all of the steps of any one of the methods described in the above method embodiments, and the computer includes an electronic device.
Embodiments of the present application also provide a computer program product comprising a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods as described in the above method embodiments. The computer program product may be a software installation package, the computer comprising an electronic device.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the above-described division of the units is only one type of division of logical functions, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of some interfaces, devices or units, and may be an electric or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit may be stored in a computer readable memory if it is implemented in the form of a software functional unit and sold or used as a stand-alone product. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a memory, and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the above-mentioned method of the embodiments of the present application. And the aforementioned memory comprises: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable memory, which may include: flash Memory disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
The foregoing detailed description of the embodiments of the present application has been presented to illustrate the principles and implementations of the present application, and the above description of the embodiments is only provided to help understand the method and the core concept of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (10)
1. A gesture recognition control method, characterized in that the method comprises:
acquiring an image containing a plurality of users;
determining a target starting gesture closest to a central area of the image according to the image;
and triggering a control mode of the business logic according to the target starting gesture.
2. The method of claim 1, wherein determining from the image a target-initiating gesture closest to a center region of the image comprises:
performing gesture recognition on the image to obtain at least one starting gesture for triggering the service logic;
and if the at least one starting gesture comes from more than two users, taking the starting gesture which is closest to the central area of the image in the at least one starting gesture as a target starting gesture.
3. The method of claim 2, wherein the gesture recognition of the image resulting in at least one initiation gesture that triggers business logic comprises:
performing gesture recognition on the image to obtain at least one reference gesture;
and matching the at least one reference gesture with a preset unlocking gesture for triggering the service logic to obtain at least one successfully matched reference gesture, and taking the at least one successfully matched reference gesture as the at least one starting gesture.
4. The method according to any of claims 1-3, wherein after the triggering of the control mode of the business logic according to the target initiation gesture, the method further comprises:
carrying out face tracking on a target user corresponding to the target starting gesture;
when the face of the target user is tracked, acquiring behavior information of the target user;
and if the behavior information meets a preset condition, controlling the service logic according to the behavior information.
5. The method of claim 4, further comprising:
if the face of the target user is not detected, acquiring the shape information of the target user;
and tracking the target user according to the appearance information.
6. The method of claim 4, wherein the behavior information comprises at least one of: the duration of the target start gesture, the location information of the target user, the location information of the target start gesture, and the gesture change information of the target user, the method further comprising:
if the behavior information meets at least one of the following conditions, determining that the behavior information meets a preset condition:
the duration of the target starting gesture is longer than a first preset duration;
the position information of the target user and/or the position information of the target starting gesture are not changed within a second preset time length;
the gesture change information of the target user represents that the gesture of the target user changes according to a preset sequence.
7. The method of claim 1, 5 or 6, further comprising:
re-recognizing a new target start gesture when at least one of the following conditions is met;
the target user is not identified within a third preset time length;
the gesture of the target user is not recognized within a fourth preset time length;
the business logic is complete.
8. A gesture recognition control apparatus, the apparatus comprising:
a processing unit for acquiring an image containing a plurality of users;
the processing unit is further used for determining a target starting gesture which is closest to the central area of the image according to the image;
and the processing unit is also used for triggering a control mode of the business logic according to the target starting gesture.
9. An electronic device comprising a processor, a memory for storing one or more programs and configured for execution by the processor, the programs comprising instructions for performing the steps of the method of any of claims 1-7.
10. A computer-readable storage medium, characterized in that a computer program for electronic data exchange is stored, wherein the computer program causes a computer to perform the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011635321.0A CN112686169A (en) | 2020-12-31 | 2020-12-31 | Gesture recognition control method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011635321.0A CN112686169A (en) | 2020-12-31 | 2020-12-31 | Gesture recognition control method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112686169A true CN112686169A (en) | 2021-04-20 |
Family
ID=75456338
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011635321.0A Pending CN112686169A (en) | 2020-12-31 | 2020-12-31 | Gesture recognition control method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112686169A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113342170A (en) * | 2021-06-11 | 2021-09-03 | 北京字节跳动网络技术有限公司 | Gesture control method, device, terminal and storage medium |
| CN113791548A (en) * | 2021-09-26 | 2021-12-14 | 北京市商汤科技开发有限公司 | Device control method, device, electronic device and storage medium |
| CN114374791A (en) * | 2021-11-25 | 2022-04-19 | 慧之安信息技术股份有限公司 | Camera adjusting method and system based on gesture recognition |
| CN114615429A (en) * | 2022-03-03 | 2022-06-10 | 深圳市明日实业有限责任公司 | Gesture recognition method, gesture recognition system, shooting terminal, shooting system and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130222232A1 (en) * | 2012-02-24 | 2013-08-29 | Pantech Co., Ltd. | Gesture recognition device and method thereof |
| CN105354551A (en) * | 2015-11-03 | 2016-02-24 | 北京英梅吉科技有限公司 | Gesture recognition method based on monocular camera |
| CN107422859A (en) * | 2017-07-26 | 2017-12-01 | 广东美的制冷设备有限公司 | Regulation and control method, apparatus and computer-readable recording medium and air-conditioning based on gesture |
| US20180293433A1 (en) * | 2015-06-30 | 2018-10-11 | Yutou Technology (Hangzhou) Co., Ltd. | Gesture detection and recognition method and system |
| CN109872160A (en) * | 2019-02-01 | 2019-06-11 | 广州逗号智能科技有限公司 | Electric paying method and device |
| CN109977906A (en) * | 2019-04-04 | 2019-07-05 | 睿魔智能科技（深圳）有限公司 | Gesture identification method and system, computer equipment and storage medium |
| CN110321001A (en) * | 2019-05-09 | 2019-10-11 | 江苏紫米软件技术有限公司 | A kind of wireless charging bracket and face tracking methods |
| CN111556350A (en) * | 2020-04-21 | 2020-08-18 | 海信集团有限公司 | Intelligent terminal and man-machine interaction method |
| CN111580652A (en) * | 2020-05-06 | 2020-08-25 | Oppo广东移动通信有限公司 | Control method and device for video playing, augmented reality equipment and storage medium |
-
2020
- 2020-12-31 CN CN202011635321.0A patent/CN112686169A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130222232A1 (en) * | 2012-02-24 | 2013-08-29 | Pantech Co., Ltd. | Gesture recognition device and method thereof |
| US20180293433A1 (en) * | 2015-06-30 | 2018-10-11 | Yutou Technology (Hangzhou) Co., Ltd. | Gesture detection and recognition method and system |
| CN105354551A (en) * | 2015-11-03 | 2016-02-24 | 北京英梅吉科技有限公司 | Gesture recognition method based on monocular camera |
| CN107422859A (en) * | 2017-07-26 | 2017-12-01 | 广东美的制冷设备有限公司 | Regulation and control method, apparatus and computer-readable recording medium and air-conditioning based on gesture |
| CN109872160A (en) * | 2019-02-01 | 2019-06-11 | 广州逗号智能科技有限公司 | Electric paying method and device |
| CN109977906A (en) * | 2019-04-04 | 2019-07-05 | 睿魔智能科技（深圳）有限公司 | Gesture identification method and system, computer equipment and storage medium |
| CN110321001A (en) * | 2019-05-09 | 2019-10-11 | 江苏紫米软件技术有限公司 | A kind of wireless charging bracket and face tracking methods |
| CN111556350A (en) * | 2020-04-21 | 2020-08-18 | 海信集团有限公司 | Intelligent terminal and man-machine interaction method |
| CN111580652A (en) * | 2020-05-06 | 2020-08-25 | Oppo广东移动通信有限公司 | Control method and device for video playing, augmented reality equipment and storage medium |
Non-Patent Citations (4)
| Title |
|---|
| LIN ZEYI 等: "Image-based Pose Representation for Action Recognition and Hand Gesture Recognition", 《2020 15TH IEEE INTERNATIONAL CONFERENCE ON AUTOMATIC FACE AND GESTURE RECOGNITION (FG 2020)》, 20 November 2020 (2020-11-20), pages 532 - 539, XP033879890, DOI: 10.1109/FG47880.2020.00066 * |
| 伍杰: "基于视觉的实时手势识别方法研究", 《中国优秀硕士学位论文全文数据库 (信息科技辑)》, no. 02, 15 February 2020 (2020-02-15), pages 138 - 1858 * |
| 张莹莹: "基于Kinect的大屏幕手势互动***研究与实现", 《中国优秀硕士学位论文全文数据库 (信息科技辑)》, no. 08, 15 August 2017 (2017-08-15), pages 138 - 306 * |
| 王艳 等: "基于特征距离加权的手势识别", 《计算机科学》, vol. 44, no. 1, 15 June 2017 (2017-06-15), pages 220 - 223 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113342170A (en) * | 2021-06-11 | 2021-09-03 | 北京字节跳动网络技术有限公司 | Gesture control method, device, terminal and storage medium |
| CN113791548A (en) * | 2021-09-26 | 2021-12-14 | 北京市商汤科技开发有限公司 | Device control method, device, electronic device and storage medium |
| CN114374791A (en) * | 2021-11-25 | 2022-04-19 | 慧之安信息技术股份有限公司 | Camera adjusting method and system based on gesture recognition |
| CN114615429A (en) * | 2022-03-03 | 2022-06-10 | 深圳市明日实业有限责任公司 | Gesture recognition method, gesture recognition system, shooting terminal, shooting system and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112686169A (en) | Gesture recognition control method and device, electronic equipment and storage medium | |
| CN110913132B (en) | Object tracking method and electronic equipment | |
| CN108495029B (en) | Photographing method and mobile terminal | |
| CN108989678B (en) | Image processing method and mobile terminal | |
| CN109499061B (en) | Game scene picture adjusting method and device, mobile terminal and storage medium | |
| CN107977652B (en) | Method for extracting screen display content and mobile terminal | |
| CN109005336B (en) | Image shooting method and terminal equipment | |
| CN107784232B (en) | Picture processing method and mobile terminal | |
| CN109240577B (en) | Screen capturing method and terminal | |
| CN111263071B (en) | Shooting method and electronic equipment | |
| CN107241552B (en) | Image acquisition method, device, storage medium and terminal | |
| EP3208742B1 (en) | Method and apparatus for detecting pressure | |
| WO2020048392A1 (en) | Application virus detection method, apparatus, computer device, and storage medium | |
| CN109558000B (en) | Man-machine interaction method and electronic equipment | |
| CN109495616B (en) | Photographing method and terminal equipment | |
| CN109618218B (en) | Video processing method and mobile terminal | |
| CN108257104B (en) | Image processing method and mobile terminal | |
| CN111031234B (en) | Image processing method and electronic equipment | |
| CN109815462B (en) | Text generation method and terminal equipment | |
| US11290591B2 (en) | Dialing method and mobile terminal | |
| WO2021104266A1 (en) | Object display method and electronic device | |
| KR20220157485A (en) | Detection result output method, electronic device and medium | |
| CN111031178A (en) | Video stream clipping method and electronic equipment | |
| CN109669710B (en) | Note processing method and terminal | |
| CN108833791B (en) | Shooting method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |