WO2019062682A1 - 手势识别方法及电子设备 - Google Patents

手势识别方法及电子设备 Download PDF

Info

Publication number
WO2019062682A1
WO2019062682A1 PCT/CN2018/107214 CN2018107214W WO2019062682A1 WO 2019062682 A1 WO2019062682 A1 WO 2019062682A1 CN 2018107214 W CN2018107214 W CN 2018107214W WO 2019062682 A1 WO2019062682 A1 WO 2019062682A1
Authority
WO
WIPO (PCT)
Prior art keywords
dimensional image
interval
electronic device
preset size
target
Prior art date
Application number
PCT/CN2018/107214
Other languages
English (en)
French (fr)
Inventor
李英杰
Original Assignee
京东方科技集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司 filed Critical 京东方科技集团股份有限公司
Priority to EP18861362.4A priority Critical patent/EP3690605A4/en
Priority to US16/340,497 priority patent/US10866649B2/en
Publication of WO2019062682A1 publication Critical patent/WO2019062682A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/0304—Detection arrangements using opto-electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm
    • G06V40/113—Recognition of static hand signs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language

Definitions

  • the present disclosure relates to the field of communications technologies, and in particular, to a gesture recognition method and an electronic device.
  • the electronic device and the user can interact in various ways, such as using a remote controller, a mouse, a voice or a gesture. Interact and so on. Since the process of gesture interaction is simpler and the user interacts with the electronic device naturally, the way of using gestures to interact with the electronic device has been applied to many scenarios.
  • the depth camera is often used to achieve target tracking and gesture recognition.
  • the current high-precision depth cameras are very expensive, such as the Mesa Imaging SwissRanger 4000 (SR4000), which costs tens of thousands of dollars.
  • a common two-dimensional image acquisition device is used to implement gesture recognition in many scenarios, but the method of implementing gesture recognition based on the two-dimensional image acquisition device in the related art has a problem of low recognition accuracy.
  • the user uses gestures for human-computer interaction, since there is no actual plane as a support, the arm is easy to shake. After the jitter is collected, the gesture recognition by the processor will result in an erroneous recognition result.
  • an embodiment of the present disclosure provides a gesture recognition method, including:
  • a corresponding instruction is determined and executed using the at least one target two-dimensional image.
  • an embodiment of the present disclosure further provides an electronic device, including:
  • An acquiring module configured to process each of the plurality of two-dimensional images including the pointing object, and obtain an imaging size of the pointing object in each of the two-dimensional images
  • a first determining module configured to determine at least one target two-dimensional image from the plurality of two-dimensional images according to an imaging size of the pointing object in each two-dimensional image
  • an execution module configured to determine and execute a corresponding instruction by using the at least one target two-dimensional image.
  • an embodiment of the present disclosure further provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the computer program The steps in the above gesture recognition method are implemented.
  • an embodiment of the present disclosure further provides a computer readable storage medium, where the computer readable storage medium stores a computer program, and when the computer program is executed by the processor, implements the steps in the gesture recognition method.
  • FIG. 1 is a schematic diagram of an image collection distance provided by some embodiments of the present disclosure.
  • FIG. 2 is a schematic diagram of an image display provided by some embodiments of the present disclosure.
  • FIG. 3 is a second schematic diagram of an image collection distance provided by some embodiments of the present disclosure.
  • FIG. 4 is a second schematic diagram of an image display provided by some embodiments of the present disclosure.
  • FIG. 5 is a flowchart of a gesture recognition method according to some embodiments of the present disclosure.
  • FIG. 6 is a flowchart of a gesture recognition method provided by some embodiments of the present disclosure.
  • FIG. 7 is a schematic diagram of a connected area provided by some embodiments of the present disclosure.
  • FIG. 8 is a second schematic diagram of a connected area provided by some embodiments of the present disclosure.
  • FIG. 9 is a schematic diagram of different operational regions provided by some embodiments of the present disclosure.
  • FIG. 10 is a structural diagram of an electronic device according to some embodiments of the present disclosure.
  • FIG. 11 is a structural diagram of a first acquiring module of an electronic device according to some embodiments of the present disclosure.
  • FIG. 12 is a second structural diagram of an electronic device according to some embodiments of the present disclosure.
  • FIG. 13 is a third structural diagram of an electronic device according to some embodiments of the present disclosure.
  • FIG. 14 is a structural diagram of an electronic device according to some embodiments of the present disclosure.
  • the imaging size of the pointing object in the image and the distance between the pointing object and the image capture device are closely related. The following is explained below in conjunction with Figs. 1 to 4.
  • the imaging size refers to the size of the object in the image.
  • the imaging size of the pointing object in the image is the size of the pointing object in the image.
  • the pointing object shown in Figures 1 - 4 is a spheroid.
  • the image acquisition device is relatively close to the spheroid, and the imaging size of the spheroid in the image acquired by the image acquisition device is as shown in Fig. 2.
  • the imaging size of the spheroid in the image acquired by the image acquisition device is relatively large.
  • FIG. 4 Comparing Fig. 4 with Fig. 2, it can be found that the imaging size of the spheroid in Fig. 4 in the image acquired by the image acquisition device is relatively small. That is, the imaging size can indirectly describe the distance between the pointing object and the image capture device.
  • an operation area is set for the user, and only the user operation in the operation area is recognized.
  • the imaging size can indirectly describe the distance between the pointing object and the image capturing device. Therefore, in the specific embodiment of the present disclosure, the operations performed outside the operating region can be excluded according to the imaging size of the pointing object, and only the identification is performed. The operation performed by the user in the operation area, thereby improving the recognition accuracy.
  • FIG. 5 is a flowchart of a gesture recognition method according to some embodiments of the present disclosure. As shown in FIG. 5, the gesture recognition method includes the following steps 101 to 103.
  • Step 101 Process each of the plurality of two-dimensional images including the pointing object, and obtain an imaging size of the pointing object in each of the two-dimensional images.
  • the pointing object may be a user's finger, the user's palm, or may be an object that the user can hold (for example, a long object), or may be an object attached to the user's finger (such as a reflective film or a sheet-shaped reflector having a specific shape) and the like.
  • Step 102 Determine at least one target two-dimensional image from the plurality of two-dimensional images according to an imaging size of the pointing object in each two-dimensional image.
  • the embodiment of the present disclosure sets an operation area for the user from the perspective of the user, so the user operation can be divided into two parts: an operation within the operation area and an operation outside the operation area.
  • the target two-dimensional image is a two-dimensional image when the imaging size of the pointing object in the two-dimensional image is within a preset size interval.
  • only the operation of the user in the operation area is recognized, and in some operations outside the operation area, the electronic device does not respond. It can be known from the foregoing principle that determining whether the user operates in the operation area can be determined according to the imaging size of the pointing object in each two-dimensional image. When the imaging size of the pointing object in the two-dimensional image is within a preset size interval, it can be determined that the user is operating within the operating area.
  • Step 103 Determine and execute a corresponding instruction by using the at least one target two-dimensional image.
  • the image sequence formed by the target two-dimensional image may be used to perform control of the device.
  • the trajectory of the pointing object is determined according to the plurality of target two-dimensional images, and an instruction matching the trajectory is determined.
  • the imaging size can indirectly describe the distance between the pointing object and the image capturing device, that is, when the displacement between the pointing object and the image capturing device changes, the pointing object is imaged in the image acquired by the image capturing device.
  • the size will also change. Therefore, in a specific embodiment of the present disclosure, the imaging size of the pointing object in the two-dimensional image collected by the ordinary two-dimensional image capturing device can recognize the movement of the pointing object in the direction toward the image capturing device.
  • the imaging size of the pointing object in the two-dimensional image collected by the ordinary two-dimensional image capturing device can also determine whether the user operation is performed in a set space, that is, it can exclude those that are performed outside the operating area. It will lead to misidentification operation and improve the recognition accuracy of the method based on the two-dimensional image acquisition device to realize gesture recognition.
  • the recognition accuracy can be improved due to the existence of the operation area and only the operations occurring within the operation area are identified.
  • the user due to the existence of the operation area, the user can lift the hand, leave the set area, and then enter the set area at the midpoint of the horizontal line segment, since the user's finger returns from the end point of the horizontal line segment.
  • the action to the midpoint of the horizontal line segment is performed outside the operating area, so it is not recognized, and misidentification is avoided.
  • the way to avoid misrecognition only requires the user to change the movement route of the finger, and does not require the user to perform extra actions, and the implementation process is simple.
  • the electronic device may be a mobile phone, a tablet personal computer, a laptop computer, a personal digital assistant (PDA), or a mobile Internet device (Mobile Internet Device, MID) or wearable device (Wearable Device) and so on.
  • PDA personal digital assistant
  • Mobile Internet Device MID
  • Wearable Device Wearable Device
  • At least one target two-dimensional image needs to be determined from the plurality of two-dimensional images according to an imaging size of the pointing object in each two-dimensional image. From an operational point of view, it is necessary to select those operations that occur in the operating area in accordance with the imaging size of the pointing object in each two-dimensional image.
  • the target two-dimensional image may be determined according to various manners, and one of the following manners is specifically described as follows.
  • determining whether the user operates in the operation area can be determined according to the imaging size of the pointing object in each two-dimensional image. In the direction toward the image capturing device, it is determined whether the user operates in the operation region, and the determination may be made according to whether the imaging size of the pointing object in the two-dimensional image is within a preset size interval.
  • FIG. 6 is a gesture recognition method provided by some embodiments of the present disclosure, including the following steps 201 to 203.
  • Step 201 Process each of the plurality of two-dimensional images including the pointing object, and obtain an imaging size of the pointing object in each of the two-dimensional images.
  • the pointing object may be a user's finger, the user's palm, or may be a long object held by the user or the like.
  • the pointing object In the process of collecting the two-dimensional image including the pointing object by the two-dimensional image capturing device of the electronic device, the pointing object can be moved back and forth or up and down. And some movements may cause the pointing objects to have different imaging sizes in different two-dimensional images.
  • Step 202 Determine at least one target two-dimensional image from the plurality of two-dimensional images according to an imaging size of the pointing object in each two-dimensional image, wherein the pointing object is in the target two-dimensional image.
  • the image size is within the first preset size interval.
  • the first preset size interval may be an arbitrarily defined reasonable interval, which is not limited in this embodiment of the present disclosure.
  • the first predetermined size interval described above defines the depth of the operating region in the direction toward the two-dimensional image capturing device. After the operation area is available, only the operation in the operation area is recognized, that is to say, the user can interact with the electronic device only in this operation area. When the user does not need to interact with the electronic device, it is only necessary to leave the operation area, which makes the user interaction process and the stopping process more convenient.
  • Step 203 Determine and execute a corresponding instruction by using the at least one target two-dimensional image.
  • the pointing objects are various, such as a finger, a palm, a strip, or a ball.
  • the following uses the pointing object as a finger to explain how to obtain the imaging size of the pointing object.
  • the two-dimensional image of each of the plurality of two-dimensional images including the pointing object is processed to obtain an imaging size of the pointing object in each of the two-dimensional images, specifically include:
  • the width of the connected region corresponding to the fingertip is determined based on the fingertip coordinates.
  • the connected area corresponding to the finger refers to the area containing the finger for determining the position of the finger in the two-dimensional image.
  • the connected area corresponding to the finger included in the two-dimensional image can be identified by the image recognition method.
  • the entire palm is a connected area including the finger.
  • the connected region corresponding to the fingertip may be determined based on the fingertip coordinates, for example, the region indicated by the circle M in FIG. 8, and the width of the connected region at this time may be the width L of the fingertip.
  • the connected area corresponding to the finger included in the two-dimensional image can be identified by the method of image recognition.
  • the connected area here can be understood as the area of the entire palm, which may be an area formed by the palm opening, or an area formed by the palm grip.
  • the fingertip of the finger can be recognized by the image recognition method, and the coordinates of the fingertip can be obtained.
  • the width of the connected region corresponding to the fingertip can be directly recorded based on the fingertip coordinates.
  • the width of the connected area corresponding to the fingertip can be understood as the width of the fingertip in the two-dimensional image.
  • the method of obtaining the width of the fingertip for example, first, the skin color segmentation, for example, using the Otsu algorithm to calculate and obtain a binary image. Then, according to the actually acquired gesture, the range of the finger connected area within the range can be manipulated, that is, the setting of the maximum pixel value and the minimum pixel value of the fingertip width is excluded, and some images that do not satisfy the requirement are excluded.
  • the user does not need to use other tools (such as a strip or a spheroid, etc.) to perform gesture interaction with the electronic device, and can directly perform gesture interaction with the electronic device by using a finger.
  • other tools such as a strip or a spheroid, etc.
  • the width between most people's fingers is generally not too different, so that the electronic device can conveniently set a reasonable operating area for the public.
  • the first preset size interval is a preset size interval of the at least two preset size intervals, and any two preset size intervals of the at least two preset size intervals are spaced apart.
  • the first preset size interval is a preset size interval of the at least two preset size intervals, and any two of the at least two preset size intervals are spaced apart, then There can be at least two operating areas in the space, and there is no intersection between the two operating areas. Conventionally, it can be understood that there are at least two operation areas in front of the two-dimensional image acquisition device, and operations can be performed in each operation area to interact with the electronic device.
  • FIG. 9 is a schematic diagram of different operating regions provided by some embodiments of the present disclosure. It can be seen from FIG. 9 that there are two operation areas in the space, the operation area A and the operation area B, and there is a certain spatial interval between the operation area A and the operation area B, and the operation within the space interval does not
  • the electronic devices interact with each other, and only when operating in the operation area A or the operation area B can the interaction with the electronic device.
  • the plurality of two-dimensional images further includes at least one two-dimensional image corresponding to a second preset size interval of the at least two preset size intervals; and the utilizing the at least one target two-dimensional image Determining and executing the corresponding instructions specifically includes:
  • a corresponding instruction is determined and executed using at least one target two-dimensional image corresponding to the first predetermined size interval.
  • different preset size intervals correspond to different operation areas, and determining and executing corresponding instructions by using at least one target two-dimensional image corresponding to the first preset size interval may ensure that an instruction or an operation should be performed. In an operation area, it is avoided to put the operations in the two operation areas into one sequence for action matching, which ensures that the matching process is more accurate.
  • the size of the object in the image and the distance between the object and the image acquisition device are related.
  • the slight change of the pointing object in the depth direction will result in a larger size of the pointing object in the image.
  • the change, so the operating area closer to the image acquisition device, is to be set to a relatively large size range to reduce the user's operational accuracy requirements.
  • the preset size interval in which the imaging size value is larger the preset size interval in which the imaging size value is larger, the corresponding interval length is larger.
  • the imaging size of the pointing object in the image collected by the image capturing device is relatively large; when the pointing object is far away from the image capturing device, The imaging size of the pointing object in the image acquired by the image acquisition device will be relatively small. And when the pointing object is relatively close to the image capturing device, the slight movement of the pointing object in the depth direction may cause the imaging size of the pointing object to change relatively.
  • the value of the preset size interval when relatively large, it can correspond to a relatively large operation area, so that the operation area can have a relatively good tolerance, so that the pointing object is at a relatively close distance from the image acquisition device. In some operations, it can be well recognized by the image acquisition device.
  • the gesture recognition method further includes:
  • the target controllable object set includes a plurality of preset controllable objects, and the controllable objects may be some icons, some buttons or some options, etc., and the corresponding operations may be implemented by the controllable objects.
  • the first preset size interval may correspond to one operation area, and after determining the target controllable object set corresponding to the first preset size interval, the controllable object in the target controllable object set may be displayed. In this way, in the operation area corresponding to the first preset size interval, the controllable object in the target controllable object set can be operated. In this way, these controllable objects are dispersed into a layer for display, and the controllable object can be enlarged and displayed, which can reduce the operation precision of the user and reduce the accuracy of image recognition. The user can simply select the one that he needs. Controllable object.
  • all controllable objects are generally displayed in an area, so that the icon of each controllable object may be relatively small, and the user's finger may be relatively large, so that the finger is Choosing a smaller icon will have a higher probability of choosing the wrong one, which will affect the accuracy of the selection.
  • controllable objects in the set of controllable objects can be displayed separately, so that a large display area is used to display less controllable objects, and each controllable object can be enlarged. And after zooming in, the display area can also accommodate the zoomed controllable object. And in the case of determining the user's fingertip coordinates, a controllable object can be operated corresponding to a controllable object. Since the controllable object is enlarged, the finger can easily select the controllable object after the enlargement, so that the user can select the desired controllable object and operate it at will.
  • the displaying the controllable object in the target controllable object set includes:
  • the current first display interface is updated to include a second display interface of the controllable object in the target controllable object set.
  • the currently displayed controllable object set may be the target controllable object set
  • secondary display is not required. It is only necessary to update the current first display interface to a second display interface including the controllable object in the target controllable object set when the target controllable object set is different from the currently displayed controllable object set. In this way, the controllable object can be switched only when the operation area is switched, without having to judge and refresh each time.
  • the first operating area at this time can display a collection of buttons for image cropping. If the determined set of target controllable objects is a collection of buttons that are cropped by the image, then there is no need to update the display. If the user leaves the first operation area and enters the second operation area, the target controllable object set determined at this time is a set of some buttons for image rendering, and the currently displayed collection of some buttons for image cropping, Then you can update to update the collection of some of the buttons cropped by the image to a collection of buttons for image rendering.
  • buttons with a large number of buttons are dispersed into different operation areas, and the switching of the operation areas only needs to be controlled by the user to control the operation distance, which not only realizes convenient operation switching, but also distributes the buttons to multiple operation interfaces, thereby reducing the number of buttons.
  • User's operational accuracy and image recognition accuracy requirements are provided.
  • the gesture recognition method further includes:
  • the process of determining the first preset size interval according to the reference two-dimensional image may be: expanding the imaging size of the reference two-dimensional image forward and backward by the same size determination first preset size interval, or It may be that the imaging size of the reference two-dimensional image is expanded forward and backward by a different size to determine the first predetermined size interval or the like.
  • the size of the preset size interval may be defined in advance, and the embodiment of the present disclosure is not limited thereto.
  • the first preset size interval does not need to be defined in advance, but the first preset size interval is determined according to the reference two-dimensional image acquired in real time, so that the electronic device and the user have stronger adaptability. The interaction process is more flexible.
  • the first preset size interval is (W-dw, W+dw), where W is an imaging size of the pointing object in the reference two-dimensional image, and dw is the first pre-predetermined Set the interval length threshold for the size interval.
  • the reference two-dimensional image includes:
  • the first two-dimensional image acquired by the electronic device after this startup is the first two-dimensional image acquired by the electronic device after this startup.
  • the acquisition time is a two-dimensional image in which the interval between the acquisition time of the two-dimensional image acquired by the electronic device is longer than a predetermined time threshold.
  • the two-dimensional image acquired by the electronic device for the first time after the startup is started, so that the electronic device can be initialized in real time.
  • the electronic device can prompt and so on, asking the user whether it needs to be initialized. If not initialized, the last used 2D image can also be used as a reference 2D image.
  • the reference two-dimensional image may also be a two-dimensional image in which the interval between the acquisition time and the acquisition time of the two-dimensional image acquired by the electronic device exceeds a predetermined time threshold, so that the user returns after a period of time , can also be re-initialized.
  • the electronic device can also prompt, ask the user whether it needs to be initialized, and so on. In this way, the interaction between the electronic device and the user can be more intelligent, and the user is more convenient to use.
  • the electronic device can capture the fingertip of the user by the method of image recognition, and acquire the imaging size of the fingertip.
  • the electronic device is preset with a size interval through which an operation region can be determined. If the imaging size of the fingertip does not meet the requirements of the operating area, the electronic device cannot be interacted with, and only the operating area determined by the size interval can interact with the electronic device.
  • the electronic device does not have a size interval in advance, and from the user's point of view, it can be considered that there is no fixed operation area.
  • an operation area can be determined by the imaging size of the currently acquired fingertip, which is equivalent to an initialization process, and specifically initializes an operation area.
  • controllable objects related to the current operating area within the current operating area, and only display these controllable objects related to the current operating area.
  • some relatively small controllable objects can be enlarged in the display area, and the user can easily select the controllable object that he needs to operate.
  • the electronic device can set a plurality of size intervals, so that it can be understood that the user sets a plurality of operation areas from the perspective of the user.
  • the imaging size of the fingertip changes accordingly.
  • the imaging size of the fingertip falls from the current size interval into another size interval, it can be recognized by the image acquisition device, thereby being able to interact with the electronic device.
  • the electronic device can display some controllable corresponding to the other operation area.
  • controllable objects can also be enlarged for easy operation by the user.
  • this embodiment provides another process of interaction.
  • the image acquisition device of the electronic device can acquire the position and width of the fingertip, and then initialize the first layer according to the position and width of the collected fingertip.
  • the first layer may be the first layer in a particular direction, which may be the direction of the fingertip toward the image capture device.
  • the first layer may be a region farther from the image capturing device, or may be a region closer to the image capturing device.
  • the initialization method may be that the fingertip is detected for the first time, and the fingertip position remains unchanged for several seconds. After the fingertip is not detected for more than a certain period of time, the first layer information is reinitialized when the fingertip is detected again.
  • the operation position in the electronic device is mapped; in addition, the initialization width of the base layer is W, and the minimum fingertip preset pixel number is X, and the pixel is far and near.
  • the change threshold dw indicates that the operation is left, so there are the following cases:
  • the change is greater than dw, but is still within the nth layer operating range, it is considered to be the raised state, that is, temporarily leaving the nth layer, such as the above example of inputting the letter "T";
  • the corresponding interface switches the corresponding operation layer
  • a corresponding prompt is given on the display interface, or a menu of the corresponding layer is displayed, and the functions of the different layers and the corresponding menu are defined in the software of the application, different The function is placed on different layers, which simplifies the operation of each layer and makes positioning more accurate. In addition, multiple layers can be switched between and more efficient.
  • a gesture recognition method processing each of the plurality of two-dimensional images including the pointing object, and acquiring an imaging size of the pointing object in each of the two-dimensional images; An imaging size of the pointing object in each two-dimensional image, determining at least one target two-dimensional image from the plurality of two-dimensional images, wherein an imaging size of the pointing object in the target two-dimensional image is at Within the first predetermined size interval; using the at least one target two-dimensional image, the corresponding instruction is determined and executed.
  • the two-dimensional image can be screened using the imaging size of the pointing object, and the target two-dimensional image that satisfies the condition is used for interaction, thereby improving the recognition accuracy.
  • FIG. 10 is a structural diagram of an electronic device according to an embodiment of the present disclosure, which can implement the details of the gesture recognition method in the foregoing embodiment, and achieve the same effect.
  • the electronic device 1000 includes a first obtaining module 1001, a first determining module 1002, and an executing module 1003.
  • the first obtaining module 1001 is connected to the first determining module 1002, and the first determining module 1002 is connected to the executing module 1003.
  • the first obtaining module 1001 is configured to process each of the plurality of two-dimensional images including the pointing object, and acquire an imaging size of the pointing object in each of the two-dimensional images.
  • the first determining module 1002 is configured to determine at least one target two-dimensional image from the plurality of two-dimensional images according to an imaging size of the pointing object in each two-dimensional image;
  • the execution module 1003 is configured to determine and execute a corresponding instruction by using the at least one target two-dimensional image.
  • the pointing object includes a finger
  • the first obtaining module 1001 includes: a first determining submodule 10011 , an obtaining submodule 10012 , and a second determining submodule 10013 .
  • the first determining sub-module 10011 is configured to determine a connected area corresponding to the finger included in the two-dimensional image.
  • the acquisition sub-module 10012 is configured to perform fingertip positioning on the finger based on geometric features of the finger to acquire fingertip coordinates.
  • the second determining sub-module 10013 is configured to determine a connected area width corresponding to the fingertip coordinates.
  • the imaging size of the pointing object in the target two-dimensional image is within a first preset size interval.
  • the first preset size interval is a preset size interval of the at least two preset size intervals, and any two preset size intervals of the at least two preset size intervals are spaced apart.
  • the plurality of two-dimensional images further includes at least one two-dimensional image corresponding to a second preset size interval of the at least two preset size intervals.
  • the executing module 1003 is specifically configured to: determine and execute a corresponding instruction by using at least one target two-dimensional image corresponding to the first preset size interval.
  • the preset size interval with a larger value is larger, and the corresponding interval length is larger.
  • the electronic device 1000 further includes: a second determining module 1004 and a display module 1005.
  • the second determining module 1004 is configured to determine, according to a mapping relationship between the preset size interval and the controllable object set, the target controllable object set corresponding to the first preset size interval.
  • the display module 1005 is configured to display the controllable object in the target controllable object set.
  • the display module 1005 is configured to: when the target controllable object set is different from the currently displayed control object set, update the current first display interface to include the target controllable object set.
  • the second display interface of the controllable object is configured to: when the target controllable object set is different from the currently displayed control object set, update the current first display interface to include the target controllable object set.
  • the second display interface of the controllable object is configured to: when the target controllable object set is different from the currently displayed control object set, update the current first display interface to include the target controllable object set.
  • the second display interface of the controllable object is configured to: when the target controllable object set is different from the currently displayed control object set, update the current first display interface to include the target controllable object set.
  • the second display interface of the controllable object is configured to: when the target controllable object set is different from the currently displayed control object set, update the current first display interface to include the target controllable object set.
  • the electronic device 1000 further includes: a second obtaining module 1006 and a third determining module 1007.
  • the second acquisition module 1006 is configured to acquire a reference two-dimensional image.
  • the third determining module 1007 is configured to determine the first preset size interval according to the reference two-dimensional image.
  • the first preset size interval is (W-dw, W+dw), where W is an imaging size of the pointing object in the reference two-dimensional image, and dw is the first pre-predetermined Set the interval length threshold for the size interval.
  • the reference two-dimensional image includes:
  • the first two-dimensional image acquired by the electronic device after this startup is the first two-dimensional image acquired by the electronic device after this startup.
  • the acquisition time is a two-dimensional image in which the interval between the acquisition time of the two-dimensional image acquired by the electronic device is longer than a predetermined time threshold.
  • the electronic device 1000 can implement various processes implemented by the electronic device in the method embodiments of FIG. 5 to FIG. 6. To avoid repetition, details are not described herein again.
  • the electronic device 1000 of the embodiment of the present disclosure processes each of the plurality of two-dimensional images including the pointing object, and acquires an imaging size of the pointing object in each of the two-dimensional images; Pointing at an imaging size in each two-dimensional image, determining at least one target two-dimensional image from the plurality of two-dimensional images; and using the at least one target two-dimensional image, determining and executing a corresponding instruction.
  • the two-dimensional image can be screened using the imaging size of the pointing object, and the target two-dimensional image that satisfies the condition is used for interaction, so that some misoperations can be avoided as much as possible.
  • FIG. 14 is a schematic diagram of a hardware structure of an electronic device that implements various embodiments of the present disclosure.
  • the electronic device 1400 includes a memory 1409 and a processor 1410 .
  • the electronic device includes, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a palmtop computer, an in-vehicle terminal, a wearable device, a pedometer, and the like.
  • the processor 1410 is configured to process each of the plurality of two-dimensional images including the pointing object, and acquire an imaging size of the pointing object in each of the two-dimensional images; according to the pointing object Determining at least one target two-dimensional image from the plurality of two-dimensional images; and using the at least one target two-dimensional image to determine and execute a corresponding instruction.
  • the two-dimensional image can be screened using the imaging size of the pointing object, and the target two-dimensional image that satisfies the condition is used for interaction, thereby improving the recognition accuracy.
  • the pointing object includes a finger
  • the processor 1410 is further configured to determine a connected area corresponding to the finger included in the two-dimensional image; and perform fingertip positioning on the finger based on geometric features of the finger to obtain a fingertip Coordinates; determining the width of the connected region corresponding to the fingertip coordinates.
  • the imaging size of the pointing object in the target two-dimensional image is within a first preset size interval.
  • the first preset size interval is a preset size interval of the at least two preset size intervals, and any two preset size intervals of the at least two preset size intervals are spaced apart.
  • the plurality of two-dimensional images further includes at least one two-dimensional image corresponding to a second preset size interval of the at least two preset size intervals; the processor 1410 is further configured to utilize the At least one target two-dimensional image corresponding to a predetermined size interval determines and executes a corresponding instruction.
  • the preset size interval with a larger value is larger, and the corresponding interval length is larger.
  • the processor 1410 is further configured to: determine, according to a mapping relationship between the preset size interval and the set of controllable objects, a target controllable object set corresponding to the first preset size interval; and display the target controllable A controllable object in a collection of objects.
  • the processor 1410 is further configured to: when the target controllable object set is different from the currently displayed control object set, update the current first display interface to include the controllable in the target controllable object set.
  • the second display interface of the object is further configured to: when the target controllable object set is different from the currently displayed control object set, update the current first display interface to include the controllable in the target controllable object set.
  • the second display interface of the object is further configured to: when the target controllable object set is different from the currently displayed control object set, update the current first display interface to include the controllable in the target controllable object set.
  • the second display interface of the object is further configured to: when the target controllable object set is different from the currently displayed control object set, update the current first display interface to include the controllable in the target controllable object set.
  • the processor 1410 is further configured to acquire a reference two-dimensional image; and determine the first preset size interval according to the reference two-dimensional image.
  • the first preset size interval is (W-dw, W+dw), where W is an imaging size of the pointing object in the reference two-dimensional image, and dw is the first pre-predetermined Set the interval length threshold for the size interval.
  • the reference two-dimensional image includes: a two-dimensional image that is acquired for the first time after the electronic device is started this time; or that the time between the acquisition time and the acquisition time of the two-dimensional image acquired by the electronic device exceeds a predetermined time threshold. Dimensional image.
  • the electronic device of the specific embodiment of the present disclosure may further include one or more of the following components: a radio frequency unit 1401 , a network module 1402 , an audio output unit 1403 , an input unit 1404 , and a sensor 1405 .
  • the radio frequency unit 1401 may be used for receiving and transmitting signals during and after receiving or transmitting information or a call, and specifically, after receiving downlink data from the base station, processing the processor 1410; The uplink data is sent to the base station.
  • radio frequency unit 1401 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like.
  • the radio unit 1401 can also communicate with the network and other devices through a wireless communication system.
  • the electronic device provides the user with wireless broadband Internet access through the network module 1402, such as helping the user to send and receive emails, browse web pages, and access streaming media.
  • the audio output unit 1403 may convert the audio data received by the radio frequency unit 1401 or the network module 1402 or stored in the memory 1409 into an audio signal and output as a sound. Moreover, the audio output unit 1403 can also provide audio output (eg, call signal reception sound, message reception sound, etc.) associated with a particular function performed by the electronic device 1400.
  • the audio output unit 1403 includes a speaker, a buzzer, a receiver, and the like.
  • the input unit 1404 is for receiving an audio or video signal.
  • the input unit 1404 may include a graphics processing unit (GPU) 14041 and a microphone 14042, and the graphics processor 14041 images an still picture or video obtained by an image capturing device (such as a camera) in a video capturing mode or an image capturing mode.
  • the data is processed.
  • the processed image frame can be displayed on display unit 1406.
  • the image frames processed by the graphics processor 14041 may be stored in the memory 1409 (or other storage medium) or transmitted via the radio unit 1401 or the network module 1402.
  • the microphone 14042 can receive sound and can process such sound as audio data.
  • the processed audio data can be converted to a format output that can be transmitted to the mobile communication base station via the radio unit 1401 in the case of a telephone call mode.
  • the electronic device 1400 also includes at least one sensor 1405, such as a light sensor, motion sensor, and other sensors.
  • the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 14061 according to the brightness of the ambient light, and the proximity sensor can close the display panel 14061 when the electronic device 1400 moves to the ear. / or backlight.
  • the accelerometer sensor can detect the acceleration of each direction (usually three axes). When it is stationary, it can detect the magnitude and direction of gravity. It can be used to identify the posture of electronic devices (such as horizontal and vertical screen switching, related games).
  • sensor 1405 may also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, Infrared sensors and the like are not described here.
  • the display unit 1406 is for displaying information input by the user or information provided to the user.
  • the display unit 1406 can include a display panel 14061.
  • the display panel 14061 can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.
  • the user input unit 1407 can be configured to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the electronic device.
  • the user input unit 1407 includes a touch panel 14071 and other input devices 14072.
  • the touch panel 14071 also referred to as a touch screen, can collect touch operations on or near the user (such as the user using a finger, a stylus, or the like on the touch panel 1471 or near the touch panel 14071. operating).
  • the touch panel 14071 may include two parts of a touch detection device and a touch controller.
  • the touch detection device detects the touch orientation of the user, and detects a signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts the touch information into contact coordinates, and sends the touch information.
  • the processor 1410 receives the commands from the processor 1410 and executes them.
  • the touch panel 14071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves.
  • the user input unit 1407 may also include other input devices 14072.
  • the other input devices 14072 may include, but are not limited to, a physical keyboard, function keys (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which are not described herein.
  • the touch panel 14071 can be overlaid on the display panel 14061.
  • the touch panel 14071 detects a touch operation thereon or nearby, the touch panel 14071 transmits to the processor 1410 to determine the type of the touch event, and then the processor 1410 according to the touch.
  • the type of event provides a corresponding visual output on display panel 14061.
  • the touch panel 14071 and the display panel 14061 are used as two independent components to implement the input and output functions of the electronic device, in some embodiments, the touch panel 14071 and the display panel 14061 may be integrated. The input and output functions of the electronic device are implemented, and are not limited herein.
  • the interface unit 1408 is an interface in which an external device is connected to the electronic device 1400.
  • the external device may include a wired or wireless headset port, an external power (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, and an audio input/output. (I/O) port, video I/O port, headphone port, and more.
  • the interface unit 1408 can be configured to receive input from an external device (eg, data information, power, etc.) and transmit the received input to one or more components within the electronic device 1400 or can be used at the electronic device 1400 and externally Data is transferred between devices.
  • Memory 1409 can be used to store software programs as well as various data.
  • the memory 1409 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may be stored according to Data created by the use of the mobile phone (such as audio data, phone book, etc.).
  • memory 1409 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
  • the processor 1410 is a control center for the electronic device that connects various portions of the entire electronic device using various interfaces and lines, by running or executing software programs and/or modules stored in the memory 1409, and recalling data stored in the memory 1409. , performing various functions and processing data of the electronic device to perform overall monitoring of the electronic device.
  • the processor 1410 can include one or more processing units; optionally, the processor 1410 can integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, an application, etc., and a modulation solution
  • the processor mainly handles wireless communication. It will be appreciated that the above described modem processor may also not be integrated into the processor 1410.
  • the electronic device 1400 can also include a power source 1411 (such as a battery) that supplies power to various components.
  • a power source 1411 such as a battery
  • the power source 1411 can be logically coupled to the processor 1410 through a power management system to manage charging, discharging, and power consumption through the power management system. Management and other functions.
  • an embodiment of the present disclosure further provides an electronic device, including a processor 1410, a memory 1409, a computer program stored on the memory 1409 and executable on the processor 1410, the computer program being executed by the processor 1410.
  • an electronic device including a processor 1410, a memory 1409, a computer program stored on the memory 1409 and executable on the processor 1410, the computer program being executed by the processor 1410.
  • the embodiment of the present disclosure further provides a computer readable storage medium.
  • the computer readable storage medium stores a computer program, which is executed by the processor to implement various processes of the foregoing gesture recognition method embodiment, and can achieve the same technology. The effect, to avoid repetition, will not be repeated here.
  • the computer readable storage medium such as a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
  • the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former is better.
  • Implementation Based on such understanding, the technical solution of the present disclosure, which is essential or contributes to the related art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, CD-ROM).
  • the instructions include a number of instructions for causing a terminal (which may be a cell phone, computer, server, air conditioner, or network device, etc.) to perform the methods described in various embodiments of the present disclosure.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • User Interface Of Digital Computer (AREA)
  • Position Input By Displaying (AREA)

Abstract

本公开提供一种手势识别方法及电子设备，该方法包括：对包括指点物的多个二维图像中的每个二维图像进行处理，获取所述指点物在所述每个二维图像中的成像尺寸；依据所述指点物在每个二维图像中的成像尺寸，从所述多个二维图像中确定至少一个目标二维图像；利用所述至少一个目标二维图像，确定并执行对应的指令。

Description

手势识别方法及电子设备
相关申请的交叉引用
本申请主张在2017年9月26日在中国提交的中国专利申请号No.201710882327.X的优先权，其全部内容通过引用包含于此。
技术领域
本公开涉及通信技术领域，尤其涉及一种手势识别方法及电子设备。
背景技术
随着电子设备技术的迅速发展，电子设备自身的运算能力越来越强，功能越来越多，电子设备与用户之间可以以多种方式进行交互，例如使用遥控器、鼠标、语音或者手势进行交互等等。由于手势交互的过程更加简单，且用户与电子设备交互的过程中自然随意，使用手势与电子设备之间进行交互的方式已经应用于很多场景之中。
相关技术中，很多时候都是基于深度摄像头来实现目标的跟踪以及手势的识别。但目前精度较高的深度摄像机的价格非常昂贵，如Mesa Imaging SwissRanger 4000(SR4000)的价格高达上万美金。
为了降低成本，在很多的场景中使用普通的二维图像采集装置来实现手势识别，但相关技术中的基于二维图像采集装置实现手势识别的方式存在识别精度较低的问题。举例如下，由于用户使用手势进行人机交互的过程中，由于没有实际的平面作为支撑，胳膊悬空容易抖动，这种抖动被采集到之后，由处理器进行手势识别就会得到错误的识别结果。
发明内容
第一方面，本公开实施例提供了一种手势识别方法，包括：
对包括指点物的多个二维图像中的每个二维图像进行处理，获取所述指点物在所述每个二维图像中的成像尺寸；
依据所述指点物在每个二维图像中的成像尺寸，从所述多个二维图像中 确定至少一个目标二维图像；
利用所述至少一个目标二维图像，确定并执行对应的指令。
第二方面，本公开实施例还提供一种电子设备，包括：
获取模块，用于对包括指点物的多个二维图像中的每个二维图像进行处理，获取所述指点物在所述每个二维图像中的成像尺寸；
第一确定模块，用于依据所述指点物在每个二维图像中的成像尺寸，从所述多个二维图像中确定至少一个目标二维图像；
执行模块，用于利用所述至少一个目标二维图像，判断并执行对应的指令。
第三方面，本公开实施例还提供一种电子设备，包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述手势识别方法中的步骤。
第四方面，本公开实施例还提供一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现上述手势识别方法中的步骤。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对本公开实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1是本公开一些实施例提供的图像采集距离的示意图之一；
图2是本公开一些实施例提供的图像显示的示意图之一；
图3是本公开一些实施例提供的图像采集距离的示意图之二；
图4是本公开一些实施例提供的图像显示的示意图之二；
图5是本公开一些实施例提供的手势识别方法的流程图；
图6是本公开一些实施例提供的手势识别方法的流程图；
图7是本公开一些实施例提供的连通区域的示意图之一；
图8是本公开一些实施例提供的连通区域的示意图之二；
图9是本公开一些实施例提供的不同操作区域的示意图；
图10是本公开一些实施例提供的电子设备的结构图之一；
图11是本公开一些实施例提供的电子设备的第一获取模块的结构图；
图12是本公开一些实施例提供的电子设备的结构图之二；
图13是本公开一些实施例提供的电子设备的结构图之三；
图14是本公开一些实施例提供的电子设备的结构图。
具体实施方式
下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本公开一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
为了更好的理解本公开实施例的实现过程，以下先进行原理的说明。
首先，指点物在图像中的成像尺寸和指点物与图像采集装置之间的距离是密切相关的。以下结合图1至图4解释如下。
成像尺寸是指物体在图像中的尺寸。例如，指点物在图像中的成像尺寸为指点物在图像中的尺寸。如图1-图4所示的指点物为一个球状体。从图1中可以看到，图像采集装置与球状体之间离的比较近，此时球状体在图像采集装置采集到的图像中的成像尺寸如图2所示。从图2中可以观察到，球状体在图像采集装置采集到的图像中的成像尺寸比较大。相对于图1来说，图3中的图像采集装置与球状体之间离的比较远，此时球状体在图像采集装置采集到的图像中的成像尺寸如图4所示。将图4与图2进行对比，可以发现，图4中球状体在图像采集装置采集到的图像中的成像尺寸就比较小。也就是说，成像尺寸可以间接的描述指点物与图像采集装置之间的距离。
本公开具体实施例中，为了提高识别精度，从用户角度来看，是为用户设置了一个操作区域，只有在操作区域内的用户操作才会被识别。根据之前的描述可知，成像尺寸可以间接的描述指点物与图像采集装置之间的距离，因此本公开具体实施例中可以根据指点物的成像尺寸来排除那些在操作区域外进行的操作，仅识别用户在操作区域内进行的操作，进而提高识别精度。
参见图5，图5是本公开一些实施例提供的手势识别方法的流程图，如图5所示，所述手势识别方法包括以下步骤101至103。
步骤101、对包括指点物的多个二维图像中的每个二维图像进行处理，获取所述指点物在所述每个二维图像中的成像尺寸。
本公开实施例中，上述指点物可以是用户的手指，用户的手掌，或者也可以是用户可持握的物体(例如长条状物体)，或者还可以是贴附于用户手指上的物体(如反光膜片或者具有特定形状的片状反光体)等等。
步骤102、依据所述指点物在每个二维图像中的成像尺寸，从所述多个二维图像中确定至少一个目标二维图像。
结合前面的原理，本公开实施例从用户角度而言，是为用户设置了一个操作区域，因此用户操作可以分为两部分：在操作区域内的操作和在操作区域外的操作。目标二维图像为指点物在二维图像中的成像尺寸处于预先设置的尺寸区间内时的二维图像。本公开具体实施例中，仅识别用户在操作区域内的操作，而在操作区域外的一些操作，电子设备不进行响应。由前面的原理可知，判断用户是否在操作区域内进行操作，可以依据指点物在每个二维图像中的成像尺寸进行判断。当指点物在二维图像中的成像尺寸处于预先设置的尺寸区间内时，就可以确定用户是在操作区域内进行操作。
步骤103、利用所述至少一个目标二维图像，判断并执行对应的指令。
本公开具体实施例中，在得到目标二维图像之后，则可以利用目标二维图像形成的图像序列进行设备的控制。如依据多个目标二维图像确定指点物的轨迹，并确定与该轨迹匹配的指令。
由于成像尺寸可以间接的描述指点物与图像采集装置之间的距离，也就是说，当指点物与图像采集装置之间的位移发生变化时，指点物在图像采集装置采集到的图像中的成像尺寸也会发生变化。因此本公开具体实施例中，基于普通的二维图像采集装置采集到的二维图像中指点物的成像尺寸能够识别指点物在朝向图像采集装置方向上的运动。
同时，基于普通的二维图像采集装置采集到的二维图像中指点物的成像尺寸还可以判断用户操作是否在一个设定的空间中进行，也就是说，能够排除那些在操作区域外进行并会导致误识别的操作，提高基于二维图像采集装 置实现手势识别的方式的识别精度。
以下以写字板应用程序输入字母“T”为例对本公开具体实施例的效果进行举例说明。本公开具体实施例中，由于操作区域的存在，并且只识别这个操作区域内发生的操作，可以提高识别精度。
在相关技术中，当用户需要通过写字板应用程序输入字母“T”时，首先需要画一条水平的线段，然后需要从水平线段的中点开始往下画一条垂直的线段，形成字母“T”。然而，在用户的手指从水平线段的端点回到水平线段的中点时，这个无需识别的动作也会被识别，造成误识别。
而本公开具体实施例中，由于操作区域的存在，用户可以将手抬起，离开设定的区域，然后在水平线段的中点进入设定的区域，由于用户的手指从水平线段的端点回到水平线段的中点的动作是在操作区域之外进行的，所以并不会被识别，也就避免了误识别的发生。而这种避免误识别的方式仅需要用户改变一下手指的运动路线，并不需要用户进行多余的动作，实现过程简单。
本公开实施例中，上述电子设备可以是手机、平板电脑(Tablet Personal Computer)、膝上型电脑(Laptop Computer)、个人数字助理(personal digital assistant，简称PDA)、移动上网装置(Mobile Internet Device，MID)或可穿戴式设备(Wearable Device)等等。
本公开具体实施例中，需要依据所述指点物在每个二维图像中的成像尺寸，从所述多个二维图像中确定至少一个目标二维图像。而从操作角度而言，也就是需要依据指点物在每个二维图像中的成像尺寸选择那些在操作区域中发生的操作。本公开具体实施例中可以依据多种方式来确定目标二维图像，以下对其中一种方式具体说明如下。
由前面的原理可知，判断用户是否在操作区域内进行操作，可以依据指点物在每个二维图像中的成像尺寸进行判断。在朝向图像采集装置的方向上，判断用户是否在操作区域内进行操作，则可以依据指点物在二维图像中的成像尺寸是否处于预先设置的尺寸区间内进行判断。
参见图6，图6是本公开一些实施例提供的手势识别方法包括以下步骤201至203。
步骤201、对包括指点物的多个二维图像中的每个二维图像进行处理，获取所述指点物在所述每个二维图像中的成像尺寸。
本公开实施例中，上述指点物可以是用户的手指，用户的手掌，或者也可以是用户持握的长条状物体等等。在电子设备的二维图像采集装置对包括指点物的二维图像进行采集的过程中，指点物是可以前后或者上下进行移动的。而某些移动就可能造成指点物在不同的二维图像中的成像尺寸不同。
步骤202、依据所述指点物在每个二维图像中的成像尺寸，从所述多个二维图像中确定至少一个目标二维图像，其中，所述指点物在所述目标二维图像中的成像尺寸处于第一预设尺寸区间内。
本公开实施例中，上述第一预设尺寸区间可以是任意定义的合理区间，对此本公开实施例不作限定。
基于简单的映射关系可以知道，上述的第一预设尺寸区间定义了操作区域在朝向二维图像采集装置方向上的深度。在有了这个操作区域之后，只有在操作区域中的操作才会被识别，也就是说用户只有在这个操作区域内才能实现与电子设备进行交互。当用户不需要与电子设备之间进行交互时，只需要离开这个操作区域就可以，使用户的交互过程以及停止过程更加便捷。
步骤203、利用所述至少一个目标二维图像，判断并执行对应的指令。
本公开具体实施例中，指点物多种多样，如可以为手指，可以为手掌，可以为条状物或者也可以为球状物等等。下面以指点物为手指说明具体如何获取指点物的成像尺寸。
当所述指点物为手指时，所述对包括指点物的多个二维图像中的每个二维图像进行处理，获取所述指点物在所述每个二维图像中的成像尺寸，具体包括：
确定所述二维图像中包括的手指对应的连通区域；
基于手指的几何特征对所述手指进行指尖定位，获取指尖坐标；
基于指尖坐标确定所述指尖对应的连通区域宽度。
其中，手指对应的连通区域是指二维图像中用于确定手指位置的包含手指的区域。为了更好的理解上述过程，可以参阅图7至图8。首先，可以通过图像识别的方法识别出二维图像中包括的手指对应的连通区域，这时候就 可以如图7和图8所示，整个手掌就为包括手指对应的连通区域。将整个手掌作为包括手指对应的连通区域有利于在二维图像中快速准确地识别出手指所在的区域，进而在该区域内基于手指的几何特征通过一些方式对手指的指尖进行定位并获取指尖的坐标。此时，可以基于指尖坐标确定指尖对应的连通区域，例如，图8中圈M所示的区域，并且此时的连通区域宽度就可以为指尖的宽度L。
本实施方式中，通过图像识别的方法可以识别出二维图像中包括的手指对应的连通区域，如手掌。这里的连通区域可以理解为整个手掌的区域，该区域可以是由手掌张开形成的区域，也可以是由手掌持握形成的区域。同理，亦可以通过图像识别的方法识别出手指的指尖，并进行定位以及获取指尖的坐标。
在确定指尖坐标之后，就可以基于指尖坐标直接记录指尖对应的连通区域宽度。并且指尖对应的连通区域宽度，就可以通俗的理解为手指指尖在二维图像中成像的宽度。指尖宽度的获取方式，例如：首先，进行肤色分割，比如采用大津算法进行计算，并获取二值图像。然后，根据实际获取的手势可操控范围内手指连通区域的范围，即由指尖宽度的最大像素值和最小像素值的设定，以来排除一些不满足要求的图像。再针对每个连通区域，获取重心后，从轮廓中找到距离重心最远的点，然后获取此点所在轮廓的宽度和长度，并记录长度与宽度比。最后，从多个长度与宽度比找出处于经验比值范围内最大的一个，作为指尖，并记录指尖的宽度，供后续判断手指距离二维图像采集装置距离变化使用。
通过这种方式，用户不需要使用其他工具(例如条状体或者球状体等等)与电子设备进行手势交互，直接使用手指就可以与电子设备进行手势交互。并且大部分人的手指之间的宽度一般来说相差不会太大，从而电子设备可以方便的为大众设置一个合理的操作区域。
可选的，所述第一预设尺寸区间为至少两个预设尺寸区间中的预设尺寸区间，该至少两个预设尺寸区间中的任意两个预设尺寸区间间隔分布。
本实施方式中，由于第一预设尺寸区间为至少两个预设尺寸区间中的预设尺寸区间，且该至少两个预设尺寸区间中的任意两个预设尺寸区间间隔分 布，那么在空间中就可以存在至少两个操作区域，并且这两个操作区域之间不会有交集。通俗的可以理解为在二维图像采集装置的前方存在至少两个操作区域，并且在每个操作区域中都可以进行操作以与电子设备之间进行交互。
为了更好的理解上述的两个操作区域，可以参阅图9，图9为本公开一些实施例提供的不同操作区域的示意图。从图9中可以看到，在空间中存在两个操作区域，操作区域A和操作区域B，操作区域A与操作区域B之间存在一定的空间间隔，在该空间间隔内进行操作不会与电子设备之间进行交互，只有在操作区域A或操作区域B中进行操作才可以与电子设备之间进行交互。
本实施方式中，存在至少两个操作区域供用户操作，使用户可以进行多个选择，并可以选择离自己比较近的那个操作区域进行操作，使用户与电子设备之间的交互更加方便。
可选的，所述多个二维图像还包括与所述至少两个预设尺寸区间中的第二预设尺寸区间对应的至少一个二维图像；所述利用所述至少一个目标二维图像，确定并执行对应的指令具体包括：
利用与所述第一预设尺寸区间对应的至少一个目标二维图像确定并执行对应的指令。
本实施方式中，不同的预设尺寸区间对应着不同的操作区域，而利用与第一预设尺寸区间对应的至少一个目标二维图像确定并执行对应的指令，可以确保一个指令或者一次操作应该在一个操作区域中发生，避免把两个操作区域中的操作放到一个序列进行动作匹配，这样可以确保匹配的过程更加精确。
并且，图像中物体的尺寸和物体与图像采集设备之间的距离是相关的，当距离图像采集设备较近时，指点物在深度方向上的细微变化都将导致图像中指点物尺寸较大的变化，因此距离所述图像采集设备越近的操作区域要设置相对较大的尺寸范围，以降低用户的操作精度要求。
因此，在本公开具体实施例，所述至少两个预设尺寸区间中，成像尺寸数值越大的预设尺寸区间，对应的区间长度越大。
本实施方式中，当指点物离图像采集装置的距离比较近的时候，指点物在图像采集装置采集的图像中的成像尺寸会比较大；当指点物离图像采集装 置的距离比较远的时候，指点物在图像采集装置采集的图像中的成像尺寸会比较小。并且当指点物离图像采集装置比较近的时候，指点物在深度方向上稍微进行一些移动可能就会导致指点物的成像尺寸变化比较大。
这样，当预设尺寸区间的数值比较大时，就可以对应一个长度比较大的操作区域，从而该操作区域可以有一个比较好的容忍度，使指点物在离图像采集装置比较近的距离进行一些操作时，可以很好的被图像采集装置所识别。
可选的，所述手势识别方法还包括：
根据预设尺寸区间与可控对象集合之间的映射关系，确定与所述第一预设尺寸区间对应的目标可控对象集合；
显示所述目标可控对象集合中的可控对象。
本实施方式中，目标可控对象集合包括多个预设的可控对象，且上述可控对象可以是一些图标、一些按钮或者一些选项等等，通过这些可控对象可以实现相应的操作。
本实施方式中，第一预设尺寸区间可以对应一个操作区域，在确定与第一预设尺寸区间对应的目标可控对象集合之后，可以显示目标可控对象集合中的可控对象。这样，在第一预设尺寸区间对应的操作区域中，就可以操作目标可控对象集合中的可控对象。这样将这些可控对象分散到某个层中进行显示，亦可以对可控对象进行放大显示，既能降低用户的操作精度，也可以降低图像识别的精度，用户很简单可以选择到自己需要的可控对象。
相关技术中，一般都是将所有的可控对象放在一个区域内进行显示，这样每个可控对象的图标可能就比较小，而用户的手指相对来说可能就比较大，这样通过手指去选择一个比较小的图标，选择错误的概率会比较大，从而影响选择的精度。
本实施方式中，可以将一些可控对象集合中的可控对象拿出来单独显示，这样使用一个大的显示区域来显示比较少的可控对象，每个可控对象就可以进行一定的放大，并且在放大之后显示区域还可以容纳下放大的可控对象。并且在确定用户的指尖坐标的情况下，对应到某个可控对象就可以操作某个可控对象。由于可控对象得到了放大，手指也很容易选择放大之后的可控对象，使用户很随意就可以选择到需要的可控对象并进行操作。
可选的，所述显示所述目标可控对象集合中的可控对象，包括：
在所述目标可控对象集合不同于当前显示的可控对象集合时，将当前的第一显示界面更新为包括所述目标可控对象集合中的可控对象的第二显示界面。
本实施方式中，由于当前显示的可控对象集合可以就是目标可控对象集合，从而不需要进行二次显示。只需要当目标可控对象集合不同于当前显示的可控对象集合时，将当前的第一显示界面更新为包括所述目标可控对象集合中的可控对象的第二显示界面。这样可以只在操作区域进行切换时才进行可控对象的切换，而不用每次都进行判断及刷新。
例如，用户在第一个操作区域进行操作时，此时第一个操作区域可以显示图像裁剪的一些按钮的集合。若确定的目标可控对象集合就为图像裁剪的一些按钮的集合，那么就不需要对显示进行更新。若此时用户离开第一个操作区域进入了第二个操作区域，此时确定的目标可控对象集合为图像渲染的一些按钮的集合，而当前显示的还是为图像裁剪的一些按钮的集合，那么就可以进行更新，将图像裁剪的一些按钮的集合更新为图像渲染的一些按钮的集合。
这样，在不同的操作区域可以显示不同的可控对象集合，就可以使用不同的可控对象集合进行不同功能的操作，使用户操作起来更加便捷。将数量较多的按钮分散到不同的操作区域，而操作区域的切换只需要用户进行操作距离的控制即可实现，不但实现了便利的操作切换，而且将按钮分散到多个操作界面，降低了用户的操作精度和图像识别精度的要求。
可选的，所述手势识别方法还包括：
获取参考二维图像；
根据所述参考二维图像，确定所述第一预设尺寸区间。
本实施方式中，就相当于一个第一预设尺寸区间初始化的过程。这里不需要预先设置第一预设尺寸区间，而是根据获取参考二维图像来确定第一预设尺寸区间。当然，这里根据所述参考二维图像，确定所述第一预设尺寸区间的过程，可以是将参考二维图像的成像尺寸向前后扩大相同的尺寸确定的第一预设尺寸区间，或者也可以是将参考二维图像的成像尺寸向前后扩大不 同的尺寸确定的第一预设尺寸区间等等。
当然，除了根据参考二维图像来确定第一预设尺寸区间，也可以预先定义预设尺寸区间的尺寸，对此本公开实施方式不作限定。本实施方式中，不需要预先定义第一预设尺寸区间，而是根据实时获取的参考二维图像来确定第一预设尺寸区间，可以使电子设备与用户之间有更强的适应性，交互过程更加灵活。
可选的，所述第一预设尺寸区间为(W-dw，W+dw)，所述W为所述指点物在所述参考二维图像中的成像尺寸，dw为所述第一预设尺寸区间的区间长度阈值。
可选的，所述参考二维图像包括：
电子设备本次启动后初次获取的二维图像；或者
获取时间与电子设备上次获取到的二维图像的获取时间的间隔超过预定时间门限的二维图像。
本实施方式中，电子设备本次启动后初次获取的二维图像，这样可以保证电子设备实时可以进行初始化。当然在初始化的过程中，电子设备可以进行提示等等，询问用户是否需要进行初始化。若没有进行初始化，还可以使用最后一次使用的二维图像来作为参考二维图像。
本实施方式中，所述参考二维图像还可以为获取时间与电子设备上次获取到的二维图像的获取时间的间隔超过预定时间门限的二维图像，这样使用户离开一段时间后再回来，亦可以进行重新初始化。当然这里电子设备也可以进行提示，询问用户是否需要初始化等等。这样，电子设备与用户之间的交互可以更加智能，用户使用也更加方便。
为了更好的理解交互的过程，使用下例进行说明。当用户开始与电子设备进行交互时，电子设备可以通过图像识别的方法捕捉到用户的指尖，并且获取指尖的成像尺寸。
当捕捉到指尖的成像尺寸之后有两种情况，一种是电子设备预先设置了尺寸区间，通过该尺寸区间可以确定一个操作区域。若指尖的成像尺寸不满足操作区域的要求，则无法与电子设备进行交互，只有在通过尺寸区间确定的操作区域内，才可以与电子设备进行交互。
而另一种是电子设备预先没有设置尺寸区间，从用户的角度来说就可以认为没有一个固定的操作区域。这样就可以通过当前采集的指尖的成像尺寸确定一个操作区域，这里就相当于一个初始化的过程，并且具体为初始化一个操作区域。
当然，用户在当前的操作区域内，可以显示与当前操作区域内相关的一些可控对象，并且仅仅显示与当前操作区域内相关的这些可控对象。那么通过这种方式，一些比较小的可控对象可以在显示区域内进行放大，用户可以比较轻松的选择到自己需要操作的可控对象。
在选择自己需要操作的可控对象的过程中，可以先确定用户的指尖的坐标，从而使用指尖坐标去在显示区域选择用户需要的可控对象，这样用户可以快速准确的选择到自己需要操作的可控对象。
当然电子设备可以设置多个尺寸区间，这样在用户的角度就可以理解为用户设置了多个操作区域。当用户的手指在深度方向上产生位移时，手指指尖的成像尺寸也会相应发生变化。当指尖的成像尺寸从当前的尺寸区间落入另外一个尺寸区间时，可以被图像采集装置所识别，从而又可以与电子设备进行交互。
当指尖的成像尺寸从当前的尺寸区间落入另外一个尺寸区间时，用户可以直观的理解为从一个操作区域进入另一个操作区域，电子设备就可以显示该另一个操作区域对应的一些可控对象。这些可控对象亦可以进行放大显示，便于用户进行操作。
除了上述交互方式之外，本实施例还提供另一种交互的过程。
电子设备的图像采集装置可以获取指尖的位置和宽度，然后根据采集的指尖的位置和宽度初始化第一层。第一层可以是特定方向上的第一层，该特定方向可以是指尖朝向图像采集装置的方向。当然第一层可以是离图像采集装置比较远的区域，也可以是离图像采集装置比较近的区域。
以第一层为离图像采集装置比较远的区域为例，初始化的方法可以是第一次检测到指尖，并且指尖位置保持若干秒内没有发生变化。对于超过一定时间没有检测到指尖后，再次检测到指尖时，重新初始化第一层信息。
根据指尖移动像素数以及获取到的图片分辨率，映射在电子设备中的操 作位置；另外设基层的初始化宽度为W，由远及近的方式，最小指尖预设像素数为X，像素变化阈值dw表示离开本层操作，那么有如下情况：
按照***需要的层数，计算每一层的预设宽度，比如三层则第N(0～2)层宽度为W3＝W-N*(W-X)/(3-1)，如果当前指尖的宽度在W3±((W-X)/(3-1)/2)，则是在第三层操作范围内；当然也可以采用其他方式来设置预置值，以保证各个层的深度区别；
如果指尖宽度从Wn开始变化，变化小于dw，则认为还处于第n个层；
如果指尖宽度从Wn开始变化，变化大于dw，但是仍然在第n层操作范围内，则认为是抬起状态，即暂时离开第n个层，例如上述输入字母“T”的例子；
如果指尖宽度从Wn开始变化，并且进入其他层的操作范围，则对应界面切换对应操作层；
如果发现指尖已经离开本层并且进入其他层，则在显示界面给出对应提示，或者显示对应层的菜单，对于不同层的功能和对应菜单的定义是在应用程序的软件中实现，不同的功能放在不同的层上，可以简化每一个层的操作范围，使定位更加准确。另外多个层之间可以进行切换，操作起来更加高效。
本公开实施例的一种手势识别方法，对包括指点物的多个二维图像中的每个二维图像进行处理，获取所述指点物在所述每个二维图像中的成像尺寸；依据所述指点物在每个二维图像中的成像尺寸，从所述多个二维图像中确定至少一个目标二维图像，其中，所述指点物在所述目标二维图像中的成像尺寸处于第一预设尺寸区间内；利用所述至少一个目标二维图像，确定并执行对应的指令。这样可以使用指点物的成像尺寸对二维图像进行筛选，使用满足条件的目标二维图像进行交互，从而可以提高识别精度。
参见图10，图10是本公开实施例提供的电子设备的结构图，能实现上述实施例中手势识别方法的细节，并达到相同的效果。如图10所示，电子设备1000包括第一获取模块1001、第一确定模块1002和执行模块1003，第一获取模块1001和第一确定模块1002连接，第一确定模块1002和执行模块1003连接。
第一获取模块1001用于对包括指点物的多个二维图像中的每个二维图 像进行处理，获取所述指点物在所述每个二维图像中的成像尺寸。
第一确定模块1002用于依据所述指点物在每个二维图像中的成像尺寸，从所述多个二维图像中确定至少一个目标二维图像；
执行模块1003用于利用所述至少一个目标二维图像，确定并执行对应的指令。
可选的，如图11所示，所述指点物包括手指，所述第一获取模块1001，具体包括：第一确定子模块10011、获取子模块10012和第二确定子模块10013。
第一确定子模块10011用于确定所述二维图像中包括的手指对应的连通区域。
获取子模块10012用于基于手指的几何特征对所述手指进行指尖定位，获取指尖坐标。
第二确定子模块10013用于确定所述指尖坐标对应的连通区域宽度。
可选的，所述指点物在所述目标二维图像中的成像尺寸处于第一预设尺寸区间内。
可选的，所述第一预设尺寸区间为至少两个预设尺寸区间中的预设尺寸区间，该至少两个预设尺寸区间中的任意两个预设尺寸区间间隔分布。
可选的，所述多个二维图像还包括与所述至少两个预设尺寸区间中的第二预设尺寸区间对应的至少一个二维图像。所述执行模块1003，具体用于：利用与所述第一预设尺寸区间对应的至少一个目标二维图像确定并执行对应的指令。
可选的，所述至少两个预设尺寸区间中，数值越大的预设尺寸区间，对应的区间长度越大。
可选的，如图12所示，所述电子设备1000还包括：第二确定模块1004和显示模块1005。
第二确定模块1004用于根据预设尺寸区间与可控对象集合之间的映射关系，确定与所述第一预设尺寸区间对应的目标可控对象集合。
显示模块1005用于显示所述目标可控对象集合中的可控对象。
可选的，所述显示模块1005，用于：在所述目标可控对象集合不同于当前显示的可控对象集合时，将当前的第一显示界面更新为包括所述目标可控 对象集合中的可控对象的第二显示界面。
可选的，如图13所示，所述电子设备1000还包括：第二获取模块1006和第三确定模块1007。
第二获取模块1006用于获取参考二维图像。
第三确定模块1007用于根据所述参考二维图像，确定所述第一预设尺寸区间。
可选的，所述第一预设尺寸区间为(W-dw，W+dw)，所述W为所述指点物在所述参考二维图像中的成像尺寸，dw为所述第一预设尺寸区间的区间长度阈值。
可选的，所述参考二维图像包括：
电子设备本次启动后初次获取的二维图像；或者
获取时间与电子设备上次获取到的二维图像的获取时间的间隔超过预定时间门限的二维图像。
电子设备1000能实现图5至图6的方法实施例中电子设备实现的各个过程，为避免重复，这里不再赘述。
本公开实施例的电子设备1000，对包括指点物的多个二维图像中的每个二维图像进行处理，获取所述指点物在所述每个二维图像中的成像尺寸；依据所述指点物在每个二维图像中的成像尺寸，从所述多个二维图像中确定至少一个目标二维图像；利用所述至少一个目标二维图像，确定并执行对应的指令。这样可以使用指点物的成像尺寸对二维图像进行筛选，使用满足条件的目标二维图像进行交互，从而可以尽量避免一些误操作。
参见图14，图14为实现本公开各个实施例的一种电子设备的硬件结构示意图，该电子设备1400包括：存储器1409和处理器1410。在本公开实施例中，电子设备包括但不限于手机、平板电脑、笔记本电脑、掌上电脑、车载终端、可穿戴设备、以及计步器等。
其中，处理器1410用于对包括指点物的多个二维图像中的每个二维图像进行处理，获取所述指点物在所述每个二维图像中的成像尺寸；依据所述指点物在每个二维图像中的成像尺寸，从所述多个二维图像中确定至少一个目标二维图像；利用所述至少一个目标二维图像，确定并执行对应的指令。这 样可以使用指点物的成像尺寸对二维图像进行筛选，使用满足条件的目标二维图像进行交互，从而可以提高识别精度。
可选的，所述指点物包括手指，处理器1410，还用于确定所述二维图像中包括的手指对应的连通区域；基于手指的几何特征对所述手指进行指尖定位，获取指尖坐标；确定所述指尖坐标对应的连通区域宽度。
可选的，所述指点物在所述目标二维图像中的成像尺寸处于第一预设尺寸区间内。
可选的，所述第一预设尺寸区间为至少两个预设尺寸区间中的预设尺寸区间，该至少两个预设尺寸区间中的任意两个预设尺寸区间间隔分布。
可选的，所述多个二维图像还包括与所述至少两个预设尺寸区间中的第二预设尺寸区间对应的至少一个二维图像；处理器1410还用于利用与所述第一预设尺寸区间对应的至少一个目标二维图像确定并执行对应的指令。
可选的，所述至少两个预设尺寸区间中，数值越大的预设尺寸区间，对应的区间长度越大。
可选的，处理器1410还用于根据预设尺寸区间与可控对象集合之间的映射关系，确定与所述第一预设尺寸区间对应的目标可控对象集合；显示所述目标可控对象集合中的可控对象。
可选的，处理器1410还用于在所述目标可控对象集合不同于当前显示的可控对象集合时，将当前的第一显示界面更新为包括所述目标可控对象集合中的可控对象的第二显示界面。
可选的，处理器1410还用于获取参考二维图像；根据所述参考二维图像，确定所述第一预设尺寸区间。
可选的，所述第一预设尺寸区间为(W-dw，W+dw)，所述W为所述指点物在所述参考二维图像中的成像尺寸，dw为所述第一预设尺寸区间的区间长度阈值。
可选的，所述参考二维图像包括：电子设备本次启动后初次获取的二维图像；或者获取时间与电子设备上次获取到的二维图像的获取时间的间隔超过预定时间门限的二维图像。
应当了解的是，如图14所示，本公开具体实施例的电子设备还可以包括 如下部件中的一个或多个：射频单元1401、网络模块1402、音频输出单元1403、输入单元1404、传感器1405、显示单元1406、用户输入单元1407、接口单元1408以及电源1411等部件。
应理解的是，本公开实施例中，射频单元1401可用于收发信息或通话过程中，信号的接收和发送，具体的，将来自基站的下行数据接收后，给处理器1410处理；另外，将上行的数据发送给基站。通常，射频单元1401包括但不限于天线、至少一个放大器、收发信机、耦合器、低噪声放大器、双工器等。此外，射频单元1401还可以通过无线通信***与网络和其他设备通信。
电子设备通过网络模块1402为用户提供了无线的宽带互联网访问，如帮助用户收发电子邮件、浏览网页和访问流式媒体等。
音频输出单元1403可以将射频单元1401或网络模块1402接收的或者在存储器1409中存储的音频数据转换成音频信号并且输出为声音。而且，音频输出单元1403还可以提供与电子设备1400执行的特定功能相关的音频输出(例如，呼叫信号接收声音、消息接收声音等等)。音频输出单元1403包括扬声器、蜂鸣器以及受话器等。
输入单元1404用于接收音频或视频信号。输入单元1404可以包括图形处理器(Graphics Processing Unit，GPU)14041和麦克风14042，图形处理器14041对在视频捕获模式或图像捕获模式中由图像捕获装置(如摄像头)获得的静态图片或视频的图像数据进行处理。处理后的图像帧可以显示在显示单元1406上。经图形处理器14041处理后的图像帧可以存储在存储器1409(或其它存储介质)中或者经由射频单元1401或网络模块1402进行发送。麦克风14042可以接收声音，并且能够将这样的声音处理为音频数据。处理后的音频数据可以在电话通话模式的情况下转换为可经由射频单元1401发送到移动通信基站的格式输出。电子设备1400还包括至少一种传感器1405，比如光传感器、运动传感器以及其他传感器。具体地，光传感器包括环境光传感器及接近传感器，其中，环境光传感器可根据环境光线的明暗来调节显示面板14061的亮度，接近传感器可在电子设备1400移动到耳边时，关闭显示面板14061和/或背光。作为运动传感器的一种，加速计传感器可检测各个方向上(一般为三轴)加速度的大小，静止时可检测出重力的大小及方向， 可用于识别电子设备姿态(比如横竖屏切换、相关游戏、磁力计姿态校准)、振动识别相关功能(比如计步器、敲击)等；传感器1405还可以包括指纹传感器、压力传感器、虹膜传感器、分子传感器、陀螺仪、气压计、湿度计、温度计、红外线传感器等，在此不再赘述。
显示单元1406用于显示由用户输入的信息或提供给用户的信息。显示单元1406可包括显示面板14061，可以采用液晶显示器(Liquid Crystal Display，LCD)、有机发光二极管(Organic Light-Emitting Diode,OLED)等形式来配置显示面板14061。
用户输入单元1407可用于接收输入的数字或字符信息，以及产生与电子设备的用户设置以及功能控制有关的键信号输入。具体地，用户输入单元1407包括触控面板14071以及其他输入设备14072。触控面板14071，也称为触摸屏，可收集用户在其上或附近的触摸操作(比如用户使用手指、触笔等任何适合的物体或附件在触控面板14071上或在触控面板14071附近的操作)。触控面板14071可包括触摸检测装置和触摸控制器两个部分。其中，触摸检测装置检测用户的触摸方位，并检测触摸操作带来的信号，将信号传送给触摸控制器；触摸控制器从触摸检测装置上接收触摸信息，并将它转换成触点坐标，再送给处理器1410，接收处理器1410发来的命令并加以执行。此外，可以采用电阻式、电容式、红外线以及表面声波等多种类型实现触控面板14071。除了触控面板14071，用户输入单元1407还可以包括其他输入设备14072。具体地，其他输入设备14072可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆，在此不再赘述。
进一步的，触控面板14071可覆盖在显示面板14061上，当触控面板14071检测到在其上或附近的触摸操作后，传送给处理器1410以确定触摸事件的类型，随后处理器1410根据触摸事件的类型在显示面板14061上提供相应的视觉输出。虽然在图14中，触控面板14071与显示面板14061是作为两个独立的部件来实现电子设备的输入和输出功能，但是在某些实施例中，可以将触控面板14071与显示面板14061集成而实现电子设备的输入和输出功能，具体此处不做限定。
接口单元1408为外部装置与电子设备1400连接的接口。例如，外部装 置可以包括有线或无线头戴式耳机端口、外部电源(或电池充电器)端口、有线或无线数据端口、存储卡端口、用于连接具有识别模块的装置的端口、音频输入/输出(I/O)端口、视频I/O端口、耳机端口等等。接口单元1408可以用于接收来自外部装置的输入(例如，数据信息、电力等等)并且将接收到的输入传输到电子设备1400内的一个或多个元件或者可以用于在电子设备1400和外部装置之间传输数据。
存储器1409可用于存储软件程序以及各种数据。存储器1409可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作***、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等；存储数据区可存储根据手机的使用所创建的数据(比如音频数据、电话本等)等。此外，存储器1409可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。处理器1410是电子设备的控制中心，利用各种接口和线路连接整个电子设备的各个部分，通过运行或执行存储在存储器1409内的软件程序和/或模块，以及调用存储在存储器1409内的数据，执行电子设备的各种功能和处理数据，从而对电子设备进行整体监控。处理器1410可包括一个或多个处理单元；可选的，处理器1410可集成应用处理器和调制解调处理器，其中，应用处理器主要处理操作***、用户界面和应用程序等，调制解调处理器主要处理无线通信。可以理解的是，上述调制解调处理器也可以不集成到处理器1410中。
电子设备1400还可以包括给各个部件供电的电源1411(比如电池)，可选的，电源1411可以通过电源管理***与处理器1410逻辑相连，从而通过电源管理***实现管理充电、放电、以及功耗管理等功能。
可选的，本公开实施例还提供一种电子设备，包括处理器1410，存储器1409，存储在存储器1409上并可在所述处理器1410上运行的计算机程序，该计算机程序被处理器1410执行时实现上述手势识别方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
本公开实施例还提供一种计算机可读存储介质，计算机可读存储介质上存储有计算机程序，该计算机程序被处理器执行时实现上述手势识别方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。 其中，所述的计算机可读存储介质，如只读存储器(Read-Only Memory，简称ROM)、随机存取存储器(Random Access Memory，简称RAM)、磁碟或者光盘等。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本公开的技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本公开各个实施例所述的方法。
上面结合附图对本公开的实施例进行了描述，但是本公开并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本公开的启示下，在不脱离本公开宗旨和权利要求所保护的范围情况下，还可做出很多形式，均属于本公开的保护之内。

Claims (20)

  1. 一种手势识别方法，包括：
    对包括指点物的多个二维图像中的每个二维图像进行处理，获取所述指点物在所述每个二维图像中的成像尺寸；
    依据所述指点物在每个二维图像中的成像尺寸，从所述多个二维图像中确定至少一个目标二维图像；
    利用所述至少一个目标二维图像，确定并执行对应的指令。
  2. 根据权利要求1所述的手势识别方法，其中，所述指点物包括手指，所述对包括指点物的多个二维图像中的每个二维图像进行处理，获取所述指点物在所述每个二维图像中的成像尺寸，包括：
    确定所述二维图像中包括的手指对应的连通区域；
    基于手指的几何特征对所述手指进行指尖定位，获取指尖坐标；
    基于所述指尖坐标确定所述指尖对应的连通区域宽度。
  3. 根据权利要求1所述的手势识别方法，其中，所述指点物在所述目标二维图像中的成像尺寸处于第一预设尺寸区间内。
  4. 根据权利要求3所述的手势识别方法，其中，所述第一预设尺寸区间为至少两个预设尺寸区间中的预设尺寸区间，该至少两个预设尺寸区间中的任意两个预设尺寸区间间隔分布。
  5. 根据权利要求4所述的手势识别方法，其中，所述多个二维图像还包括与所述至少两个预设尺寸区间中的第二预设尺寸区间对应的至少一个二维图像；
    所述利用所述至少一个目标二维图像，确定并执行对应的指令具体包括：
    利用与所述第一预设尺寸区间对应的至少一个目标二维图像确定并执行对应的指令。
  6. 根据权利要求4所述的手势识别方法，其中，所述至少两个预设尺寸区间中，成像尺寸数值越大的预设尺寸区间，对应的区间长度越大。
  7. 根据权利要求4所述的手势识别方法，其中，所述手势识别方法还包括：
    根据预设尺寸区间与可控对象集合之间的映射关系，确定与所述第一预设尺寸区间对应的目标可控对象集合；
    显示所述目标可控对象集合中的可控对象。
  8. 根据权利要求7所述的手势识别方法，其中，所述显示所述目标可控对象集合中的可控对象，包括：
    在所述目标可控对象集合不同于当前显示的可控对象集合时，将当前的第一显示界面更新为包括所述目标可控对象集合中的可控对象的第二显示界面。
  9. 根据权利要求3所述的手势识别方法，其中，所述手势识别方法还包括：
    获取参考二维图像；
    根据所述参考二维图像，确定所述第一预设尺寸区间。
  10. 根据权利要求9所述的手势识别方法，其中，所述第一预设尺寸区间为(W-dw，W+dw)，所述W为所述指点物在所述参考二维图像中的成像尺寸，dw为所述第一预设尺寸区间的区间长度阈值。
  11. 根据权利要求9所述的手势识别方法，其中，所述参考二维图像包括：
    电子设备本次启动后初次获取的二维图像；或者
    获取时间与电子设备上次获取到的二维图像的获取时间的间隔超过预定时间门限的二维图像。
  12. 一种电子设备，包括：
    第一获取模块，用于对包括指点物的多个二维图像中的每个二维图像进行处理，获取所述指点物在所述每个二维图像中的成像尺寸；
    第一确定模块，用于依据所述指点物在每个二维图像中的成像尺寸，从所述多个二维图像中确定至少一个目标二维图像；
    执行模块，用于利用所述至少一个目标二维图像，判断并执行对应的指令。
  13. 根据权利要求12所述的电子设备，其中，所述指点物在所述目标二维图像中的成像尺寸处于第一预设尺寸区间内。
  14. 根据权利要求13所述的电子设备，其中，所述第一预设尺寸区间为至少两个预设尺寸区间中的预设尺寸区间，该至少两个预设尺寸区间中的任意两个预设尺寸区间间隔分布。
  15. 根据权利要求14所述的电子设备，其中，所述多个二维图像还包括与所述至少两个预设尺寸区间中的第二预设尺寸区间对应的至少一个二维图像；
    所述执行模块，具体用于：
    利用与所述第一预设尺寸区间对应的至少一个目标二维图像确定并执行对应的指令。
  16. 根据权利要求14所述的电子设备，其中，所述至少两个预设尺寸区间中，成像尺寸数值越大的预设尺寸区间，对应的区间长度越大。
  17. 根据权利要求14所述的电子设备，其中，所述电子设备还包括：
    第二确定模块，用于根据预设尺寸区间与可控对象集合之间的映射关系，确定与所述第一预设尺寸区间对应的目标可控对象集合；
    显示模块，用于显示所述目标可控对象集合中的可控对象。
  18. 根据权利要求17所述的电子设备，其中，所述显示模块，用于：
    在所述目标可控对象集合不同于当前显示的可控对象集合时，将当前的第一显示界面更新为包括所述目标可控对象集合中的可控对象的第二显示界面。
  19. 根据权利要求13所述的电子设备，其中，所述电子设备还包括：
    第二获取模块，用于获取参考二维图像；
    第三确定模块，用于根据所述参考二维图像，确定所述第一预设尺寸区间。
  20. 根据权利要求19所述的电子设备，其中，所述第一预设尺寸区间为(W-dw，W+dw)，所述W为所述指点物在所述参考二维图像中的成像尺寸，dw为所述第一预设尺寸区间的区间长度阈值。
PCT/CN2018/107214 2017-09-26 2018-09-25 手势识别方法及电子设备 WO2019062682A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP18861362.4A EP3690605A4 (en) 2017-09-26 2018-09-25 GESTURE RECOGNITION PROCESS AND ELECTRONIC DEVICE
US16/340,497 US10866649B2 (en) 2017-09-26 2018-09-25 Gesture identification method and electronic device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710882327.X 2017-09-26
CN201710882327.XA CN109558000B (zh) 2017-09-26 2017-09-26 一种人机交互方法及电子设备

Publications (1)

Publication Number Publication Date
WO2019062682A1 true WO2019062682A1 (zh) 2019-04-04

Family

ID=65861989

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/107214 WO2019062682A1 (zh) 2017-09-26 2018-09-25 手势识别方法及电子设备

Country Status (4)

Country Link
US (1) US10866649B2 (zh)
EP (1) EP3690605A4 (zh)
CN (1) CN109558000B (zh)
WO (1) WO2019062682A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108230383B (zh) * 2017-03-29 2021-03-23 北京市商汤科技开发有限公司 手部三维数据确定方法、装置及电子设备
US11270531B2 (en) 2019-06-28 2022-03-08 GM Cruise Holdings, LLC Autonomous vehicle data management platform
CN113946206A (zh) * 2020-07-17 2022-01-18 云米互联科技(广东)有限公司 家电交互方法、家电设备及存储介质
CN113946205B (zh) * 2020-07-17 2024-02-09 云米互联科技(广东)有限公司 燃气控制方法、燃气设备及存储介质
US20230196823A1 (en) * 2021-12-16 2023-06-22 Nanjing Easthouse Electrical Co., Ltd. Finger vein sensors
JP2023139535A (ja) * 2022-03-22 2023-10-04 キヤノン株式会社 ジェスチャ認識装置、頭部装着型表示装置、ジェスチャ認識方法、プログラム、および記憶媒体

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102508547A (zh) * 2011-11-04 2012-06-20 哈尔滨工业大学深圳研究生院 基于计算机视觉的手势输入法构建方法及***
CN103135756A (zh) * 2011-12-02 2013-06-05 深圳泰山在线科技有限公司 生成控制指令的方法和***
CN103440033A (zh) * 2013-08-19 2013-12-11 中国科学院深圳先进技术研究院 一种基于徒手和单目摄像头实现人机交互的方法和装置

Family Cites Families (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1148411A3 (en) * 2000-04-21 2005-09-14 Sony Corporation Information processing apparatus and method for recognising user gesture
US7227526B2 (en) * 2000-07-24 2007-06-05 Gesturetek, Inc. Video-based image control system
JP3996015B2 (ja) * 2002-08-09 2007-10-24 本田技研工業株式会社 姿勢認識装置及び自律ロボット
US7623731B2 (en) * 2005-06-20 2009-11-24 Honda Motor Co., Ltd. Direct method for modeling non-rigid motion with thin plate spline transformation
EP2296079A3 (en) * 2005-10-26 2011-04-13 Sony Computer Entertainment Inc. System and method for interfacing with a computer program
US9910497B2 (en) * 2006-02-08 2018-03-06 Oblong Industries, Inc. Gestural control of autonomous and semi-autonomous systems
US8472665B2 (en) * 2007-05-04 2013-06-25 Qualcomm Incorporated Camera-based user input for compact devices
TW200928892A (en) * 2007-12-28 2009-07-01 Wistron Corp Electronic apparatus and operation method thereof
US8818027B2 (en) * 2010-04-01 2014-08-26 Qualcomm Incorporated Computing device interface
JP2011258159A (ja) * 2010-06-11 2011-12-22 Namco Bandai Games Inc プログラム、情報記憶媒体及び画像生成システム
EP2620849B1 (en) * 2010-09-22 2019-08-28 Shimane Prefectural Government Operation input apparatus, operation input method, and program
JP5439347B2 (ja) * 2010-12-06 2014-03-12 日立コンシューマエレクトロニクス株式会社 操作制御装置
US9189068B2 (en) * 2011-03-14 2015-11-17 Lg Electronics Inc. Apparatus and a method for gesture recognition
US8840466B2 (en) * 2011-04-25 2014-09-23 Aquifi, Inc. Method and system to create three-dimensional mapping in a two-dimensional game
US9329673B2 (en) * 2011-04-28 2016-05-03 Nec Solution Innovators, Ltd. Information processing device, information processing method, and recording medium
CN202075718U (zh) 2011-05-17 2011-12-14 东华大学 基于颜色识别的手指多点触控***
JP2012248067A (ja) * 2011-05-30 2012-12-13 Canon Inc 情報入力装置、その制御方法、および制御プログラム
US8881051B2 (en) * 2011-07-05 2014-11-04 Primesense Ltd Zoom-based gesture user interface
US9268406B2 (en) * 2011-09-30 2016-02-23 Microsoft Technology Licensing, Llc Virtual spectator experience with a personal audio/visual apparatus
CN103376890B (zh) * 2012-04-16 2016-08-31 富士通株式会社 基于视觉的手势遥控***
TWI454966B (zh) * 2012-04-24 2014-10-01 Wistron Corp 手勢控制方法及手勢控制裝置
US9323352B1 (en) * 2012-10-23 2016-04-26 Amazon Technologies, Inc. Child-appropriate interface selection using hand recognition
JP6202810B2 (ja) * 2012-12-04 2017-09-27 アルパイン株式会社 ジェスチャ認識装置および方法ならびにプログラム
CN103926999B (zh) * 2013-01-16 2017-03-01 株式会社理光 手掌开合手势识别方法和装置、人机交互方法和设备
US9274607B2 (en) * 2013-03-15 2016-03-01 Bruno Delean Authenticating a user using hand gesture
US20140282274A1 (en) * 2013-03-15 2014-09-18 Qualcomm Incorporated Detection of a gesture performed with at least two control objects
CN103440035A (zh) * 2013-08-20 2013-12-11 华南理工大学 一种三维空间中的手势识别***及其识别方法
US20150084859A1 (en) * 2013-09-23 2015-03-26 Yair ITZHAIK System and Method for Recognition and Response to Gesture Based Input
US9412012B2 (en) * 2013-10-16 2016-08-09 Qualcomm Incorporated Z-axis determination in a 2D gesture system
PL2921936T3 (pl) * 2014-03-22 2019-09-30 Monster & Devices Home sp. z o.o. Metoda i aparatura do sterowania urządzeniem za pomocą gestów
JP6603024B2 (ja) * 2015-02-10 2019-11-06 任天堂株式会社 情報処理プログラム、情報処理装置、情報処理システム、および、情報処理方法
US9848780B1 (en) * 2015-04-08 2017-12-26 Google Inc. Assessing cardiovascular function using an optical sensor
JP2017059103A (ja) * 2015-09-18 2017-03-23 パナソニックＩｐマネジメント株式会社 判定装置、判定方法、判定プログラムおよび記録媒体

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102508547A (zh) * 2011-11-04 2012-06-20 哈尔滨工业大学深圳研究生院 基于计算机视觉的手势输入法构建方法及***
CN103135756A (zh) * 2011-12-02 2013-06-05 深圳泰山在线科技有限公司 生成控制指令的方法和***
CN103440033A (zh) * 2013-08-19 2013-12-11 中国科学院深圳先进技术研究院 一种基于徒手和单目摄像头实现人机交互的方法和装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3690605A4 *

Also Published As

Publication number Publication date
US10866649B2 (en) 2020-12-15
EP3690605A4 (en) 2021-12-15
EP3690605A1 (en) 2020-08-05
CN109558000B (zh) 2021-01-22
CN109558000A (zh) 2019-04-02
US20190243462A1 (en) 2019-08-08

Similar Documents

Publication Publication Date Title
WO2019062682A1 (zh) 手势识别方法及电子设备
US20210150171A1 (en) Object recognition method and mobile terminal
WO2019154181A1 (zh) 显示控制方法及移动终端
WO2019174629A1 (zh) 图像处理方法及柔性屏终端
WO2019174628A1 (zh) 拍照方法及移动终端
WO2020042727A1 (zh) 一种应用场景的交互方法和移动终端以及存储介质
CN108989672B (zh) 一种拍摄方法及移动终端
WO2019179332A1 (zh) 应用程序的关闭方法及移动终端
WO2019206036A1 (zh) 消息管理方法及终端
AU2014312569A1 (en) Multi display method, storage medium, and electronic device
WO2019184947A1 (zh) 图像查看方法及移动终端
WO2019120192A1 (zh) 文本编辑方法及移动终端
WO2019196691A1 (zh) 一种键盘界面显示方法和移动终端
WO2019174541A1 (zh) 移动终端的操作方法及移动终端
WO2019114522A1 (zh) 屏幕控制方法、屏幕控制装置及移动终端
WO2021036603A1 (zh) 应用程序的控制方法及终端
WO2019154360A1 (zh) 界面切换方法及移动终端
WO2021036623A1 (zh) 显示方法及电子设备
CN109495616B (zh) 一种拍照方法及终端设备
CN108881721B (zh) 一种显示方法及终端
WO2019120190A1 (zh) 拨号方法及移动终端
CN110908517B (zh) 图像编辑方法、装置、电子设备及介质
EP3905037B1 (en) Session creation method and terminal device
JP7221305B2 (ja) 対象物認識方法及び移動端末
US11886643B2 (en) Information processing apparatus and information processing method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18861362

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2018861362

Country of ref document: EP

Effective date: 20200428