WO2022046047A1

WO2022046047A1 - Skin interface for wearables: sensor fusion to improve signal quality

Info

Publication number: WO2022046047A1
Application number: PCT/US2020/047958
Authority: WO
Inventors: Matteo CARRARA; Yao Ding
Original assignee: Google Llc
Priority date: 2020-08-26
Filing date: 2020-08-26
Publication date: 2022-03-03

Abstract

The present disclosure provides devices and methods for determining an input command based on a gesture of a user. The device may be a wearable device having one or more accelerometers configured to receive input and/or detect movement of the wearable device. The input may be a mechanical wave created by the gesture. For example, the gesture may be a swipe gesture or a tap gesture on the skin in the region of the device. Based on the detected movement of the device and/or the vibration of the accelerometers, the type of gesture may be determined. The type of gesture may be used to determine an input command.

Description

SKIN INTERFACE FOR WEARABLES: SENSOR FUSION TO IMPROVE SIGNAL QUALITY BACKGROUND

[0001] Wearable and hearable devices have inputs that allow a user to provide commands. For example, when the device is a pair of earbuds, the earbuds may have inputs that allow the user to start or stop content, adjust the playback volume, or fast forward or rewind content. These inputs may be a touchpad that receives a physical input or a microphone that receives a voice command. A physical input, such as a touch or a tap, may provide undesirable noise or discomfort for the user. Moreover, a touch or a tap on may affect the performance of the antenna as it may be in the same location or nearby the touch area.

BRIEF SUMMARY

[0002] The present disclosure provides systems and methods for determining an input command based on a gesture of a user using one or more accelerometers within the wearable device. The gesture may be a swipe gesture or a tap gesture performed on the skin of the body of the user in a region near a wearable device. The gesture may create a mechanical wave that propagates through the portion of a body of the user between an input region and the wearable device. The one or more accelerometers may detect movement of the device due to the mechanical wave and determine a type of gesture based on the detected movement.

[0003] One aspect of the disclosure includes a wearable electronic device comprising one or more accelerometers and one or more processors in communication with the one or more accelerometers. The one or more processors may be configured to receive, at the one or more accelerometers, an input based on a gesture of a user on a region of the user’s skin near the wearable electronic device, detect, based on the received input, a movement of the device, and determine, based on the movement of the device, an input command. At least one of the one or more accelerometers may be an internal measurement unit (“IMU”) accelerometer. At least one of the one or more accelerometers may be a voice accelerometer. The gesture may be a swipe gesture or a tap gesture.

[0004] The one or more processors may be further configured to compare the detected movement of the device and one or more stored waveforms, and determine, based on the comparison of the detected movement and the one or more stored waveforms, a type of gesture. The input may be a mechanical wave that propagates through a portion of a body of the user between an input region and the wearable device. The mechanical wave may be based on an external force exerted on the body of the user. The one or more processors may be further configured to determine, based on the mechanical wave, a type of the gesture.

[0005] Determining the input command may comprise applying a machine learning model. The one or more processors may be further configured to perform a function corresponding to the determined input command.

[0006] Another aspect of the disclosure includes a method comprising receiving, at each of one or more accelerometers of a wearable device, an input based on a gesture of a user on a region of the user’s skin near the wearable electronic device, detecting, by one or more processors based on the received input, a movement, and determining, by the one or more processors based on the movement of the device, an input command.

[0007] Yet another aspect of the disclosure provides for a non-transitory computer-readable medium storing instructions, which when executed by one or more processors, cause the one or more processors to receive, at the one or more accelerometers, an input based on a gesture of a user on a region of the user’s skin near the wearable device, detect, based on the received input, a movement of the device, and determine, based on the movement of the device, an input command.

BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A is a pictorial diagram of an example device according to aspects of the disclosure.

[0009] Figure IB is a functional block diagram of a device system in accordance with aspects of the disclosure.

[0010] Figure 2 is a pictorial diagram illustrating an example use of the device according to aspects of the disclosure.

[0011] Figure 3 is a pictorial diagram illustrating an example use of the device according to aspects of the disclosure.

[0012] Figure 4A is a pictorial diagram illustrating an example use of the device according to aspects of the disclosure.

[0013] Figure 4B is a graphical representation illustrating the example use of the device shown in Figure 4A according to aspects of the disclosure.

[0014] Figure 4C is a graphical representation illustrations the comparison of a mechanical wave detected by an accelerometer and audio input received by one or more microphones according to aspects of the disclosure.

[0015] Figure 5A is a pictorial diagram illustrating an example use of the device according to aspects of the disclosure.

[0016] Figure 5B is a graphical representation illustrating the example use of the device shown in Figure 5A according to aspects of the disclosure.

[0017] Figure 6A is a pictorial diagram illustrating an example use of the device according to aspects of the disclosure.

[0018] Figure 6B is a graphical representation illustrating the example use of the device shown in Figure 6A according to aspects of the disclosure.

[0019] Figure 7 is a flow diagram illustrating a method of determining a gesture of a user according to aspects of the disclosure.

DETAILED DESCRIPTION

[0020] The wearable or hearable device may use one or more accelerometers that are already within the device to determine an input command based on a gesture of a user. Wearable and hearable devices may include, for example, earbuds, smartwatches, audio visual and/or virtual reality headsets, smart glasses, etc. The one or more accelerometers within the device may receive, as input, a mechanical wave created by a gesture of the user. In some examples, the accelerometers may detect movement of the device due to the gesture of the user. The gesture may be, by way of example only, a swipe or a tap gesture on the skin in a region around the device. In some examples, the gesture may be an external force exerted on the region of the body of the user near the wearable device. The mechanical wave created by friction between the user’s finger(s) as the user swipes or taps the skin near the device may be received as input by the one or more accelerometers. Additionally or alternatively, the one or more accelerometers may detect movement of the device caused by the mechanical wave created by a user swiping or tapping their finger(s) on the skin. In examples where the device is a pair of earbuds, the user may provide a gesture on the skin of an ear wearing the earbud or the skin on the face of the user near the ear. In examples where the device is a smartwatch, the user may provide a gesture on the skin near the wrist, forearm, or hand in a region near the smartwatch.

[0021] The accelerometers may receive the mechanical wave as input and/or may detect a movement and/or vibration of the device due to the gesture. Based on the received input and/or detected movement, the device may determine a type of gesture. The type of gesture may be a swipe gesture, such as a vertical or sideways swipe, a tap gesture, or the like. The accelerometer may output a signal corresponding to the type of gesture. For example, one or more processors in communication with the accelerometer may receive measurements from the accelerometer and determine the type of gesture that was received as input based on the measurements. In turn, the one or more processors may cause a corresponding function to be performed.

[0022] Enabling receipt of input gestures on the skin in a region near the device may provide for an improved user experience. A gesture input on the skin of the user in the region of the device provides a larger surface for the user to enter the input. This may allow for adaptive use for a plurality of users. For example, the larger surface area available for input facilitates use of the wearable device by people lacking fine motor skills. Additionally, the larger surface for the gesture input may make it easier for the user to provide an up-down swipe, left-right swipe, or a tapping gesture without having to worry about where to start the gesture on the device. According to some examples, a gesture input on the skin in the region near the device may prevent any loud or uncomfortable sounds in the ear canal as compared to when the gesture input is provided on the device itself. Moreover, a gesture input on the skin in the region near the device may reduce the risk of moving the device. In examples where the device is an earbud, the gesture input on the skin may reduce the risk of moving the earbud in the user’ s ear, which may be uncomfortable.

[0023] Figure 1A illustrates an example system 100A in which the features described herein may be implemented. It should not be considered limiting the scope of the disclosure or usefulness of the features described herein. In this example, system 100A may include a wearable device 130. The wearable device 130 may be a device that is capable of detecting and/or receiving audio input using one or more microphones. For example, the wearable device may be earbuds, a smartwatch, a headset, smart glasses, a VR/AR headset, etc. As shown, the wearable device 130 is a pair of earbuds 110, 120.

[0024] Figure IB illustrates an example system 100B in which the features described above and herein may be implemented. In this example, system 100B may include wearable devices 110, 120. Wearable device 110 may contain one or more processors 114, memory 116, instructions 111, data 119, one or more accelerometers 118, and one or more microphones 115.

[0025] The one or more processors 114 may be any conventional processors, such as commercially available microprocessors. Alternatively, the one or more processors may be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware-based processor. Although Figure IB functionally illustrates the processor, memory, and other elements of wearable device 110 as being within the same block, it will be understood by those of ordinary skill in the art that the processor, computing device, or memory may actually include multiple processors, computing devices, or memories that may or may not be stored within the same physical housing. Similarly, the memory may be a hard drive or other storage media located in a housing different from that of wearable device 110. Accordingly, references to a processor or computing device will be understood to include references to a collection of processors or computing devices or memories that may or may not operate in parallel.

[0026] Memory 116 may store information that is accessible by the processors, including instructions 111 that may be executed by the processors 114, and data 119. The memory 116 may be a type of memory operative to store information accessible by the processors 114, including a non-transitory computer-readable medium, or other medium that stores data that may be read with the aid of an electronic device, such as a hard-drive, memory card, read-only memory ("ROM"), random access memory ("RAM"), optical disks, as well as other write-capable and read-only memories. The subject matter disclosed herein may include different combinations of the foregoing, whereby different portions of the instructions 101 and data 119 are stored on different types of media.

[0027] Memory 116 may be retrieved, stored or modified by processors 114 in accordance with the instructions 111. For instance, although the present disclosure is not limited by a particular data structure, the data 119 may be stored in computer registers, in a relational database as a table having a plurality of different fields and records, XML documents, or flat files. The data 119 may also be formatted in a computer-readable format such as, but not limited to, binary values, ASCII or Unicode. By further way of example only, the data 119 may be stored as bitmaps comprised of pixels that are stored in compressed or uncompressed, or various image formats (e.g., JPEG), vector-based formats (e.g., SVG) or computer instructions for drawing graphics. Moreover, the data 119 may comprise information sufficient to identify the relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memories (including other network locations) or information that is used by a function to calculate the relevant data.

[0028] The instructions 111 can be any set of instructions to be executed directly, such as machine code, or indirectly, such as scripts, by the processor 114. In that regard, the terms “instructions,” “application,” “steps,” and “programs” can be used interchangeably herein. The instructions can be stored in object code format for direct processing by the processor, or in any other computing device language including scripts or collections of independent source code modules that are interpreted on demand or compiled in advance. Functions, methods and routines of the instructions are explained in more detail below. [0029] The wearable device 110 may further include one or more accelerometers 118. According to one example, the accelerometers may be voice accelerometers. A voice accelerometer may have high bandwidth capabilities, high quality signal, and/or robust external noise rejection. The accelerometers may be part of an inertial measurement unit (“IMU”). The IMU accelerometers may have robust internal and external noise rejection and/or a low power requirement.

[0030] The gesture of the user may provide a mechanical wave that propagates through the portion of a body of the user between an input region and wearable device 110. The accelerometers 118 may receive the mechanical wave as input. For example, the accelerometers 118 may detect movement of the wearable device 110 due to the mechanical wave. According to some examples, the mechanical wave may cause the accelerometers 118 to vibrate. Based on the detected movement of the wearable device 110 and/or vibration of the accelerometers 118, the device may determine the type of gesture provided by the user.

[0031] According to some examples, the mechanical wave received by accelerometers 118 may be compared against details of previously identified or determined gestures using a trained machine learning model to determine the type and/or direction of the gesture. The machine learning model may be trained to determine the type and/or direction of the gesture. Each training example may consist of a gesture provided by the user. The input features to the machine learning model may include the time the mechanical wave is received by each accelerometer, the vibration intensity of each accelerometer, the speed at which the user provides the gesture, the direction and/or location of the gesture, etc. The machine learning model may use the input features to more accurately detect the type and/or direction of the gesture. The output of the machine learning model may be a detected type and/or direction of the gesture provided by the user. In some examples, the device may request feedback from the user. For example, the user may be asked whether the detected type and/or direction of the gesture is accurate. The user may provide feedback, such as a yes or no, indicating that the detected type and/or direction of the gesture is accurate.

[0032] The microphones 115 may be able to receive audio input. The audio input may be the sound created by a user tapping or swiping on the skin near the wearable device 110. For example, as the user swipes the skin near wearable device 110, friction between the skin in the region near the device and the object being used to swipe the skin may create an audible sound. The audio input received by the microphones 115 may be compared to the input received by the accelerometers 118 to determine or confirm the type of gesture.

[0033] The device 110 may compare the mechanical wave received by the accelerometer 118 with the audio input received by the microphone 115. The comparison may confirm whether the mechanical wave received by the accelerometers 118 was due to a gesture input from the user. For example, the machine learning model may be used to differentiate between inputs and non-inputs. Non-inputs may be a user nodding their head, walking, talking, head scratching, etc. The machine learning model may use the comparisons between the input received by the accelerometers 118 and microphones 115 as an input feature to more accurately detect the gesture of a user. [0034] Wearable device 120 may include one or more processors 124, memory 126, instructions 121, data 129, one or more accelerometers 128, and one or more microphones 125 that are substantially similar to those described herein with respect to wearable device 110.

[0035] Figure 2 illustrates a user wearing the wearable device. User 240 may be wearing at least one wearable device 210. For example, the user 240 may be wearing earbud 210. Earbud 210 may have a housing shaped to be worn on a human body and, more specifically, within the inner portion of ear 242. The housing may include a first surface shaped to be in contact with ear 242 and at least one second surface 215 shaped to be exposed when worn on the body. The earbud 210 may include accelerometers 218A, 218B located within the housing. The accelerometers 218A, 218B may receive mechanical waves created by the input gesture of the user . In some examples, the accelerometers 218A, 218B may detect movement of earbud 210 caused by the gesture.

[0036] The accelerometers 218A, 218B may be located at the top of earbud 210 and bottom of earbud 210. In some examples, the top of earbud 210 may be the considered the area or region of the earbud 210 that is closest to the helix of the ear 242 and the bottom of earbud 210 may be the area or region of the earbud 210 that is closest to the lobule, or ear lobe, of ear 242. However, the top and bottom of the earbud 210 may be relative and, therefore, may be used for descriptive purposes only. However, the accelerometers 218A, 218B may be located anywhere within the housing of the earbud 210 and, therefore, the accelerometers 218A, 218B shown at the top and bottom of the earbud 210 is only one example and is not intended to be limiting. Additionally or alternatively, while only two accelerometers 218A, 218B are shown, there may be only one accelerometer or more than two accelerometers. Therefore, only having two accelerometers 218 A, 218B shown is merely one example and is not intended to be limiting.

[0037] The gesture may be provided in a touch region near the wearable device. For example, a first touch region 244 may be on the ear 242 of user 240. A second touch region 246 may be on the face of the user 240, such as the flat area below the temple. While the touch regions 244, 246 are shown as rectangular, the touch regions can be of any shape or size. For example, the touch regions may be oblong, polygonal, circular, irregular, etc. and may take up more or less room on the user’s ear 242 or face than shown in Figure 2B. The touch regions may, additionally or alternatively, be located on different regions of the user’s ear 242, face, or other nearby areas. Thus, touch regions 244, 246 are only one example and are not meant to be limiting.

[0038] The touch regions may provide a larger surface area for a user to provide an input command as compared to a surface area for touch input on the device. For example, the touch input on earbud 210 may be on the second surface 215 of earbud 210. By providing a gesture input to a region of skin near the wearable device, any negative effects of touching the device itself or issues with providing an input command on a small surface may be negated.

[0039] Figure 3 illustrates an example where the wearable device is a smartwatch. Smartwatch 310 may be worn on the wrist 350 of a user. Smartwatch 310 may include all the components as described above with respect to wearable devices 110, 120, 210. In particular, smartwatch 310 may include one or more accelerometers 318 A, 318B within the housing of the smartwatch. While the accelerometers 318 A, 318B are shown as being located internally at the left and right side of the smartwatch 310, the accelerometers 318A, 318B may be located anywhere within the housing of the smartwatch 310.

[0040] The touch regions 344, 346 may be located on the wrist 350 of the user and/or on the back of the hand 348 of the user. As shown, the touch regions 344, 346 may be ellipses. However, as described above, the touch regions 344, 346 may be any shape and size.

[0041] Figure 4A illustrates an example where the input gesture is a tap. A tap gesture may produce a distinctive signature, or mechanical wave. For example, a tap gesture may have a certain oscillation frequency, duration, signal to noise ratio, etc. as compared to a swipe gesture. In some examples, the tap gesture may have a certain range of expected oscillation frequencies, durations, signal to noise ration, etc. According to some examples, the oscillation frequency of a tap gesture may be approximately 100Hz or less. In other examples, the oscillation frequency may be between 25Hz and 125Hz, between 35Hz and 155Hz, etc. In some examples, the duration of the tap may be approximately 50ms to 60ms. The duration may be more or less than 60ms, such as between 25ms and 70ms, 45ms and 80ms, etc. The signal to noise ratio may be approximately 30dB RMS or less. However, in some examples the signal to noise ratio may be between 20dB RMS and 40dB RMS, between 30dB RMS and 50dB RMS, etc. Thus, the example oscillation frequency of 100Hz or less, duration of 50ms to 60ms, and signal to noise ratio of 30dB RMS or less is merely one set of examples and is not intended to be limiting.

[0042] According to some examples, the oscillation frequency, duration, signal to noise ratio, etc. may be user dependent. The machine learning model may use the oscillation frequency, duration, signal to noise ratio as input to train the model specific to a user. For example, the machine learning model may learn that the user has a tap gesture with an oscillation frequency of approximately 90Hz, a duration of approximately 55ms, and a signal to noise ratio of approximately 33dB RMS. The machine learning model may compare the received input to the known approximate values to predict whether the gesture was a tap gesture.

[0043] A user 440 may use one or more fingers 448 to provide a tap gesture in touch region 446. Additionally or alternatively, the user 440 may provide the tap gesture in touch region 444. The tap gesture may create a mechanical wave 450A that propagates through user 440, including ear 442. The mechanical wave 450A may vibrate the accelerometers 418A, 418B within earbud 410. In some examples, the accelerometers 418 A, 418B may detect movement of the earbud 410 due to the mechanical wave 450A. The device may determine the type of gesture based vibration of the accelerometers 418 A, 418B and/or the detected movement of earbud 410. For example, the mechanical wave may have a different shape based on the input gesture. In some examples, the mechanical wave may have a different magnitude, duration, frequency, etc. based on the input gesture. The characteristics of the mechanical waves may be stored with information corresponding to a type of gesture, an input command, etc. When a new input gesture is received, the mechanical wave of the new gesture may be compared to the stored mechanical waves to determine which stored mechanical wave the new input most closely matches. When a match is found, the stored corresponding information may identify the type of gesture and be used to determine an associated function to be performed by the device.

[0044] The gesture may be associated with an input command. For example, a tap may be an input command to start or stop content from being output. According to some examples, a tap may answer or hang up a phone call. Based on the determined gesture, the device may output a signal corresponding to the determined gesture. The signal may cause the device to perform the associated input command.

[0045] Figure 4B illustrates a graphical representation of the accelerometers receiving the mechanical wave associated with the tap gesture. The graph 400B may illustrate the mechanical wave received by the accelerometer as a function of amplitude versus time. For example, the graph 400B may illustrate two distinct and/or separate mechanical waves 450B, 450C that were received by accelerometers 418A, 418B. The two mechanical waves 450B, 450C may indicate that the user performed two tap gestures in a touch region 444, 446. For example, mechanical waves 450B, 450C may be short wave forms, indicating that the duration of the mechanical wave 450B, 450C may have been short in duration. Wave 450B may have started approximately at time 1.675s and ended at approximately 1.72s. Therefore, wave 450B may have lasted approximately 45ms. Wave 450C may have started approximately at time 1.84s and ended at approximate time 1.875s. Therefore, wave 450C may have lasted approximately 35ms. The short duration of the waves 450B, 450C may indicate that the gesture is a tap gesture.

[0046] Figure 4C is a graphical illustration comparing the mechanical wave detected by the accelerometers and the audio input received by the one or more microphones. The graphical illustration 400C includes graph 460 and graph 462. Graph 460 illustrates the input received by the accelerometers. For example, the accelerometers may detect movement of the wearable device due to the input gesture and/or may receive the mechanical wave caused by the gesture as input. The detected movement and/or received input may be represented graphically as a function of amplitude, or waveform, and time, as shown in graph 460. Graph 462 illustrates the audio input received by the one or more microphones. The audio input may include the sound created by the input gesture as well as any external noises, such as the user talking, ambient sounds, etc. The audio input may be represented graphically as a function of frequency and time, as shown in graph 462.

[0047] According to some examples, the device may compare the mechanical wave form received by the accelerometers with the audio input received by one or more microphones. The accelerometers may detect movement and/or vibration of the device. However, not all the movement is due to an input gesture. For example, the accelerometers may detect movement of the device as the user is walking, talking, chewing, etc. The microphones may receive audio input caused the by the input gesture as well as from the user talking, chewing, etc. The device may compare the input received by the accelerometers with the audio input received by the microphones to confirm and/or determine whether the input received by the accelerometers corresponds to an input gesture. For example, windows 464, 466 identify regions of graphs 460, 462 in which the input received by the accelerometers, identified as corresponding to a gesture input, aligns with audio input received by the microphones, also identified as corresponding to the gesture input. The comparison between the input received by the accelerometer and the input received by the microphones may confirm whether an input gesture occurred.

[0048] According to some examples, the audio input may be processed separately from the mechanical wave received by the accelerometers. By processing the different types of waves separately, the device may compare the different waves and confirm that the mechanical wave received by the accelerometers is a tap gesture.

[0049] The machine learning model may use both the audio input received by the microphones and the mechanical wave received by the accelerometers as input to the model. According to some examples, when the audio input and the mechanical wave both identify the gesture as a tap gesture, the audio input and the mechanical wave may be used as input for the machine learning model as having positively identified a tap gesture. The machine learning model may use these examples to predict the type of gesture. Additionally or alternatively, the machine learning model may use positively identified gestures to differentiate between an intended gesture, such as a tap, as compared to an unintended gesture, such as a user chewing.

[0050] Figure 5A illustrates an example where the input gesture is a swipe. The swipe gesture is shown as a sideways swipe, such as a swipe from front to back or back to front. In other examples, the swipe may be in different directions, such as a vertical swipe, and may correspond to other input commands. In the context of earbud 510, the front may be closer to the user’s nose or mouth and the back may be closer to the user’s ear 542. In the context of a smartwatch, the front may be closer to the user’s hand and the back may be closer to the user’ s elbow.

[0051] The accelerometers 518A, 518B may detect a swipe gesture of one or more fingers 548 of the user in touch region 546 and/or touch region 444. For example, the swipe gesture may create a mechanical wave 550A that propagates through user 540, including ear 542. The propagation of mechanical wave 550A may vibrate the accelerometers 518A, 518B within earbud 510. In some examples, the accelerometers 518A, 518B may detect movement of the earbud 510 due to the propagation of mechanical wave 550A.

[0052] A swipe gesture may produce extended mechanical waves with a longer duration, as compared to a tap gesture. For example, the duration of the swipe gesture may be greater than 60ms, such as 82ms, 125ms, 212ms, etc. In some examples the duration of the swipe gesture may be less than 30ms. Therefore, the example of 60ms is merely one example and is not intended to be limiting. According to some examples, the swipe gesture may have a longer time signature as compared to a tap gesture. Additionally or alternatively, the swipe gesture may have a wider frequency band as compared to a tap gesture.

[0053] The earbud 510 may determine the type of gesture based on the vibration of the accelerometers 518A, 518B and/or the detected movement of earbud 510. The process for determining a swipe gesture may be similar to the process for determining a tap gesture, described above.

[0054] The gesture may correspond to an input command. For example, a front to back or back to front swipe may be an input command to fast-forward or rewind the content, change the song that is playing, swipe through notifications, etc. Based on the determined gesture, the accelerometers may output a signal to the one or more processors. The signal may correspond to the determined gesture. The signal may cause the device to perform a function corresponding to the input command associated with the gesture.

[0055] Figure 5B illustrates a graphical representation of the accelerometers receiving the mechanical wave associated with a sideways swipe gesture. The graph 500B may illustrate the mechanical wave received by the accelerometer as a function of frequency versus time. For example, the graph 500B may illustrate three distinct and/or separate mechanical waves 550B, 550C, 550D.

[0056] The duration of the mechanical wave may be an indication of the type of gesture. For example, 550B may have started approximately at time 1.85s and ended at approximately 2.05s. Therefore, wave 550B may have lasted, or had a duration of, approximately 0.20s. Wave 440C may have lasted approximately 0.10s and wave 550D may have lasted approximately 0.19s. The example tap gesture waves 450B, 450C had a shorter duration than the sideways swipe gesture waves 550B, 550C, 550D. For example, wave 450B had a duration of approximately 45ms and wave 450C had a duration of approximately 35ms whereas wave 550B had a duration of approximately 0.2s, wave 550C had a duration of approximately 0.10s, and wave 550D had a duration of approximately 0.19s. Accordingly, the duration of the waveform, along with other features such as shape, magnitude, frequency, or the like, may indicate the type of gesture input on the user’ s skin in the region near the device.

[0057] Figure 6A illustrates an example where the input gesture is a swipe. In this example, the swipe gesture may be a vertical swipe, such as a swipe from top to bottom or bottom to top. In the context of earbud 610, the top may be closer to the user’s forehead or helix and the bottom may be closer to the user’s earlobe or mouth.

[0058] A vertical swipe may be received and processed similar to a sideways swipe. The accelerometers may detect gesture input from one or more fingers 648 of the user in a swiping motion in touch region 646 and/or touch region 644. The swipe gesture may create a mechanical wave 650A that propagates through user 640, including ear 642. The accelerometers may receive mechanical wave 650A as input. In some examples, mechanical wave 650A may vibrate the accelerometers 518A, 518B within earbud 510 and/or the accelerometers 618A, 618B may detect movement of the earbud 610 due to mechanical wave 650A.

[0059] The earbud 610 may determine the type of gesture based on the vibration of the accelerometers 618A, 618B and/or the detected movement of earbud 610. The process for determining a swipe gesture may be similar to the process for determining a tap gesture or sideways gesture, described above.

[0060] The gesture may correspond to an input command. For example, a vertical swipe may be an input command to increase or decrease the playback volume. Based on the determined gesture, the device may output a signal corresponding to the determined gesture. The signal may cause the device to perform the associated input command.

[0061] Figure 6B illustrates a graphical representation of the accelerometers receiving the mechanical wave associated with a vertical swipe gesture. The graph 600B may illustrate the mechanical wave received by the accelerometer as a function of frequency versus time. For example, the graph 600B may illustrate two distinct and/or separate mechanical waves 650B, 650C. Similar to the sideways swipe gesture described above in connection with Figs. 5A-B, the vertical swipe gesture may have a longer duration than the tap gesture described above in connection with Figs. 4A-B. The vertical swipe gesture may have a similar duration as a sideways swipe gesture. A vertical swipe may have a different shape, magnitude, duration, frequency, etc. as compared to a sideways swipe. The device may store waveforms of past vertical swipes and horizontal swipes. The stored waveforms may include characteristics pertaining to the shape, magnitude, duration, frequency, etc. The stored waveforms may be associated with a type of gesture, an input command, etc. When the vertical and/or sideways swipe is received, the mechanical wave of the new gesture may be compared to the stored mechanical waves to determine whether the new gesture is a vertical swipe or a sideways swipe.

[0062] Figure 7 illustrates an example method of determining a gesture input using one or more accelerometers of a wearable device. The following operations do not have to be performed in the precise order described below. Rather, various operations can be handled in a different order or simultaneously, and operations may be added or omitted.

[0063] For example, in block 710 the device may receive an input from a gesture of a user. The input may be a mechanical wave created by gesture of the user and received by each of the one or more accelerometers.

[0064] In block 720, the device may determine, based on the received input, a movement of the device. For example, the accelerometers may detect movement of the device caused by the mechanical wave. Additionally or alternatively, the accelerometers may vibrate due to the mechanical wave.

[0065] In block 730, the device may determine, based on the movement of the device, an input command. For example, a mechanical wave with a short duration may be a tap gesture. A short duration may be a duration of 60ms or less. In some examples, a mechanical wave with a longer duration, such as a duration between 60ms and 0.75s, may be a swipe gesture. The swipe gesture may be a vertical swipe and/or a sideways swipe. According to some examples, a vertical swipe, such as a downward swipe, may be an input command, such as to decrease the playback volume of the content. An upward swipe may be an input command, such as to increase the playback volume of the content. In some examples, a sideways swipe may be an input command to fast-forward or rewind the content. A tap may be an input command start or stop content from being output. According to some examples, a tap may answer or hang up a phone call. Therefore, the types of input commands described herein, such as increasing volume, decreasing volume, fast-forwarding, rewinding, stopping, and playing, are merely examples and are not meant to be limiting.

[0066] Unless otherwise stated, the foregoing alternative examples are not mutually exclusive, but may be implemented in various combinations to achieve unique advantages. As these and other variations and combinations of the features discussed above can be utilized without departing from the subject matter defined by the claims, the foregoing description of the embodiments should be taken by way of illustration rather than by way of limitation of the subject matter defined by the claims. In addition, the pro vision of the examples described herein, as well as clauses phrased as "such as," "including" and the like, should not be interpreted as limiting the subject matter of the claims to the specific examples; rather, the examples are intended to illustrate only one of many possible embodiments. Further, the same reference numbers in different drawings can identify the same or similar elements.

Claims

1. A wearable electronic device, comprising: one or more accelerometers; and one or more processors in communication with the one or more accelerometers, the one or more processors configured to: receive, at the one or more accelerometers, an input based on a gesture of a user on a region of the user’s skin near the wearable electronic device; detect, based on the received input, a movement of the wearable electronic device; and determine, based on the movement of the device, an input command.

2. The wearable electronic device of claim 1, wherein at least one of the one or more accelerometers is an inertial measurement unit (“IMU”) accelerometer.

3. The wearable electronic device of claim 1, wherein at least one of the one or more accelerometers is a voice accelerometer.

4. The wearable electronic device of claim 1, wherein the gesture is a swipe gesture or a tap gesture.

5. The wearable electronic device of claim 1, wherein the one or more processors are further configured to: compare the detected movement of the device and one or more stored waveforms; and determine, based on the comparison of the detected movement and the one or more stored waveforms, a type of gesture.

6. The wearable electronic device of claim 1, wherein the input is a mechanical wave that propagates through a portion of a body of the user between an input region and the wearable electronic device.

7. The wearable electronic device of claim 6, wherein the mechanical wave is based on an external force exerted on the body of the user.

8. The wearable electronic device of claim 6, wherein the one or more processors are further configured to determine, based on the mechanical wave, a type of the gesture.

9. The wearable electronic device of claim 1, wherein determining the input command comprises applying a machine learning model.

10. The wearable electronic device of claim 1, wherein the one or more processors are further configured to perform a function corresponding to the determined input command.

11. A method, comprising : receiving, at each of one or more accelerometers of a wearable electronic device, an input based on a gesture of a user on a region of the user’s skin near the wearable electronic device; detecting, by one or more processors based on the received input, a movement; and determining, by the one or more processors based on the movement of the wearable electronic device, an input command.

12. The method of claim 11, wherein at least one of the one or more accelerometers is an inertial measurement unit (“IMU”) accelerometer or a voice accelerometer.

13. The method of claim 11, wherein the gesture is a swipe gesture or a tap gesture.

14. The method of claim 11, further comprising: comparing, by one or more processors, the detected movement and one or more stored wave forms; and determine, by one or more processors based on the comparison between the detected movement and the one or more stored waveforms, a type of gesture.

15. The method of claim 11, wherein the input is a mechanical wave that propagates through a portion of a body of the user between an input region and the wearable device.

16. The method of claim 15, further comprising determining, by the one or more processors based on the mechanical wave, a type of the gesture.

17. The method of claim 11, further comprising performing, by the one or more processors, a function corresponding to the determined input command.

18. The method of claim 11, wherein determining the input command further comprises applying, by the one or more processors, a machine learning model.

19. A non-transitory computer-readable medium storing instructions, which when executed by one or more processors, cause the one or more processors to: receive, at one or more accelerometers, an input based on a gesture of a user on a region of the user’s skin near a wearable electronic device; detect, based on the received input, a movement of the wearable electronic device; and determine, based on the movement of the wearable electronic device, an input command.

20. The non-transitory computer-readable medium of claim 19, wherein at least one of the one or more accelerometers is an inertial measurement unit accelerometer or a voice accelerometer.