CN106095203B - Sensing touches the calculating device and method that sound is inputted as user gesture - Google Patents
Sensing touches the calculating device and method that sound is inputted as user gesture Download PDFInfo
- Publication number
- CN106095203B CN106095203B CN201610578907.5A CN201610578907A CN106095203B CN 106095203 B CN106095203 B CN 106095203B CN 201610578907 A CN201610578907 A CN 201610578907A CN 106095203 B CN106095203 B CN 106095203B
- Authority
- CN
- China
- Prior art keywords
- gesture
- sound
- instruction
- touch
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/043—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using propagating acoustic waves
- G06F3/0436—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using propagating acoustic waves in which generating transducers and detecting transducers are attached to a single acoustic waves transmission substrate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/041—Indexing scheme relating to G06F3/041 - G06F3/045
- G06F2203/04108—Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Acoustics & Sound (AREA)
- Human Computer Interaction (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The present invention relates to sensings to touch the technical field that sound is inputted as user gesture, specifically discloses a kind of calculating equipment that sensing touch sound is inputted as user gesture comprising: processor and at least one sonic sensor；At least one described sonic sensor and the processor communication, and the touch sound corresponding with gesture for being configured as to sense is converted to electric signal；The processor determines the wave character of electric signal according to the electric signal, and the wave character by analyzing the electric signal identifies that the gesture of user is inputted as user gesture.The present invention is to provide a kind of sensing schemes of sound based on solid transmitting, even if not direct touch apparatus, can also sense the sound for touching other objects indirectly.The sound that the present invention senses does not depend on air transmitting, not vulnerable to the interference of environmental noise.In the occasion of certain inconvenient directly touches devices, the present invention can control for equipment and provide greater flexibility.
Description
Technical field
The present invention relates to calculate equipment technical field, more particularly to the technology that sensing touch sound is inputted as user gesture
Field.
Background technique
It calculates equipment and various input equipments offer inputs, such as keyboard, mouse, trace ball, control stick, touch is provided
The input equipments such as screen, button.Wherein, touch screen is more common input equipment.Touch screen allow user using finger, stylus or
Other objects are at the position indicated by the user interface shown by display equipment by touch sensor panel come to meter
It calculates equipment and various types of inputs is provided.In general, touch screen can recognize touch event and touch event in touch sensor
Position on panel, and the computing system display content that then occurs when can be occurred according to touch event explains touch thing
Part, and one or more movements then can be executed based on touch event.Touch sensor panel is configured generally to use
Capacitive character input detects the touch of finger from the user.This touch screen can only receive the touch in touchscreen senses region.
In the prior art, there are also a kind of input equipments can be sensed after touch-control surface connects pressure sensor by power
Come detection gesture and touch screen input.When user is pressed with certain gestures, the pressure sensor of distribution visits this
It surveys.According to the different pressures numerical value that each pressure sensor obtains, " Center of Pressure " can be determined by calculation.Center of Pressure can be with
Touch-control input is used cooperatively, but pressure sensing area is not necessarily installed together with capacitance touching control region.When pressure sensor position
Except active touch input area domain, the input of certain user interface can be made to move to outside touchscreen senses region and carried out.
Obviously, the input equipment of above-mentioned touch screen one kind needs user directly to contact and calculates equipment.But in some cases
Under, the inconvenient directly contact of possible user calculates equipment, needs when thinking the alarm of mobile phone or fruit washing on closing table such as early morning
Answering cell phone incoming call etc..
For wearable electronic, volume is too small may not to have touch screen.Voice input is a kind of more satisfactory input
Mode.Voice input is considered as one of most easy, most easy-to-use input method, does not need user and connects with equipment is calculated
Touching.Voice input is that voice is switched to voice signal by microphone, is identified by computing system, explains speech events, and
Input is provided to equipment is calculated.But since human speech outwardly transmits sound wave by air, microphone acquires voice letter
Number, other noise jamming vulnerable to air dielectric transmitting.When environmental noise is too big, it may cause speech recognition and mistake occur.
Summary of the invention
The technical problem to be solved in the present invention is that providing a kind of calculating that sensing touch sound is inputted as user gesture
Device and method, even if can also enable calculating equipment pass through sensing in the case where user is not directly contacted with and calculates equipment and touch
The mode of sound obtains user gesture input, while identification human speech can also be overcome to input the defect vulnerable to interference.
In order to solve the above technical problems, the technical solution adopted by the present invention is that:
A kind of sensing touches the calculating equipment that inputs as user gesture of sound, comprising: processor, transaudient interface and at least
One sonic sensor；
The transaudient interface is connect at least one described sonic sensor, is configured as transmitting touch corresponding with gesture
Sound is at least one described sonic sensor；
At least one described sonic sensor and the processor communication, and be configured as to sense corresponding with gesture
Touch sound be converted to electric signal；
The processor determines the wave character of electric signal according to the electric signal, passes through the wave character of the electric signal
Identify that the gesture of user is inputted as user gesture.
Preferably, the transaudient interface is a part of the sonic sensor or the transaudient interface is the meter
Calculate a part of the shell of equipment.
Preferably, when the transaudient interface is configured as the transaudient interfacial contact other objects, transaudient interface can be transmitted
User touches the sound issued when other objects and/or user and directly touches transaudient interface when the transaudient interface surface
It can make a sound.
Preferably, the transaudient interface is equipped with one or several protrusion/sunk structures and/or the shell for calculating equipment
Body surface face is equipped with one or several protrusion/sunk structures.
Preferably, at least one described sonic sensor be measure vibration acceleration acceleration transducer or piezoelectricity it is thin
Film sensors.
In order to solve the above technical problems, another technical solution used in the present invention is:
A method of sensing touches sound and inputs as user gesture, includes the following steps:
Step 1, configuring computing devices make comprising: processor and at least one sonic sensor；
Step 2, the touch sound that touches sound and will sense is sensed by least one described sonic sensor
Be converted to electric signal；
Step 3, the electric signal transmitted by the processor according at least one described sonic sensor, determines electric signal
Wave character identifies that the gesture of user is defeated as the user gesture for calculating equipment by the wave character of the electric signal
Enter.
Preferably, the gesture of the user and the corresponding sound that touches use binary coding model split for gesture/sound
Sound coding, the gesture/acoustic coding include: ordinary instruction coding, emergency instruction coding and special instruction coding；
It is further comprising the steps of after step 3, the classification of gesture/acoustic coding is determined by the processor；
It is encoded if it is ordinary instruction, it is determined that whether ordinary instruction input state activates, and has activated if detected
Ordinary instruction input state then executes the ordinary instruction, terminates if detecting un-activation ordinary instruction input state；
The emergency instruction is then executed if it is emergency instruction coding；
If it is specific function instruction encoding, it is determined whether the authorization of specific function instruction input is obtained, if detection
To having authorized specific function instruction input then to execute specific function instruction, if detecting unauthorized specific function instruction input
Then terminate.
Preferably, further comprising the steps of before, during or after the execution step 3:
The calculating equipment detects whether that there are also touch screens to input, and if there is touch screen inputs, the processor is preferentially located
Manage touch screen input signal.
In order to solve the above technical problems, another technical solution that the present invention uses is:
A kind of mobile computer that sensing touch sound is inputted as user gesture, comprising: processor and processor communication
Touch screen, and at least one sonic sensor with processor communication；
The touch screen is configured as sensing user's finger near to or in contact with the touch action of touch screen, and will sense
The touch action is converted to touch screen input signal；
At least one described sonic sensor is configured as sensing touch sound corresponding with gesture, and the institute that will be sensed
It states touch sound and is converted to electric signal；
The processor executes the corresponding instruction of touch action according to the touch screen input signal；
The processor determines the wave character of electric signal according to the electric signal, passes through the wave character of the electric signal
It identifies that the gesture of user is inputted as user gesture, and executes the corresponding instruction of user gesture.
Preferably, the mobile computer further include: transaudient interface；
The transaudient interface is connect at least one described sonic sensor, is configured as transmitting touch corresponding with gesture
Sound is at least one described sonic sensor.
The present invention is to provide a kind of method for sensing of sound based on solid transmitting, even if not direct touch apparatus,
The sound for touching other objects indirectly can be sensed.
The sonic sensor that the present invention uses detects acoustic signals, and the vibration that will be propagated in solid using the method for contact
Turn changes electric signal into.
The sound that the present invention senses does not depend on air transmitting, not vulnerable to the interference of environmental noise.Certain inconvenient direct
The occasion of touches device, the present invention can control for equipment and provide greater flexibility.
Detailed description of the invention
Fig. 1 is top and the bottom plan view for the calculating equipment that present invention sensing touches sound as user gesture and inputs.
Fig. 2 is the simplified block diagram of the calculating equipment of Fig. 1.
Fig. 3 is the simplification cross-sectional view of the calculating equipment of the line S-S interception in Fig. 1.
Fig. 4 A is the schematic diagram that user touches sound to the input of calculating equipment indirectly.
Fig. 4 B corresponds to the calculating equipment drawing of Fig. 4 A.
Fig. 4 C is indirect touch voice input schematic diagram.
Fig. 5 A is point touching gesture schematic diagram.
Fig. 5 B is sliding touch gestures schematic diagram.
Fig. 6 A another kind point touching gesture schematic diagram.
Fig. 6 B adjusts gesture schematic diagram.
Fig. 7 is the top plan view from the calculating equipment of touch screen input signal.
Fig. 8 is the flow chart for detecting the method that user gesture inputs for using sonic sensor.
Fig. 9 is the touch voice input schematic diagram that wearable type calculates equipment.
Specific embodiment
Sound is the sound wave generated by object vibration, and sound wave is mechanical wave.Sound in the form of wave by medium (air,
Solid or liquid) Vibration propagation.The sound that human ear is heard is normally from the sound wave of air transmitting.Therefore, people are usually very
Few sound wave for directly hearing object transfer.In fact when we touch object, even slight touch, it can also be generated because of vibration
Sound, this kind of sound wave can be transmitted in the object that we touch, and can be sensed, such as with ear close to touched
When body surface.Moreover, the skin and body of people can also generate sound because of touching and transmit sound wave.
The present invention using touching object can generate sound and object can transmit the principle of sound wave (object includes common solid
The body and skin of substance and people, but not limited to this), sound is touched by sensing, the wave character of signal is analyzed, determines hand
Gesture.Generally, the spread speed of sound wave in solids is faster than air, can transmit sound wave mutually between the object of contact.The present invention mentions
What is supplied is a kind of sensing scheme of sound based on solid transmitting, even if not direct touch apparatus, can also sense and touch it indirectly
The sound of its object.The sound that the present invention senses does not depend on air transmitting, not vulnerable to the interference of environmental noise.In certain inconveniences
The occasion of direct touches device, the present invention can control for equipment and provide greater flexibility.
Turning now to attached drawing, the calculating equipment including touch screen will be discussed in more detail.Fig. 1 be include transaudient interface 102
With the top and bottom view of the calculating equipment 100 of sonic sensor 110.Calculating equipment 100 can be including touch screen input
The substantially any type of electronic equipment of mechanism such as touch screen 104 or other touch key-press and associated component.For example,
Calculate equipment 100 can be laptop computer, tablet computer, smart phone, digital music player, portable game machine,
Wearable smart machine etc..
Calculating equipment 100 may include transaudient interface 102, the shell 106 at least partially surrounding sonic sensor 110, touch
Screen 104 and/or one or more input buttons 108.The encapsulating of shell 106 calculates one or more components of equipment 100, can also enclose
Calculating equipment 100 is fixed to around a part of sonic sensor 110 and/or by a part of sonic sensor 110.One or more
A input button 108 can provide input function to equipment 100 is calculated.For example, the adjustable calculating equipment 100 of input button 108
Volume opens or closes and calculates equipment 100, or can provide other inputs to calculate equipment 100.In addition, calculating equipment 100 also
It may include one or more receiving port (not shown).Receiving port can receive one or more plugs or connector, such as but
It is not limited to universal serial bus (USB), cable connector etc..Transaudient interface 102 can be a part of sonic sensor, can also
To be a part for calculating device housings 106 or other components.
Sonic sensor 110, which is configured as sensing, may be applied to transaudient interface 102 and/or its with transaudient interfacial contact
The touch sound sensed is converted to electric signal by the touch sound of his body surface.It should be pointed out that sonic sensor can
To be operably connected to a variety of different surfaces or element that calculate transmitting in equipment and touch sound.Therefore, to for sound
The discussion of any specific embodiment of wave sensor is meant only to be exemplary, and is not intended to be limited.
Continue to refer to figure 1, in another embodiment, sonic sensor 110 and the back shell 106 for calculating equipment 100
Connection, sonic sensor be directly connected to calculate equipment shell, sensing may be applied to calculate equipment shell and/or with calculating
The touch sound of other body surfaces of the shell contact of equipment.
In a specific example, sonic sensor 106 can be electromagnetism transform class, such as electromagnetic type；It is also possible to quiet
Electric transformation classes, such as piezo-electric type；Or resistance conversion class, such as impedance changing type；Or light-to-current inversion class, as light quantity changes
Type.The acceleration transducer of measurement vibration acceleration, the vibration or impact of energy detection object can be used in sonic sensor, and exports
Corresponding signal.
Another piezoelectric film sensor, using novel high polymer piezoelectric material (such as Kynoar PVDF), structure letter
List, high sensitivity, bandwidth, it is very sensitive to " dynamic stress ", it is capable of measuring faint signal.Its acoustic impedance and tissue sound
Impedance is very close, is more suitable for the sensor of wearable device sensing sound or vibration, can be used for acquiring faint touch sound
Sound signal.Moreover, as needed, piezoelectric film sensor can adopt hard substrates and the different structure design such as hollow.
Sonic sensor is the sensor of sense vibrations, and sonic sensor should be chosen according to detection.No matter adopt
With any sonic sensor, the signal of acquisition can be used to determine amplitude, waveform and the frequency of vibration.Specifically, sound
Wave sensor can use the sensor, but be not limited to these sensors.
Calculating equipment 100 may include the one or more components that can be communicated with one another.Fig. 2 is the simplification frame for calculating equipment 100
Figure.With reference to Fig. 2, calculating equipment 100 may also include power supply 120, input/output interface 122, processor 124, one or more and deposit
Memory component 126 and/or one or more sensors 128.
Processor 124 can be the substantially any electronic equipment for being capable of handling, receiving and/or transmitting instruction.For example, place
Reason device 124 can be microprocessor or microcomputer.As described herein, term " processor " be intended to single processor or
Processing unit, multiple processors or multiple processing units or other appropriately configured computing elements.For example, electronic equipment 100
Alternative pack can be controlled by first processor and the other component of electronic equipment 100 can be controlled by second processor, wherein the
One processor and second processor may or may not communicate with one another.
Memory 126 can store the electronic data that can be utilized by electronic equipment 100.For example, memory 126 can store correspondence
In the electric data or content, such as audio file, video file, document files etc. of various application programs.
In addition to be configured as sensing touch sound sonic sensor 110 other than, electronic equipment 100 may also include one or
Multiple sensors 128.Substantially any type of input can be provided to electronic equipment 100 by sensor 128.For example, sensor
128 can be one or more accelerometers, gyroscope, optical sensor, imaging sensor, force snesor or sense speech
Microphone etc..
Power supply 120, which can be, to provide the substantially any equipment of energy to equipment 100 is calculated.For example, power supply 120 can
To be battery.
Input/output interface 122 can be configured to the trip communication for providing with calculating equipment 100.For example, input/output
Interface 122 can be conducive to be carried out by calculating equipment with plurality of devices/source trip communication.For example, input/output interface 122
Data can be received from the control button etc. in user, calculating equipment 100.In addition, input/output interface 122 can also drive from outside
It is dynamic to receive data and transmit data, the data such as universal serial bus (USB) or other video/audio/numbers to external drive
According to input.
It should be pointed out that Fig. 1 and Fig. 2 are merely exemplary.In other instances, electronic equipment may include than Fig. 1 and
The less or more component of those components shown in Fig. 2.In addition it is shown that electronic equipment be only combined with transaudient interface
102 and sonic sensor 110 calculating equipment an example.
Transaudient interface 102 and sonic sensor 110 will be discussed in more detail now.Fig. 3 is the calculating equipment intercepted in Fig. 1
100 S-S cross-sectional view.As shown in figure 3, touch screen 104 and shell 106 contact, the touch sound on touch screen 104 can lead to
It crosses shell 106 and is transferred to transaudient interface 102 and sonic sensor 110.
As shown in figure 3, sonic sensor 110 can be operatively attached to substrate or circuit board 132 and/or with substrate or
Circuit board 132 communicates.Substrate or circuit board 132 can provide sonic sensor 110 and calculate one or more components of equipment 100
Between communication, the one or more component such as, but not limited to processor 124, memory 126 and power supply 120.
Transaudient interface 102, which is configured as can produce, touches sound (for example, direct touch based on user's finger) or can
To receive voice input (for example, user's finger is touched with the object of transaudient interfacial contact or from input equipment from other objects
Touch sound) and to sonic sensor 110 transmit touch sound.
Touch screen 104 may include display screen.Display screen is configured as showing that the one or more for calculating equipment 100 is defeated
Image and/or video out.Display screen can be substantially any type of indication mechanism, liquid crystal display (LCD), etc. from
Daughter display etc..
Sensing touches the calculating equipment that sound is inputted as user gesture indirectly, as shown in Figure 4 A, calculates equipment 100 and places
On desktop 152, the side of close desktop 152, transaudient interface 102 is contacted with desktop 152.User is another desktop 152
Side cannot directly be touched and calculate equipment 100, but can touch desktop 152.
It calculates equipment and is provided with alarm clock.With reference to Fig. 4 B, the time of alarm clock setting is arrived, quarter-bell sounds, and calculates equipment 100
The display of touch screen 104 needs user to input " closing " or " delay " instruction, and touch screen 104 prepares the touch screen of detection finger 172
Input signal.
When quarter-bell starting, prepare detection touch sound with the sonic sensor 110 that apparatus casing 106 is connect is calculated.Ginseng
Fig. 4 C is examined, in the other side of desktop with gesture touch desktop 152, the sonic sensor 110 communicated with processor 124 senses user
To touch sound corresponding with gesture, the touch sound sensed is converted to electric signal, processor 124 by sonic sensor 110
The wave character of electric signal is determined based on the received electrical signal, and then identifies user gesture.If it is determined that one or more
A gesture represents " closings " instruction, it is determined that gesture input " closing " instructs.If it is determined that one or more gestures represent
" delay " instruction, it is determined that gesture input " delay " instruction, instruction is referring to gesture/acoustic coding.
The method that sound is inputted as user gesture is touched by sensing, is substantially exactly to be acquired by sonic sensor
Electric signal identifies different gesture/sound according to the wave character of electric signal corresponding with gesture, and then detects input
Calculate the user gesture of equipment.
Touch gestures/voice recognition calculates:
Sonic sensor prepares detection and touches voice input, and user carries out gesture input behaviour with direct or indirect touch manner
Make.In the processor side, receives the sonic sensor and collect electric signal in the gesture operation.Either click touching
It touches or sliding touches, the vibration signal that two kinds of gestures generate all is dynamic random signal.But what point touching gesture generated
It is transient oscillation signal, what sliding touch gestures generated is approximate stable random vibration signal.
Entire calculate mainly includes three steps: signal end detection, Signal Pretreatment, feature extraction and identification.
First step: signal end detection
The purpose of signal end detection is to extract effective signal data, and the present embodiment uses for reference speech sound signal terminal point inspection
The processing method of survey, using the double threshold end-point detecting method of short-time energy and short-time zero-crossing rate.
One higher average energy thresholding T is set1, for determining the beginning for touching sound, then according to ambient noise
To determine a slightly lower thresholding T2, for determining that the first order touches the end point of sound.Wherein T2=aEn, EnFor ambient noise
The average energy of section completes first order judgement.Equally, according to the Average zero-crossing rate Z of noisenOne thresholding T is set3=bZn,
For judging to touch sound beginning and end.The value of coefficient a, b in formula can be analyzed to obtain reasonable value according to test statistics.
Second step: Signal Pretreatment
After the endpoint that signal has been determined, the signal data of the sonic sensor of corresponding period is extracted, signal is carried out and locates in advance
Reason.
Calculate root-mean-square value RMS, calculation formula are as follows:
For the ease of analysis, data are subjected to differential transformation, difference value calculation formula are as follows:
Δyi=| yi+1-yi|
In above-mentioned two formula: yiFor output data of the sensor on ith sample point, N is the number of sampled data.
When due to carrying out voice input with touch gestures, it is understood that there may be the interference of various background noises, sensor are acquired
The signal arrived is not singly finger click/sliding action signal, but the superposition of various signals.It therefore is that can effectively extract letter
Number feature, it is necessary to first collected signal is filtered, smoothly pre-processed.
There are many kinds of the methods of filtering, such as median filtering algorithm, sliding window algorithm, Kalman filtering algorithm, can root
According to needing using any one or other filtering algorithms.
Third step: feature extraction and identification
In general, sliding the vibrational energy meeting of touch when touching some position of the same object with different gestures with finger
Lower than the vibrational energy of point touching, this is determined by the motion mode of finger.And the time that sliding touches is longer than click
The time of touch, this is to determine the time contacted by finger with object.
The interference of environment/ambient noise and the erroneous judgement that occurs in order to prevent, current embodiment require that by judging environment/back
That is, by the variable condition of signal data under statistics stationary state, and filtering threshold is arranged in the situation of change of scape noise according to this
Range, to determine whether to belong to effective identification maneuver.
But different objects being touched, such as touch desktop or touch skin, the setting of threshold value may be because of object not
It is same and different.Therefore, user should be analyzed by test statistics and set appropriate threshold value.If desired, classifier can be used
Classification and Identification is carried out to gesture.
Slide the identification of sound:
It uses root-mean-square value RMS as characteristic parameter, is because root-mean-square value RMS can represent the vibrational energy of sampling periods.
It can given threshold T by test statistics analysisgIf meeting threshold condition RMS < Tg, can determine that slide, otherwise
It is non-slip.
Effective sampling number N can be set by test statistics analysisSIf extracting the vibration signal of signal corresponding period
Sampling number N > NS, can determine that effectively to slide, otherwise to slide in vain.
In the present embodiment, when sample frequency is 100Hz, NSIt is set to 50.The time threshold for sliding sound is set as
0.5S, effectively sliding sound duration length should be greater than 0.5S.
The identification of click sound:
Use difference value as the single magnitude for clicking sound peaks signal of detection.
Appropriate peak detection threshold upper limit value Δ can be set by test statistics analysisphWith bottom threshold value Δpl, just
The peak delta Y to click signal can be achievedxIt is detected.
If meeting threshold condition Δpl< Δ Yx< Δph, can determine that Δ YxFor peak value；
If Δ Yx< Δpl, can determine that Δ YxFor non-peak；
If Δ Yx> Δph, can determine that Δ YxFor interference signal.
Sampling number N can be set by test statistics analysisminAnd NmaxIf the sampling number N in effective peak sample area is full
Sufficient condition Nmin< N < Nmax, can determine that as effective peak, be otherwise invalid peak value.
The time width threshold value T between 2 adjacent peaks can be set by test statistics analysist, when determine there are 2 or
When 3 effective peaks, if time width M >=T of all adjacent 2 peak valuest, then can determine that the double-click or click for click
Three double hits.If the time width M < T of any one adjacent 2 peak valuet, then can determine that as invalid clicks.
In the present embodiment, Tt200ms can be set to.That is the time threshold of adopting consecutive click chemical reaction movement is set as 0.2S, effectively connects
Continuous click action time interval should be greater than 0.2S.
Final gesture/voice recognition:
The signal that certain period extracts is analyzed, and is judged to effectively sliding, and effective peak is not present, and is finally confirmed as sliding
Touch gestures/sound.
The signal that certain period extracts is analyzed, and determines that there are 1 effective peaks, are finally confirmed as single point touching hand
Gesture/sound (unrelated with slide deciding result).
The signal that certain period extracts is analyzed, and determines that there are 2 effective peaks, and the double-click for being judged to clicking, final true
Think 2 point touching gesture/sound.
The signal that certain period extracts is analyzed, and determines that there are 3 effective peaks, and three double hits for being judged to clicking, finally
It is confirmed as 3 click gesture/sound.
Certain period extract signal through analyzing, when determine exist when 4 or 4 or more n effective peaks, if own
The time width M of adjacent 2 peak valuesK< Tt, finally it is confirmed as adjusting gesture/sound.The time width M of peak valueKVariation tendency
Adjusting trend is represented, the value 1 of k arrives n-1.Such as the time width M of peak valueKIt is elongated, represent adjusting trend " becoming larger from small " or " from
It secretly brightens ".
With reference to Fig. 5 A, it is of short duration touch sound accordingly that finger 172, which clicks body surface 154 with movement S1 from top to bottom,
Sound, the transient oscillation signal of generation certainly exist an effective peak.The result of identification should be single click touch gestures/sound
Sound.
With reference to Fig. 5 B, finger 172 is slided with the movement S2 moved horizontally in body surface 154, is that the time is slightly long accordingly
Touch sound.Effective peak is not present in the random vibration signal of generation.The result of identification should be sliding touch gestures/sound.
It is related with the movement of finger 172 to touch sound, it is also related with the texture structure on 154 surface of object.It is alternatively possible
" point touching " gesture/sound, with reference to Fig. 6 A.By the variation of object surface structure/texture, finger 172 is in 154 table of object
A protrusion/sunk structure 192 is encountered, signal will generate a peak value to act the process that S2 is slided in face.As long as and determining
There are 1 effective peak, the result of identification is exactly single click touch gestures/sound.
Gesture/sound is identified according to the wave character of electric signal, but is not limited to some specific gesture or movement.Its
Its possible point touching gesture/sound can realize (such as thumb and index finger) by the quick touching of two fingers；And other can
Sliding touch gestures/sound of energy, then can be realized by the sliding friction of two fingers.
With reference to Fig. 6 B, finger 172 acts the process of S2 sliding on 154 surface of object, encounters multiple protrusion/sunk structures
192, electric signal continuously generates multiple effective peaks, and the time width of adjacent 2 peak values is related with sliding speed.If adjacent 2
The time width of a peak value meets threshold condition, and the result of identification should adjust gesture/sound.
This protrusion/sunk structure can according to need, and be arranged on the surface for needing touching object, such as transaudient interface
On surface of shell that is upper or calculating equipment.
Gesture/acoustic coding:
Specifically, binary coding mode can be used in touch gestures and the corresponding sound that touches, to increase the reliable of input
Property.Each coding is made of several touch gestures/sound, and number of encoding bits are determined by user.
Point touching gesture/sound is with " " representative；Sliding touch gestures/sound is represented with "-".It is gesture/sound below
Coding and its representative instruction signal:
I class, ordinary instruction coding:
Switch "-", can be used for page turning, switching；
Suspend " .. ", can be used for music, video, game, reading, delay quarter-bell；
It closes " ... ", can be used for music, video, game, reading, quarter-bell, call；
Open "-- ", it can be used for hands-free incoming call answering, take pictures, turn on light；
Class ii, emergency instruction coding:
Morse code SOS " ...---... ", put through alarm call for help.
Group iii, special instruction coding:
Into certain news program, voice plays news "-- "；
Into music program, play music " ..-- "；
Into recorded program, start to record "-..-"；
Into (lamp) facility control program, open (lamp) "-- "；
Above-mentioned binary coding method is only schematical, rather than limits coding mode.For example, if by auxiliary sounding
Device inputs sound by different tones, carries out audio signal identification, can take more complicated coding method.
As shown in fig. 7, mobile computer is configured as user's input of at least two types of detection.Equipment 100 is calculated to set
Alarm clock is set.The time of alarm clock setting is arrived, quarter-bell sounds, and the display of touch screen 104 for calculating equipment needs user's input " to close
Close " or " delay " instruction, the touch screen input signal of the preparation detection finger of touch screen 104.
When quarter-bell starting, prepare sensing touch sound with the sonic sensor 110 that apparatus casing 106 is connect is calculated.Ginseng
Fig. 7 is examined, user senses touch screen input signal with the Close button of 174 point touching screen 104 of finger, touch screen 104.Together
When, the gesture of 174 point touching screen 104 of finger generates corresponding touch sound, since touch screen 104 is contacted with shell 106, with
The sonic sensor 110 that processor 124 communicates can sense the touch sound of the transmitting of shell 106.Equipment 100 is calculated to determine whether
Capacitive character input, if there is capacitive character inputs, according to touch screen input preferential principle processing.If touch screen 104 is in suspend mode
State can input from touch screen 104 and touch sound.
It discusses now a kind of for being detected using sonic sensor 110 to the method for calculating the input of equipment 100.Fig. 8 shows
The flow chart for using the method 200 for the sonic sensor for calculating equipment 100 is gone out.Method 200 can begin at operation 202 simultaneously
And sonic sensor 110 can sense touch sound.For example, when touching shell 106 when the user clicks, it can be by sonic sensor
110 sense touch sound.The touch sound sensed can be converted to electric signal by sonic sensor 110.
Once detecting the input for touching sound, this method may continue to proceed to operation 204 and true by processor 124
Determine gesture/acoustic coding, gesture/acoustic coding is to the gesture of user and the corresponding sound that touches using binary coding mode
Or made of other coding modes divide.The step of determining gesture/acoustic coding specifically includes according at least one described sound wave
The electric signal that sensor transmits determines the wave character of electric signal, identifies user's by the wave character of the electric signal
Gesture is as the user gesture input for calculating equipment.
When determining gesture/acoustic coding, during or after, method 200 can continue to operation 206, and count
Calculating equipment 100 can be determined whether that there are also capacitive characters to input.In other words, calculate equipment 100 can determine except touch voice input it
The touch screen input whether sensed outside.If user provides input on the touch screen 104 of equipment 100, calculate
Capacitive character input can be detected in equipment 100.
If detecting that capacitive character inputs, method 200 can continue to operation 208, handle touch screen input signal.
For example, as shown in fig. 7, touching can be due to the generation of 172 point touching screen of finger.After operation 208, method 200 can
Continue to end state 222.
If capacitive character input is not detected, method 200 can continue to operation 210.Determining gesture/sound volume
After code, the classification of gesture/acoustic coding is determined by processor 124.
I class, such as ordinary instruction are encoded, then method 200 can continue to operation 214.Calculating equipment 100 can be true
Determine whether ordinary instruction input state activates.Calculating equipment 100 may be set to, and when the operation of certain program, user needed to input
When instruction, ordinary instruction input state is activated, may be so unexpected touch voice input with refusal.For example, with reference to Fig. 4 B,
When having arrived quarter-bell setting time, quarter-bell is remembered, while activating ordinary instruction input state, connects with the transaudient interface of equipment 100 is calculated
The sonic sensor connect prepares detection and touches sound.
Ordinary instruction input state is activated if detected, method 200 can continue to operation 218.It calculates
Equipment 100 is immediately performed command adapted thereto.If detecting un-activation ordinary instruction input state, method 200 be can continue to
End state 222.
Class ii, such as emergency instruction are encoded, then method 200 can continue to operation 218.It is vertical to calculate equipment 100
Execute corresponding encoded instruction.For example, detecting SOS coding with the sonic sensor that the transaudient interface of equipment 100 is connect is calculated
Sound is touched, alarm relief phone is put through immediately, even if screen is in a dormant state.
For group iii, such as specific function instruction encoding, then method 200 can continue to operation 216.Calculate equipment
100 can be determined whether to obtain the authorization of specific function instruction input.Calculating equipment 100 may be set to, and allow user in input weight
Authorization setting is carried out before the gesture/acoustic coding wanted, may be so unexpected touch voice input with refusal, improve input
Safety.For example, the touch voice input signal 300 of equipment is calculated for wearable type, in input specific function with reference to Fig. 9
It before instruction, can first be authorized from fingerprint recognition/button 308, then calculate the shell of equipment by touching with finger 172
306, the skin of screen 304 or component 311 or the back of the hand 176 input gesture/acoustic coding and instruction.
Specific function instruction input is authorized if detected, method 200 can continue to operation 218.Calculating is set
Standby 100 are immediately performed corresponding encoded instruction.If detecting unauthorized specific function instruction input, method 200 be can proceed with
To end state 222.
After operation 218, method 200 can continue to end state 222.
Application prospect of the present invention is extensive, such as:
1. (people does not get up, and mobile phone is on cabinet) passes through in cabinet body surface click/sliding touch, closing hand phone quarter-bell；
2. (mobile phone, which is put, to be seen video on the table or listen to music) is suspended or continues to play by clicking/sliding touch in desktop；
3. (people is bathing, and mobile phone is placed on by bathtub) is by clicking/sliding touch, answering cell phone incoming call in bathtub wall；
4. (in mobile phone communication) is by clicking/sliding touch in handset shell, terminating call or entering hands-free mode；
5. (mobile phone screen suspend mode) is touched by click/sliding, a certain program/function is called in input instruction；
6. (intelligent wearable device) is touched by click/sliding, input device controls instruction, such as: turning on light/turn off the light；
7. (when meeting ruffian) is touched by hidden gesture click/sliding, input emergency help instruction.
Foregoing description has a wide range of applications.For example, although examples disclosed herein can concentrate on the meter with touch screen
Equipment is calculated, it is to be understood that, imagination disclosed herein is similarly applied to substantially any other kinds of communication equipment.Cause
This, to the discussion of any embodiment be only intended to be it is illustrative, being not intended to suggestion includes being only limitted to the rights of these examples to want
The scope of the present disclosure asked.
Claims (9)
1. a kind of sensing touches the calculating equipment that sound is inputted as user gesture, characterized in that it comprises: processor, biography
Acoustic interface and at least one sonic sensor；
The transaudient interface is connect at least one described sonic sensor, is configured as transmitting touch sound corresponding with gesture
To at least one described sonic sensor；
At least one described sonic sensor and the processor communication, and it is configured as the touching corresponding with gesture that will be sensed
It touches sound and is converted to electric signal；
The processor determines the wave character of electric signal according to the electric signal, is identified by the wave character of the electric signal
The gesture of user is inputted as user gesture out；
The sonic sensor is the sensor of detection object vibration, is that the touch sound that will be propagated in object is converted into electricity
Signal；
The gesture of the user and the corresponding sound that touches use binary coding model split for gesture/acoustic coding, described
Gesture/acoustic coding includes: ordinary instruction coding, emergency instruction coding and special instruction coding；
The processor is also used to determine the classification of gesture/acoustic coding；It is encoded if it is ordinary instruction, it is determined that commonly refer to
It enables whether input state activates, the ordinary instruction is executed if detecting and having activated ordinary instruction input state, if inspection
Measuring un-activation ordinary instruction input state then terminates；The emergency instruction is then executed if it is emergency instruction coding；If it is spy
Distinguished service energy instruction encoding, it is determined whether obtain the authorization of specific function instruction input, authorized specific function if detected
Instruction input then executes specific function instruction, terminates if detecting unauthorized specific function instruction input.
2. calculating equipment according to claim 1, which is characterized in that the transaudient interface is the one of the sonic sensor
Part or the transaudient interface are a part of the shell for calculating equipment.
3. calculating equipment according to claim 1 or 2, which is characterized in that the transaudient interface is configured as described transaudient
When the other objects of interfacial contact, transaudient interface can transmit that user touches the sound issued when other objects and/or user directly touches
The transaudient interface can make a sound when touching the transaudient interface surface.
4. calculating equipment according to claim 1 or 2, which is characterized in that the transaudient interface is equipped with one or several
Protrusion/sunk structure and/or the surface of shell for calculating equipment are equipped with one or several protrusion/sunk structures.
5. calculating equipment according to claim 1 or 2, which is characterized in that at least one described sonic sensor is measurement
The acceleration transducer or piezoelectric film sensor of vibration acceleration.
6. a kind of sensing touches the method that sound is inputted as user gesture, which is characterized in that it includes the following steps:
Step 1, configuring computing devices make comprising: processor and at least one sonic sensor；
Step 2, it is sensed by least one described sonic sensor and touches sound and convert the touch sound sensed
For electric signal, it is to turn the touch sound propagated in object that the sonic sensor, which is the sensor of detection object vibration,
Change electric signal into；
Step 3, the electric signal transmitted by the processor according at least one described sonic sensor, determines the waveform of electric signal
Feature identifies that the gesture of user is inputted as the user gesture for calculating equipment by the wave character of the electric signal；
The gesture of the user and the corresponding sound that touches use binary coding model split for gesture/acoustic coding, described
Gesture/acoustic coding includes: ordinary instruction coding, emergency instruction coding and special instruction coding；
It is further comprising the steps of after step 3, the classification of gesture/acoustic coding is determined by the processor；
It is encoded if it is ordinary instruction, it is determined that whether ordinary instruction input state activates, and has activated commonly if detected
Instruction input state then executes the ordinary instruction, terminates if detecting un-activation ordinary instruction input state；
The emergency instruction is then executed if it is emergency instruction coding；
If it is specific function instruction encoding, it is determined whether the authorization of specific function instruction input is obtained, if detected
Authorization specific function instruction input then executes specific function instruction, ties if detecting unauthorized specific function instruction input
Beam.
7. according to the method described in claim 6, it is characterized in that, also being wrapped before, during or after the execution step 3
Include following steps:
The calculating equipment detects whether that there are also touch screens to input, if there is touch screen inputs, the processor priority processing touching
Touch screen input signal.
8. a kind of sensing touches the mobile computer that sound is inputted as user gesture, characterized in that it comprises: processor,
With the touch screen of processor communication, and at least one sonic sensor with processor communication；
The touch screen is configured as sensing user's finger near to or in contact with the touch action of touch screen, and described in sensing
Touch action is converted to touch screen input signal；
At least one described sonic sensor is configured as sensing touch sound corresponding with gesture, and the touching that will be sensed
It touches sound and is converted to electric signal；
The processor executes the corresponding instruction of touch action according to the touch screen input signal；
The processor determines the wave character of electric signal according to the electric signal, is identified by the wave character of the electric signal
The gesture of user is inputted as user gesture out, and executes the corresponding instruction of user gesture；
The sonic sensor is the sensor of detection object vibration, is that the touch sound that will be propagated in object is converted into electricity
Signal；
The gesture of the user and the corresponding sound that touches use binary coding model split for gesture/acoustic coding, described
Gesture/acoustic coding includes: ordinary instruction coding, emergency instruction coding and special instruction coding；
The processor is also used to determine the classification of gesture/acoustic coding；It is encoded if it is ordinary instruction, it is determined that commonly refer to
It enables whether input state activates, the ordinary instruction is executed if detecting and having activated ordinary instruction input state, if inspection
Measuring un-activation ordinary instruction input state then terminates；The emergency instruction is then executed if it is emergency instruction coding；If it is spy
Distinguished service energy instruction encoding, it is determined whether obtain the authorization of specific function instruction input, authorized specific function if detected
Instruction input then executes specific function instruction, terminates if detecting unauthorized specific function instruction input.
9. mobile computer according to claim 8, which is characterized in that its further include: transaudient interface；
The transaudient interface is connect at least one described sonic sensor, is configured as transmitting touch sound corresponding with gesture
To at least one described sonic sensor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610578907.5A CN106095203B (en) | 2016-07-21 | 2016-07-21 | Sensing touches the calculating device and method that sound is inputted as user gesture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610578907.5A CN106095203B (en) | 2016-07-21 | 2016-07-21 | Sensing touches the calculating device and method that sound is inputted as user gesture |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106095203A CN106095203A (en) | 2016-11-09 |
| CN106095203B true CN106095203B (en) | 2019-07-09 |
Family
ID=57449917
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610578907.5A Expired - Fee Related CN106095203B (en) | 2016-07-21 | 2016-07-21 | Sensing touches the calculating device and method that sound is inputted as user gesture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106095203B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110069199B (en) * | 2019-03-29 | 2022-01-11 | 中国科学技术大学 | Skin type finger gesture recognition method based on smart watch |
| CN112261765A (en) * | 2020-09-27 | 2021-01-22 | 深圳市广和通无线股份有限公司 | Light control method and device, control module, wearable device and home system |
| CN112755511A (en) * | 2021-01-27 | 2021-05-07 | 维沃移动通信有限公司 | Operation execution method and device of electronic equipment |
| CN114371796B (en) * | 2022-01-10 | 2024-06-04 | 深聪半导体(江苏)有限公司 | Method, device and storage medium for identifying touch position |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101529363A (en) * | 2006-10-18 | 2009-09-09 | 皇家飞利浦电子股份有限公司 | Electronic system control using surface interaction |
| CN101968696A (en) * | 2009-07-27 | 2011-02-09 | 索尼公司 | Sensing a type of action used to operate a touch panel |
| CN102890557A (en) * | 2011-07-19 | 2013-01-23 | 杜比实验室特许公司 | Method and system for touch gesture detection in response to microphone output |
| CN102985899A (en) * | 2010-07-09 | 2013-03-20 | 电子触控产品解决方案公司 | Method for determining a touch event and touch sensitive device |
| CN105683881A (en) * | 2013-09-09 | 2016-06-15 | 日本电气株式会社 | Information processing device, input method, and program |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9817521B2 (en) * | 2013-11-02 | 2017-11-14 | At&T Intellectual Property I, L.P. | Gesture detection |
-
2016
- 2016-07-21 CN CN201610578907.5A patent/CN106095203B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101529363A (en) * | 2006-10-18 | 2009-09-09 | 皇家飞利浦电子股份有限公司 | Electronic system control using surface interaction |
| CN101968696A (en) * | 2009-07-27 | 2011-02-09 | 索尼公司 | Sensing a type of action used to operate a touch panel |
| CN102985899A (en) * | 2010-07-09 | 2013-03-20 | 电子触控产品解决方案公司 | Method for determining a touch event and touch sensitive device |
| CN102890557A (en) * | 2011-07-19 | 2013-01-23 | 杜比实验室特许公司 | Method and system for touch gesture detection in response to microphone output |
| CN105683881A (en) * | 2013-09-09 | 2016-06-15 | 日本电气株式会社 | Information processing device, input method, and program |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106095203A (en) | 2016-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106708254B (en) | Detector | |
| CN106095203B (en) | Sensing touches the calculating device and method that sound is inputted as user gesture | |
| Zhang et al. | FingerPing: Recognizing fine-grained hand poses using active acoustic on-body sensing | |
| KR101932210B1 (en) | Method, system for implementing operation of mobile terminal according to touching signal and mobile terminal | |
| Zhang et al. | TapSkin: Recognizing on-skin input for smartwatches | |
| Feldman et al. | ReachMedia: On-the-move interaction with everyday objects | |
| KR101442936B1 (en) | User interface methods and systems for providing force-sensitive input | |
| KR100995130B1 (en) | The system for recogniging of user touch pattern using touch sensor and accelerometer sensor | |
| US10684675B2 (en) | Method and apparatus using frictional sound | |
| CN110132458B (en) | Dynamic or quasi-dynamic force detection device and method | |
| CN105027025A (en) | Digitizer system with improved response time to wake up signal | |
| WO2018194719A1 (en) | Electronic device response to force-sensitive interface | |
| KR20150092065A (en) | Method and device for sensing touch inputs | |
| WO2012017241A1 (en) | Control of electronic devices | |
| WO2018194718A1 (en) | Force-sensitive user input interface for an electronic device | |
| US10551973B2 (en) | Method of controlling a mobile device | |
| WO2012160414A1 (en) | Finger-on display detection | |
| CN111666549A (en) | Intelligent earphone and user identification method thereof | |
| CN104423538B (en) | A kind of information processing method and device | |
| CN109754823A (en) | A kind of voice activity detection method, mobile terminal | |
| JP2018528537A (en) | System and method for double knuckle touchscreen control | |
| Ni et al. | Uncovering user interactions on smartphones via contactless wireless charging side channels | |
| CN109067978A (en) | Button operation processing method, device, storage medium and electronic equipment | |
| CN107340864A (en) | A kind of virtual input method based on sound wave | |
| Kato et al. | Surfboard: keyboard with microphone as a low-cost interactive surface |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20190611 Address after: 430080 Ganghuacun 119 Block 66 Gate 9, Qingshan District, Wuhan City, Hubei Province Applicant after: Fan Sihui Applicant after: Fan Xiaogang Address before: 430223 No. 33 University Park Road, Donghu Technology Development Zone, Wuhan City, Hubei Province Applicant before: Fan Xiaogang |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20190709 Termination date: 20200721 |