WO2017186290A1 - Method to recognize a gesture and corresponding device - Google Patents

Method to recognize a gesture and corresponding device Download PDF

Info

Publication number
WO2017186290A1
WO2017186290A1 PCT/EP2016/059451 EP2016059451W WO2017186290A1 WO 2017186290 A1 WO2017186290 A1 WO 2017186290A1 EP 2016059451 W EP2016059451 W EP 2016059451W WO 2017186290 A1 WO2017186290 A1 WO 2017186290A1
Authority
WO
WIPO (PCT)
Prior art keywords
signal
gesture
physical object
gestures
entrance
Prior art date
Application number
PCT/EP2016/059451
Other languages
French (fr)
Inventor
Bruno ZAMBORLIN
Baptiste CARAMIAUX
Carmine Emanuele CELLA
Original Assignee
Mogees Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mogees Limited filed Critical Mogees Limited
Priority to EP16721113.5A priority Critical patent/EP3308250A1/en
Priority to JP2019507992A priority patent/JP2019518294A/en
Priority to PCT/EP2016/059451 priority patent/WO2017186290A1/en
Priority to CN201680087100.XA priority patent/CN109416607A/en
Publication of WO2017186290A1 publication Critical patent/WO2017186290A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/043Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using propagating acoustic waves

Definitions

  • the present invention concerns a method to recognize a gesture, such as, for example, a physical action, on a physical object by means of a device associated with the physical object and using this method.
  • the present invention may be used in the field of sound reproduction, interaction with physical objects, command and management of devices as a function of specific gestures.
  • the recognition of physical gestures is a field of human-machine interaction with the purpose of supplying devices and methods able to identify and classify various gestures in order to then command and/or control other devices connected to the recognition device according to the specific gesture made.
  • One of the main problems in known devices for recognizing a gesture is that they are not very sensitive to gestures that are not particularly accentuated, i.e. gestures with low intensities or gestures that do not generate any detectable signal.
  • Devices which use plates and/or screens with touch-sensitive surfaces to detect gestures in contact only with the touch-sensitive surface of the plates and/or screens.
  • the measurement Since it is calibrated to detect only gestures that generate electric signals in the device above a threshold value, the measurement is normally insensitive to many types of touch, so that it does not allow a complete discrimination between the various gestures that are made.
  • Another problem of known devices is that they do not allow to quickly recognize repeated and/or simultaneous touches on the same device. To recognize two or more consecutive gestures, these known devices normally need an interruption in the contact between one touch and the next, with the result that there is a delay in detecting successive gestures.
  • Another problem in the state of the art is that methods for recognizing a gesture and the corresponding devices using said methods are pre-configured and cannot be modified by the user.
  • the user cannot vary the pre-set measurement thresholds and so he cannot personalize the recognition device in order to memorize, classify and recognize gestures that are not pre-set which can be more useful for his specific needs.
  • a disclosed method is able to identify and classify in real time a gesture on any physical object, and also to discriminate effectively the various gestures.
  • a disclosed device uses said method to recognize a gesture that allows detection of a touch done on one surface of the physical object, associating the device with a desired area of the physical object.
  • the disclosed method and/or device recognize a succession of gestures that allow recognition of the individual gestures quickly, even without interrupting contact with the physical object.
  • the present invention concerns a method to recognize a gesture made in direct or indirect contact on a physical object by means of a device using said method.
  • the method comprises the following steps, identified in a non-rigid order:
  • the method also provides a quality features step of the entrance signal, possibly carried out in parallel with the segmentation step, in order to obtain one or more adjustable descriptors of the gesture, to assess the characteristics of the entrance signal provided.
  • the exit signal is a digital signal that can be converted into an electric, computer or optical signal, or other type, able to command and/or control a device connected to the recognition device using the method to recognize a gesture.
  • the exit signal contains the values of likelihoods between a gesture made and at least part of the memorized gestures, so as to supply quantitative information on the similarity of the gesture made with the memorized gestures.
  • the invention provides a device to recognize a gesture made on the physical object which comprises at least:
  • transducer associable in contact with a physical object and configured to detect the vibrations and/or sounds generated by the gesture made on the physical object;
  • reception unit configured to receive an entrance signal from the transducer generated from a gesture made on the physical object
  • a segmentation unit configured to receive the entrance signal and to obtain a group of signal segments and at least one physical value processed from to the entrance signal;
  • processing unit configured to calculate at least a characteristic vector of signal features for each segment of the group of signal segments using the at least one physical value processed from the entrance signal
  • a memory unit configured to memorize the characteristic vectors, processed by the entrance signal, and an identification label of the gesture made on the physical object;
  • comparison-learning unit configured to recognize if a new gesture made on the physical object is one of the gestures memorized in the memory unit, comparing and learning the metric function between the characteristic vectors memorized and those processed on each occasion starting from the entrance signal correlated to the new gesture made;
  • a command unit configured to supply a specific exit signal correlated to the specific gesture made and possibly recognized resulting from the comparison with the metric function.
  • the transducer associable in contact with a physical object is an adhesive piezo-transducer contact microphone, configured to capture the sounds and/or the vibrations generated by a gesture made on the physical object.
  • the device comprises a quality features unit configured to obtain one or more adjustable descriptors of the gesture to assess the characteristics of the entrance signal supplied to it by the reception unit.
  • - fig. 1 shows a block diagram of a method to recognize a gesture according to the present invention
  • - figs. 2-4 show three block diagrams of three steps of a method according to the present invention.
  • - fig. 5 shows a perspective and schematic view of a device to recognize a gesture associated with a physical object and using a method according to the present invention.
  • gesture we mean any physical action made in direct or indirect contact, or by means of another element, by a subject on the physical object 30 associated with the device 10.
  • a gesture can be a hit, a swipe, a graze, a scratch, a touch or other type of gesture of various length and intensity even when the gesture is being made.
  • the device 10 comprises a transducer 1 1 associable in contact with any type of physical object 30.
  • the transducer 1 1 is configured to detect the vibrations and/or sounds generated by the gesture made on any desired zone of the physical object 30.
  • the gestures can be made on the whole usable surface of the physical object 30 and are no longer limited to a zone defined by a plate and/or screen with defined dimensions.
  • the transducer 1 1 is sensitive to various types of gestures which can be made in succession on the physical object 30 even without interrupting contact with the physical object 30 itself.
  • the transducer 1 1 is able to detect directly the vibrations and/or sounds mechanically propagated by the physical object 30 and, hence, it does not need to identify any variation in resistance and/or electric capacity.
  • the transducer 11 is an adhesive piezo- transducer contact microphone which, once associated in contact with a physical object 30, is able to capture the sounds and/or the vibrations generated by the gesture made on the physical object 30 and to convert them into an electric signal in function of the type of gesture made.
  • the transducer 1 1 does not detect the sounds not transmitted by the physical object 30, such as for example voices, music or other.
  • the transducer 1 1 can be calibrated to be sensitive to an acoustic or detection range, defined on each occasion according to specific operating requirements.
  • the transducer 1 1 can be sensitive to the normal human acoustic range, or to other acoustic and/or mechanical ranges to allow to detect the vibrations and/or sounds inaudible to human.
  • the possibility of adjusting the sensitivity is particularly advantageous if it is desired to detect high-resolution vibrations, potentially inaudible to human, such as for example specific mechanical vibrations generated by a piezo-electric positioner or other element.
  • the transducer 1 1 converts the vibrations and/or sounds generated by the gesture on the physical object 30 into a corresponding entrance signal 12 that is transmitted to a reception unit 13 in a timed manner.
  • the timing actuated allows to analyze and process two or more consecutive or partly overlapping gestures, in a distinct manner.
  • the entrance signal 12 is correlated to the vibration and/or sound detected by the transducer 1 1 and can be transmitted either by cable that connects the transducer 1 1 to the reception unit 13 or directly telematically.
  • the reception unit 13 can be an electronic circuit, a microprocessor or other similar and comparable electronic units that allow to receive and then transmit the entrance signal 12 to other units of the device 10.
  • the device 10 comprises a segmentation unit 14 configured to receive the entrance signal 12 from the reception unit 13 and to suitably divide it so as to obtain a group of signal segments 15 and at least one physical value 16 processed from the entrance signal 12.
  • the method to recognize a gesture provides that the entrance signal 12 is segmented by the segmentation unit 14 and the method provides that the following sub-steps are carried out: a - extracting an envelope from the entrance signal 12 to obtain a rough estimation of the sample-wise amplitude of the signal, that is, a signal defining the functional form of the amplitudes of the entrance signal 12;
  • Each segment detected is inserted into the group of signal segments 15 if the amplitude of the envelope is greater than or equal to a threshold value.
  • the temporal division of the segments is timed by setting a minimum barrier time interval.
  • the entrance signal 12 such as, for example, the gesture group velocity computed as the maximum in amplitude of the segmented signal.
  • the method in parallel to the segmentation of the entrance signal 12, the method provides that a quality features step of the entrance signal 12 can also be carried out.
  • the quality features step is carried out by a quality features unit 17 configured to calculate and obtain descriptors of the entrance signal 12 possibly adjustable to assess the characteristics of the entrance signal 12 and hence to improve the recognition of the gesture obtainable by the method.
  • the quality features unit 17 is configured to receive the entrance signal 12 from the reception unit 13 and the method provides to carry out the following sub-steps:
  • f - calculating one or more descriptors computed, for example, as the first three spectral moments, i.e. energy, median or centroid of the spectrum and the spread.
  • the descriptors can be calculated continuously and are used to obtain an accurate control of the quality of the entrance signal 12 that is transmitted to the segmentation unit 14 or directly to a command unit 23.
  • the device 10 comprises a processing unit 19 configured to calculate at least a characteristic vector 18 of signal features, processing each segment of the group of signal segments 15 using the at least one physical value 16 obtained by the segmentation unit 14.
  • the device 10 is able to memorize the entrance signals 12 generated by the gestures made on the physical object 30, associating with every specific entrance signal 12 at least one specific characteristic vector 18 of signal features for each segment of the group of signal segments 15 and learning the metric function between the characteristic vectors 18 of the gestures.
  • the characteristic vector 18 can comprise cepstral coefficients, spectral moments, i.e. energy, centroid, spread, or other physical values such as the group velocity or the amplitude of the entrance signal 12.
  • the device 10 allows to personalize the types of gestures recognizable, creating and memorizing a series of gestures in a memory unit 20 identifying each gesture with an identification label 21 and at least one characteristic vector 18.
  • the method provides to attribute for each element a relevance degree in order to give more importance to the elements, which are more useful to compare two entrance signals 12, generated from two gestures and to classify them.
  • the characteristic vectors 18 are calculated on each occasion and used to compare and quantify, by means of metric function, how much a gesture made on the physical object 30 is similar to those previously memorized.
  • the at least one characteristic vector 18 is calculated for every entrance signal 12 so as to be able to be used to compare and possibly recognize the entrance signals 12 correlated to the gestures previously memorized.
  • the device 10 can also memorize the metric function between the characteristic vectors 18 of the gestures.
  • the device 10 can be used to reproduce a specific sound based on the specific gesture made on the physical object 30.
  • the memorization or learning step must be enabled, which takes place following the calculation of the characteristic vectors 18.
  • the memorization step provides to associate an identification label 21 of the gesture made with the corresponding characteristic vector 18 calculated and then to memorize both in the memory unit 20.
  • the gestures made on the physical object 30 are memorized in the memory unit 20, attributing to each one of them at least one characteristic vector 18 and an identification label 21, so as to be able to subsequently identify, during the step of comparison and learning, to which gesture the minimal metric function between the characteristic vectors 18 of the gestures corresponds.
  • the device 10 comprises a comparison-learning unit 22, configured to compare at least one characteristic vector 18, corresponding to a gesture made on the physical object 30, with the characteristic vectors 18 memorized in the memory unit 20 and to recognize whether, possibly, the gesture is contained therein.
  • the comparison-learning unit 22 calculates the metric function between the characteristic vector 18 processed of the gesture made and each characteristic vector 18 of the gestures previously memorized.
  • the comparison-learning unit 22 is designed also to memorize and learn the metric function between the characteristic vectors 18 calculated.
  • the comparison-learning unit 22 is also called classifier because it allows to quantify the similarity of the gesture made with each gesture memorized according to the metric function between the characteristic vectors 18.
  • the comparison-learning unit 22 is also able to supply the values of likelihoods of each gesture memorized with the gesture made.
  • the present method allows to obtain a high degree of discrimination between the various gestures, even very similar to each other, because it operates on the segmented signal and therefore, if there are differences inside the group of signal segments 15 between two similar gestures, these differences are identified and hence discriminated.
  • the possibility of analyzing the entrance signal 12 segment by segment allows to identify if there are even small differences between similar gestures, so as to have a high degree of discrimination compared with the state of the art.
  • the comparison step is carried out on all the gestures memorized and in particular on the memorized characteristic vectors 18, it is also possible to verify the degree of likelihoods on the whole plurality of memorized gestures with the gesture made, making the discrimination between similar gestures even more efficient.
  • the comparison step provides the degree of likelihoods from the segments of the group of signal segments 15 of the entrance signal 12 and the memorized gestures
  • This interpolation process gives access to the continuous space of possible gestures rather than the space defined by the memorized gestures only.
  • the comparison-learning unit 22 supplies at exit the identification label 21 of the memorized gesture for which the metric function or difference between the characteristic vectors 18 are minimal, i.e. there is a high probability that it will be recognized.
  • the comparisons are carried out on a group of signal segments 15 and not on the whole group of values of the entrance signal 12, it is possible to obtain a recognition of the gesture within limited time intervals, for example in the range of milliseconds, and without needing to interrupt the contact with the physical object 30.
  • the specific gesture made has been recognized among those memorized in the memory unit 20, it is possible to supply, by means of a command unit 23, a specific exit signal 24 correlated to the specific gesture recognized.
  • the exit signal 24 can be a digital, electric, computer, or optical signal that can command and/or control a device 25 connected to the device 10.
  • the device 10 is able to supply an exit signal 24 to a smartphone, tablet or computer, to reproduce a specific signal, for example a sound, according to the specific gesture made and recognized by the device 10.
  • the device 10 is able to control devices 25 outside the device 10 such as videogame consoles, computers, tablets, smartphones or other devices.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Human Computer Interaction (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The present disclosure concerns a method to recognize a gesture, namely a physical action, made in direct or indirect contact on a physical object (30), wherein said method comprises: - associating a device (10) using said method in contact with said physical object (30); - supplying an entrance signal (12) generated by a gesture made in direct or indirect contact on said physical object (30); - segmenting said entrance signal (12) to obtain a group of signal segments (15) and at least one physical value (16) processed from said entrance signal (12); - processing at least a characteristic vector (18) of signal features for each segment of said group of signal segments (15) using said at least one physical value (16) in order to learn metric function between gestures; - supplying a specific exit signal (24) correlated to the specific gesture. Further, the present invention concerns also a device to recognize a gesture using said method to recognize a gesture.

Description

"METHOD TO RECOGNIZE A GESTURE AND CORRESPONDING DEVICE"
FIELD OF THE INVENTION
The present invention concerns a method to recognize a gesture, such as, for example, a physical action, on a physical object by means of a device associated with the physical object and using this method.
By way of non-restrictive example, the present invention may be used in the field of sound reproduction, interaction with physical objects, command and management of devices as a function of specific gestures.
BACKGROUND OF THE INVENTION
The recognition of physical gestures is a field of human-machine interaction with the purpose of supplying devices and methods able to identify and classify various gestures in order to then command and/or control other devices connected to the recognition device according to the specific gesture made.
One of the main problems in known devices for recognizing a gesture is that they are not very sensitive to gestures that are not particularly accentuated, i.e. gestures with low intensities or gestures that do not generate any detectable signal.
Devices are known, which use plates and/or screens with touch-sensitive surfaces to detect gestures in contact only with the touch-sensitive surface of the plates and/or screens.
This greatly limits the possibilities of interaction with known devices because it confines the usable detection area to only the surface of the plate and/or screen. Devices for recognizing touch are known, able to detect the contact of the subject with the device by means of a variation in the resistance and/or electric capacity measured by the device itself.
Since it is calibrated to detect only gestures that generate electric signals in the device above a threshold value, the measurement is normally insensitive to many types of touch, so that it does not allow a complete discrimination between the various gestures that are made.
Another problem of known devices is that they do not allow to quickly recognize repeated and/or simultaneous touches on the same device. To recognize two or more consecutive gestures, these known devices normally need an interruption in the contact between one touch and the next, with the result that there is a delay in detecting successive gestures.
It is also known that methods for recognizing gestures are not able to detect two distinct gestures simultaneously, but at most can detect a multiple touch, considering it as an individual gesture.
Another problem in the state of the art is that methods for recognizing a gesture and the corresponding devices using said methods are pre-configured and cannot be modified by the user.
Therefore, the user cannot vary the pre-set measurement thresholds and so he cannot personalize the recognition device in order to memorize, classify and recognize gestures that are not pre-set which can be more useful for his specific needs.
There is therefore a need to advance the state of the art and make available a method to recognize a gesture, and the corresponding device using said method, so as to overcome at least one of the disadvantages of the state of the art.
SUMMARY OF THE INVENTION
In accordance with one exemplary, a disclosed method is able to identify and classify in real time a gesture on any physical object, and also to discriminate effectively the various gestures.
In accordance with another exemplary aspect, a disclosed device uses said method to recognize a gesture that allows detection of a touch done on one surface of the physical object, associating the device with a desired area of the physical object.
In accordance with another exemplary aspect, the disclosed method and/or device recognize a succession of gestures that allow recognition of the individual gestures quickly, even without interrupting contact with the physical object.
The Applicant has devised, tested and embodied the exemplary forms outlined herein, which may address and/or overcome the shortcomings of the state of the art and which may obtain these and other purposes and advantages. The present invention is set forth and characterized in the independent claims, while the dependent claims describe other characteristics of the invention or variants to the main inventive idea.
In accordance with the above purposes, the present invention concerns a method to recognize a gesture made in direct or indirect contact on a physical object by means of a device using said method.
According to one aspect of the present invention, the method comprises the following steps, identified in a non-rigid order:
- associating the device using said method in contact with the physical object; - supplying an entrance signal generated by a gesture made in direct or indirect contact on the physical object;
- segmenting the entrance signal to obtain a group of signal segments and at least one physical value processed from the entrance signal;
- processing at least a characteristic vector of signal features for each segment of the group of signal segments using the at least one physical value in order to learn the metric function between gestures;
- memorizing the characteristic vectors processed with an identification label of the gesture made on the physical object;
- comparing a new gesture made, in direct or indirect contact on the physical object, with the gestures previously memorized in order to possibly recognize if the new gesture made is one of the memorized gestures, evaluating and learning the metric function between the characteristic vectors, processed from the gesture made, and the characteristic vectors processed from the gestures previously memorized;
- supplying a specific exit signal correlated to the specific gesture recognized and resulting from the comparison between characteristic vectors, using the learned metric function.
According to a variant, the method also provides a quality features step of the entrance signal, possibly carried out in parallel with the segmentation step, in order to obtain one or more adjustable descriptors of the gesture, to assess the characteristics of the entrance signal provided.
According to a variant, the exit signal is a digital signal that can be converted into an electric, computer or optical signal, or other type, able to command and/or control a device connected to the recognition device using the method to recognize a gesture.
According to a variant, the exit signal contains the values of likelihoods between a gesture made and at least part of the memorized gestures, so as to supply quantitative information on the similarity of the gesture made with the memorized gestures.
According to some embodiments of the present invention, the invention provides a device to recognize a gesture made on the physical object which comprises at least:
- a transducer associable in contact with a physical object and configured to detect the vibrations and/or sounds generated by the gesture made on the physical object;
- a reception unit configured to receive an entrance signal from the transducer generated from a gesture made on the physical object;
- a segmentation unit configured to receive the entrance signal and to obtain a group of signal segments and at least one physical value processed from to the entrance signal;
- a processing unit configured to calculate at least a characteristic vector of signal features for each segment of the group of signal segments using the at least one physical value processed from the entrance signal;
- a memory unit configured to memorize the characteristic vectors, processed by the entrance signal, and an identification label of the gesture made on the physical object;
- a comparison-learning unit configured to recognize if a new gesture made on the physical object is one of the gestures memorized in the memory unit, comparing and learning the metric function between the characteristic vectors memorized and those processed on each occasion starting from the entrance signal correlated to the new gesture made;
- a command unit configured to supply a specific exit signal correlated to the specific gesture made and possibly recognized resulting from the comparison with the metric function.
According to a variant, the transducer associable in contact with a physical object is an adhesive piezo-transducer contact microphone, configured to capture the sounds and/or the vibrations generated by a gesture made on the physical object.
According to a variant, the device comprises a quality features unit configured to obtain one or more adjustable descriptors of the gesture to assess the characteristics of the entrance signal supplied to it by the reception unit.
BRIEF DESCRIPTION OF THE DRAWINGS
These and other characteristics of the present invention will become apparent from the following description of some embodiments, given as a non-restrictive example with reference to the attached drawings wherein:
- fig. 1 shows a block diagram of a method to recognize a gesture according to the present invention;
- figs. 2-4 show three block diagrams of three steps of a method according to the present invention;
- fig. 5 shows a perspective and schematic view of a device to recognize a gesture associated with a physical object and using a method according to the present invention.
To facilitate comprehension, the same reference numbers have been used, where possible, to identify identical common elements in the drawings. It is understood that elements and characteristics of one embodiment can conveniently be incorporated into other embodiments without further clarifications.
DETAILED DESCRIPTION OF SOME EMBODIMENTS
With reference to the drawings, which show non-restrictive examples of the invention, we will now describe embodiments of a method to recognize a gesture made on a physical object 30 associated with a device 10 using said method. By gesture we mean any physical action made in direct or indirect contact, or by means of another element, by a subject on the physical object 30 associated with the device 10.
By way of non-restrictive example, a gesture can be a hit, a swipe, a graze, a scratch, a touch or other type of gesture of various length and intensity even when the gesture is being made.
By way of non-restrictive example, by physical object 30 we mean a board, a sheet, a wall or in any case any physical object 30 with a physical form, roughness, consistency or other characteristics. According to one aspect of the present invention, the device 10 comprises a transducer 1 1 associable in contact with any type of physical object 30.
The transducer 1 1 is configured to detect the vibrations and/or sounds generated by the gesture made on any desired zone of the physical object 30. In this way, contrary to the state of the art, the gestures can be made on the whole usable surface of the physical object 30 and are no longer limited to a zone defined by a plate and/or screen with defined dimensions.
Furthermore, because of this, it is possible to position the transducer 1 1 on zones that on each occasion have various shapes and consistency even of the same physical object 30, so as to personalize the gesture according to requirements and to the surfaces available.
According to one aspect of the present invention, the transducer 1 1 is sensitive to various types of gestures which can be made in succession on the physical object 30 even without interrupting contact with the physical object 30 itself. The transducer 1 1 is able to detect directly the vibrations and/or sounds mechanically propagated by the physical object 30 and, hence, it does not need to identify any variation in resistance and/or electric capacity.
According to one embodiment, the transducer 11 is an adhesive piezo- transducer contact microphone which, once associated in contact with a physical object 30, is able to capture the sounds and/or the vibrations generated by the gesture made on the physical object 30 and to convert them into an electric signal in function of the type of gesture made.
In this case, an improvement is obtained to insulation from sources of external noise because the transducer 1 1 does not detect the sounds not transmitted by the physical object 30, such as for example voices, music or other.
According to one embodiment, the transducer 1 1 can be calibrated to be sensitive to an acoustic or detection range, defined on each occasion according to specific operating requirements.
For example, the transducer 1 1 can be sensitive to the normal human acoustic range, or to other acoustic and/or mechanical ranges to allow to detect the vibrations and/or sounds inaudible to human.
The possibility of adjusting the sensitivity is particularly advantageous if it is desired to detect high-resolution vibrations, potentially inaudible to human, such as for example specific mechanical vibrations generated by a piezo-electric positioner or other element.
According to one aspect of the present invention, the transducer 1 1 converts the vibrations and/or sounds generated by the gesture on the physical object 30 into a corresponding entrance signal 12 that is transmitted to a reception unit 13 in a timed manner.
The timing actuated, for example by a buffer or other programmable circuit or unit, allows to analyze and process two or more consecutive or partly overlapping gestures, in a distinct manner.
The entrance signal 12 is correlated to the vibration and/or sound detected by the transducer 1 1 and can be transmitted either by cable that connects the transducer 1 1 to the reception unit 13 or directly telematically.
According to one embodiment, the reception unit 13 can be an electronic circuit, a microprocessor or other similar and comparable electronic units that allow to receive and then transmit the entrance signal 12 to other units of the device 10.
The other units comprised in the device 10 that will be described hereafter in the description, like the reception unit 13, can also be electronic circuits, microprocessors or other similar and comparable electronic units.
According to one aspect of the invention, the device 10 comprises a segmentation unit 14 configured to receive the entrance signal 12 from the reception unit 13 and to suitably divide it so as to obtain a group of signal segments 15 and at least one physical value 16 processed from the entrance signal 12.
According to one aspect of the present invention, the method to recognize a gesture provides that the entrance signal 12 is segmented by the segmentation unit 14 and the method provides that the following sub-steps are carried out: a - extracting an envelope from the entrance signal 12 to obtain a rough estimation of the sample-wise amplitude of the signal, that is, a signal defining the functional form of the amplitudes of the entrance signal 12;
b - applying a low pass filter to the envelope of the entrance signal 12 to obtain a signal with a functional profile of the envelope without noise coming from other sources or external agents that are not generated by or correlated to the gesture made;
c - selection and division of the filtered envelope of the entrance signal 12 in a group of signal segments 15 which comprises segments or parts of the entrance signal 12 which satisfy an adjustable threshold condition on the amplitude of the signal, according to the sensitivity to be obtained. Each segment detected is inserted into the group of signal segments 15 if the amplitude of the envelope is greater than or equal to a threshold value. To prevent two segments from being detected in a non-sensitive period of time, the temporal division of the segments is timed by setting a minimum barrier time interval.
d - calculating at least one physical value 16 from the entrance signal 12, such as, for example, the gesture group velocity computed as the maximum in amplitude of the segmented signal.
According to one embodiment, in parallel to the segmentation of the entrance signal 12, the method provides that a quality features step of the entrance signal 12 can also be carried out.
The quality features step is carried out by a quality features unit 17 configured to calculate and obtain descriptors of the entrance signal 12 possibly adjustable to assess the characteristics of the entrance signal 12 and hence to improve the recognition of the gesture obtainable by the method.
According to one embodiment, the quality features unit 17 is configured to receive the entrance signal 12 from the reception unit 13 and the method provides to carry out the following sub-steps:
e - calculating the fast Fourier transform (FFT) to obtain a vector of spectral amplitudes;
f - calculating one or more descriptors computed, for example, as the first three spectral moments, i.e. energy, median or centroid of the spectrum and the spread.
According to one embodiment, the descriptors can be calculated continuously and are used to obtain an accurate control of the quality of the entrance signal 12 that is transmitted to the segmentation unit 14 or directly to a command unit 23.
According to one aspect of the present invention, the device 10 comprises a processing unit 19 configured to calculate at least a characteristic vector 18 of signal features, processing each segment of the group of signal segments 15 using the at least one physical value 16 obtained by the segmentation unit 14.
According to one aspect of the present invention, the device 10 is able to memorize the entrance signals 12 generated by the gestures made on the physical object 30, associating with every specific entrance signal 12 at least one specific characteristic vector 18 of signal features for each segment of the group of signal segments 15 and learning the metric function between the characteristic vectors 18 of the gestures.
According to one embodiment, the characteristic vector 18 can comprise cepstral coefficients, spectral moments, i.e. energy, centroid, spread, or other physical values such as the group velocity or the amplitude of the entrance signal 12.
According to one embodiment, the device 10 allows to personalize the types of gestures recognizable, creating and memorizing a series of gestures in a memory unit 20 identifying each gesture with an identification label 21 and at least one characteristic vector 18.
Using the memorized gestures characteristic vectors 18 and the characteristic vector 18 of the gesture made, it is possible to calculate for each element of the characteristic vector 18 a metric function in order to quantify how much two gestures, identified by their identification label 21 , are similar, following the sub- steps:
- ordering the characteristic vectors 18 of the gestures according to their identification label 21 ;
- applying the metric function to compare characteristic vectors 18 of the gestures for classification;
- assessing the metric function on the ordered characteristic vector 18 of the gesture;
- adjusting the metric function to minimize classification errors and iterate until the errors are minimum.
In particular, the method provides to attribute for each element a relevance degree in order to give more importance to the elements, which are more useful to compare two entrance signals 12, generated from two gestures and to classify them. The characteristic vectors 18 are calculated on each occasion and used to compare and quantify, by means of metric function, how much a gesture made on the physical object 30 is similar to those previously memorized.
According to one aspect of the invention, the at least one characteristic vector 18 is calculated for every entrance signal 12 so as to be able to be used to compare and possibly recognize the entrance signals 12 correlated to the gestures previously memorized.
According to one embodiment, the device 10 can also memorize the metric function between the characteristic vectors 18 of the gestures.
Thanks to this, it is possible to personalize on each occasion the types and set of gestures to be compared and recognized, so that an external device can then be controlled and/or commanded in a desired manner, based on the specific gesture made on the physical object 30.
One of the possible applications is that the device 10 can be used to reproduce a specific sound based on the specific gesture made on the physical object 30.
If it is desired to memorize or learn a gesture, the memorization or learning step must be enabled, which takes place following the calculation of the characteristic vectors 18.
According to one aspect of the invention, the memorization step provides to associate an identification label 21 of the gesture made with the corresponding characteristic vector 18 calculated and then to memorize both in the memory unit 20.
The gestures made on the physical object 30 are memorized in the memory unit 20, attributing to each one of them at least one characteristic vector 18 and an identification label 21, so as to be able to subsequently identify, during the step of comparison and learning, to which gesture the minimal metric function between the characteristic vectors 18 of the gestures corresponds.
According to one aspect of the present invention, the device 10 comprises a comparison-learning unit 22, configured to compare at least one characteristic vector 18, corresponding to a gesture made on the physical object 30, with the characteristic vectors 18 memorized in the memory unit 20 and to recognize whether, possibly, the gesture is contained therein. In particular, according to one embodiment, the comparison-learning unit 22 calculates the metric function between the characteristic vector 18 processed of the gesture made and each characteristic vector 18 of the gestures previously memorized.
According to one aspect of the present invention, the comparison-learning unit 22 is designed also to memorize and learn the metric function between the characteristic vectors 18 calculated.
This is an important aspect of the invention because it allows to improve the recognition of the gesture made quantifying how similar it is in comparison with the memorized gestures.
According to one embodiment, the comparison-learning unit 22 is also called classifier because it allows to quantify the similarity of the gesture made with each gesture memorized according to the metric function between the characteristic vectors 18. In particular, the comparison-learning unit 22 is also able to supply the values of likelihoods of each gesture memorized with the gesture made.
The present method allows to obtain a high degree of discrimination between the various gestures, even very similar to each other, because it operates on the segmented signal and therefore, if there are differences inside the group of signal segments 15 between two similar gestures, these differences are identified and hence discriminated.
In other words, the possibility of analyzing the entrance signal 12 segment by segment allows to identify if there are even small differences between similar gestures, so as to have a high degree of discrimination compared with the state of the art.
Furthermore, given that the comparison step is carried out on all the gestures memorized and in particular on the memorized characteristic vectors 18, it is also possible to verify the degree of likelihoods on the whole plurality of memorized gestures with the gesture made, making the discrimination between similar gestures even more efficient.
In addition, given that the comparison step provides the degree of likelihoods from the segments of the group of signal segments 15 of the entrance signal 12 and the memorized gestures, it is possible to specify the entrance signal 12 as a combination of the memorized gestures weighted by their likelihood. This interpolation process gives access to the continuous space of possible gestures rather than the space defined by the memorized gestures only.
According to one possible embodiment, the comparison-learning unit 22 supplies at exit the identification label 21 of the memorized gesture for which the metric function or difference between the characteristic vectors 18 are minimal, i.e. there is a high probability that it will be recognized.
According to one embodiment of the present invention, given that the comparisons are carried out on a group of signal segments 15 and not on the whole group of values of the entrance signal 12, it is possible to obtain a recognition of the gesture within limited time intervals, for example in the range of milliseconds, and without needing to interrupt the contact with the physical object 30.
According to one aspect of the present invention, once the specific gesture made has been recognized among those memorized in the memory unit 20, it is possible to supply, by means of a command unit 23, a specific exit signal 24 correlated to the specific gesture recognized.
In particular, the exit signal 24 can be a digital, electric, computer, or optical signal that can command and/or control a device 25 connected to the device 10. According to a preferential embodiment, the device 10 is able to supply an exit signal 24 to a smartphone, tablet or computer, to reproduce a specific signal, for example a sound, according to the specific gesture made and recognized by the device 10.
According to one embodiment, the device 10 is able to control devices 25 outside the device 10 such as videogame consoles, computers, tablets, smartphones or other devices.
It is clear that modifications and/or additions of parts may be made to the method to recognize a gesture and the corresponding device 10 using said method as described heretofore, without departing from the field and scope of the present invention.
It is also clear that, although the present invention has been described with reference to some specific examples, a person of skill in the art shall certainly be able to achieve many other equivalent forms of method to recognize a gesture and the corresponding device 10 using said method, having the characteristics as set forth in the claims and hence all coming within the field of protection defined thereby.

Claims

1. Method to recognize a gesture made in direct or indirect contact on a physical object (30), characterized in that said method comprises the following steps:
- associating a device (10) using said method in contact with said physical object (30);
- supplying an entrance signal (12) generated by a gesture made in direct or indirect contact on said physical object (30);
- segmenting said entrance signal (12) to obtain a group of signal segments (15) and at least one physical value (16) processed from said entrance signal (12); - processing at least a characteristic vector (18) of signal features for each segment of said group of signal segments (15) using said at least one physical value (16) in order to learn metric function between gestures;
- supplying a specific exit signal (24) correlated to the specific gesture.
2. Method as in claim 1, characterized in that it also provides a step of memorizing said characteristic vectors (18) processed with an identification label
(21) of the gesture made on said physical object (30).
3. Method as in claim 1 or 2, characterized in that it also provides a step of comparing a new gesture made, in direct or indirect contact with said physical object (30), with the gestures previously memorized to recognize if the new gesture made is one of the memorized gestures, said comparison step being made by evaluating and learning the metric function between said characteristic vectors (18), processed from the new gesture made, and said characteristic vectors (18) processed from the gestures previously memorized.
4. Method as in any claim hereinbefore, characterized in that it also provides to calculate for each element of said characteristic vector (18) a metric function in order to quantify how much two gestures, identified by said identification labels (21), are similar, following the sub-steps:
- ordering the characteristic vectors (18) of the gestures according to their identification label (21);
- applying the metric function to compare characteristic vectors (18) of the gestures for classification;
- assessing the metric function on the ordered characteristic vector (18) of the gesture; - adjusting the metric function to minimize classification errors and iterate until the errors are minimum.
5. Method as in any claim hereinbefore, characterized in that it also provides a quality features step of said entrance signal (12), which can be carried out in parallel with the segmentation step, in order to obtain one or more adjustable descriptors of said gesture to assess the characteristics of said entrance signal (12).
6. Method as in any claim hereinbefore, characterized in that it also provides a sub-step (a) of extracting an envelope from said entrance signal (12) to obtain a signal defining the functional form of the amplitudes of said entrance signal (12).
7. Method as in claim 6, characterized in that it also provides a sub-step (b) of applying a low pass filter to said envelope of said entrance signal (12) to obtain a signal with a functional profile of said envelope without noise coming from other sources or external agents that are not generated by or correlated to the gesture made on said physical object (30).
8. Method as in claim 6 or 7, characterized in that it also provides a sub-step (c) of selection and division of said filtered envelope of said entrance signal (12) in said group of signal segments (15) comprising segments or parts of said entrance signal (12), said segments being temporally detected if they are greater than or equal to a threshold value of amplitude.
9. Method as in any claim from 6 to 8, characterized in that it also provides a sub-step (d) of calculating said at least one physical value (16) from said entrance signal (12), at least one of said physical value (16) being the gesture group velocity of said entrance signal (12) computed as the maximum in amplitude of the segmented signal.
10. Method as in any claim hereinbefore, characterized in that it also provides a sub-step (e) of calculating the fast Fourier transform (FFT) of said entrance signal (12) to obtain a vector of spectral amplitudes.
1 1. Method as in claim 10, characterized in that it also provides a sub-step (f) of calculating one or more descriptors computed as the first three spectral moments from said vector of spectral amplitudes.
12. Method as in any claim hereinbefore, characterized in that said exit signal (24) is a digital, electric, computer or optical signal that commands and/or controls a device (25) connected to said device (10) using said method.
13. Method as in any claim hereinbefore, characterized in that said exit signal (24) contains the values of likelihoods of a gesture made on said physical object
(30) with the previously memorized gestures, said likelihoods gestures being calculated by comparing said characteristic vectors (18) of the memorized gestures with the gesture made.
14. Device to recognize a gesture made in direct or indirect contact on a physical object (30), characterized in that it comprises at least:
- a transducer (1 1) associable in contact with said physical object (30) and configured to detect the vibrations and/or sounds generated by the gesture made on said physical object (30);
- a reception unit (13) configured to receive an entrance signal (12) from said transducer (1 1) generated from said gesture made on said physical object (30);
- a segmentation unit (14) configured to receive said entrance signal (12) and to obtain a group of signal segments (15) and at least one physical value (16) processed from said entrance signal (12);
- a processing unit (19) configured to calculate at least one characteristic vector (18) of signal features for each segment of said group of signal segments (15) and said at least one physical value (16);
- a command unit (23) configured to supply to a device (25) a specific exit signal (24) correlated to the specific gesture made.
15. Device as in claim 14, characterized in that said transducer (1 1) associable in contact with said physical object (30) is an adhesive piezo- transducer contact microphone configured to capture the sounds and/or the vibrations generated by a gesture made on said physical object (30).
16. Device as in claim 14 or 15, characterized in that it also comprises a memory unit (20) configured to memorize said at least one characteristic vector (18), processed by said entrance signal (12), and an identification label (21) defining the gesture made on said physical object (30).
17. Device as in any claim from 14 to 16, characterized in that it also comprises a comparison-learning unit (22) configured to recognize if a new gesture made on said physical object (30) is one of the gestures memorized in said memory unit (20).
18. Device as in any claim from 14 to 17, characterized in that it comprises a quality features unit (17) configured to obtain one or more adjustable descriptors of the gesture to assess the characteristics of said entrance signal (12) supplied to it by said reception unit (13).
PCT/EP2016/059451 2016-04-27 2016-04-27 Method to recognize a gesture and corresponding device WO2017186290A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP16721113.5A EP3308250A1 (en) 2016-04-27 2016-04-27 Method to recognize a gesture and corresponding device
JP2019507992A JP2019518294A (en) 2016-04-27 2016-04-27 Method and apparatus for recognizing gestures
PCT/EP2016/059451 WO2017186290A1 (en) 2016-04-27 2016-04-27 Method to recognize a gesture and corresponding device
CN201680087100.XA CN109416607A (en) 2016-04-27 2016-04-27 Method for identifying a gesture and relevant device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2016/059451 WO2017186290A1 (en) 2016-04-27 2016-04-27 Method to recognize a gesture and corresponding device

Publications (1)

Publication Number Publication Date
WO2017186290A1 true WO2017186290A1 (en) 2017-11-02

Family

ID=55948805

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2016/059451 WO2017186290A1 (en) 2016-04-27 2016-04-27 Method to recognize a gesture and corresponding device

Country Status (4)

Country Link
EP (1) EP3308250A1 (en)
JP (1) JP2019518294A (en)
CN (1) CN109416607A (en)
WO (1) WO2017186290A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090157206A1 (en) * 2007-12-13 2009-06-18 Georgia Tech Research Corporation Detecting User Gestures with a Personal Mobile Communication Device
US20160077615A1 (en) * 2014-09-11 2016-03-17 Qeexo, Co. Method and apparatus for differentiating touch screen users based on touch event analysis

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101678549B1 (en) * 2010-02-02 2016-11-23 삼성전자주식회사 Method and apparatus for providing user interface using surface acoustic signal, and device with the user interface
US9507464B2 (en) * 2013-03-15 2016-11-29 Elo Touch Solutions, Inc. Acoustic touch apparatus and methods using touch sensitive lamb waves
CN204945981U (en) * 2015-08-11 2016-01-06 深圳市凯健奥达科技有限公司 Surface acoustic wave touch screen

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090157206A1 (en) * 2007-12-13 2009-06-18 Georgia Tech Research Corporation Detecting User Gestures with a Personal Mobile Communication Device
US20160077615A1 (en) * 2014-09-11 2016-03-17 Qeexo, Co. Method and apparatus for differentiating touch screen users based on touch event analysis

Also Published As

Publication number Publication date
CN109416607A (en) 2019-03-01
EP3308250A1 (en) 2018-04-18
JP2019518294A (en) 2019-06-27

Similar Documents

Publication Publication Date Title
EP3614377B1 (en) Object recognition method, computer device and computer readable storage medium
JP5658261B2 (en) Method and apparatus for simultaneous touch detection of bending wave type touch screen
KR101932210B1 (en) Method, system for implementing operation of mobile terminal according to touching signal and mobile terminal
US9465461B2 (en) Object detection and tracking with audio and optical signals
US10606417B2 (en) Method for improving accuracy of touch screen event analysis by use of spatiotemporal touch patterns
KR101678549B1 (en) Method and apparatus for providing user interface using surface acoustic signal, and device with the user interface
Huang et al. Scream detection for home applications
CN108182418B (en) Keystroke identification method based on multi-dimensional sound wave characteristics
CN102890557A (en) Method and system for touch gesture detection in response to microphone output
Ono et al. Sensing touch force using active acoustic sensing
CN107133135B (en) Keyboard key detection method based on statistics and sound positioning
KR20140114766A (en) Method and device for sensing touch inputs
CN107219993A (en) Display methods and related electronic device
Foggia et al. Cascade classifiers trained on gammatonegrams for reliably detecting audio events
US20180032170A1 (en) System and method for estimating location of a touch object in a capacitive touch panel
CN109997186B (en) Apparatus and method for classifying acoustic environments
GB2576960A (en) Speaker recognition
US10817798B2 (en) Method to recognize a gesture and corresponding device
JPWO2019234429A5 (en)
EP3308250A1 (en) Method to recognize a gesture and corresponding device
Nakadai et al. Footstep detection and classification using distributed microphones
JP2021071586A (en) Sound extraction system and sound extraction method
WO2021103913A1 (en) Voice anti-counterfeiting method and apparatus, terminal device, and storage medium
JP7397801B2 (en) Apparatus for detecting a subject with breathing disorder and method of operating the apparatus for detecting a subject with breathing disorder
US20230101366A1 (en) Noise event detection and characterization

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 2016721113

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2019507992

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE