US9411468B2 - Irregular input identification - Google Patents
Irregular input identification Download PDFInfo
- Publication number
- US9411468B2 US9411468B2 US13/584,725 US201213584725A US9411468B2 US 9411468 B2 US9411468 B2 US 9411468B2 US 201213584725 A US201213584725 A US 201213584725A US 9411468 B2 US9411468 B2 US 9411468B2
- Authority
- US
- United States
- Prior art keywords
- touch
- image
- patch
- proximity image
- patches
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
- 230000001788 irregular Effects 0.000 title claims abstract description 19
- 238000000034 method Methods 0.000 claims abstract description 57
- 210000003811 finger Anatomy 0.000 claims description 35
- 210000003813 thumb Anatomy 0.000 claims description 27
- 239000006185 dispersion Substances 0.000 claims description 18
- 230000002093 peripheral effect Effects 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 13
- 210000000624 ear auricle Anatomy 0.000 claims description 12
- 238000009499 grossing Methods 0.000 claims description 8
- 238000012886 linear function Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 3
- 230000005684 electric field Effects 0.000 claims description 2
- 238000012885 constant function Methods 0.000 claims 2
- 238000005259 measurement Methods 0.000 description 18
- 102100029158 Consortin Human genes 0.000 description 15
- 101000771062 Homo sapiens Consortin Proteins 0.000 description 15
- 238000013459 approach Methods 0.000 description 14
- 238000001514 detection method Methods 0.000 description 9
- 230000009286 beneficial effect Effects 0.000 description 8
- 230000004044 response Effects 0.000 description 7
- 230000008901 benefit Effects 0.000 description 6
- 238000003709 image segmentation Methods 0.000 description 6
- 230000001419 dependent effect Effects 0.000 description 5
- 101001033293 Homo sapiens Interleukin enhancer-binding factor 3 Proteins 0.000 description 4
- 102100039062 Interleukin enhancer-binding factor 3 Human genes 0.000 description 4
- 210000005069 ears Anatomy 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 229920006395 saturated elastomer Polymers 0.000 description 4
- 230000007704 transition Effects 0.000 description 4
- 210000000689 upper leg Anatomy 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 210000003467 cheek Anatomy 0.000 description 3
- 210000003128 head Anatomy 0.000 description 3
- 210000002414 leg Anatomy 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 2
- 238000001994 activation Methods 0.000 description 2
- 238000010420 art technique Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000000875 corresponding effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 210000004905 finger nail Anatomy 0.000 description 2
- 210000004247 hand Anatomy 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000007935 neutral effect Effects 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 238000012876 topography Methods 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000001680 brushing effect Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 210000000613 ear canal Anatomy 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 230000005057 finger movement Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000007788 liquid Substances 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000010845 search algorithm Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 210000004243 sweat Anatomy 0.000 description 1
- 230000008961 swelling Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000002618 waking effect Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/0416—Control or interface arrangements specially adapted for digitisers
- G06F3/0418—Control or interface arrangements specially adapted for digitisers for error correction or compensation, e.g. based on parallax, calibration or alignment
- G06F3/04182—Filtering of noise external to the device and not generated by digitiser components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/0416—Control or interface arrangements specially adapted for digitisers
- G06F3/0418—Control or interface arrangements specially adapted for digitisers for error correction or compensation, e.g. based on parallax, calibration or alignment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/0416—Control or interface arrangements specially adapted for digitisers
- G06F3/04166—Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/0416—Control or interface arrangements specially adapted for digitisers
- G06F3/0418—Control or interface arrangements specially adapted for digitisers for error correction or compensation, e.g. based on parallax, calibration or alignment
- G06F3/04186—Touch location disambiguation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
- G06F3/0421—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means by interrupting or reflecting a light beam, e.g. optical touch-screen
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/044—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by capacitive means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G06K9/00375—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration by the use of local operators
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G06T7/401—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/40—Analysis of texture
- G06T7/41—Analysis of texture based on statistical description of texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/107—Static hand or arm
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/041—Indexing scheme relating to G06F3/041 - G06F3/045
- G06F2203/04101—2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/041—Indexing scheme relating to G06F3/041 - G06F3/045
- G06F2203/04104—Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
Definitions
- the invention relates generally to data input methods and devices for electronic equipment and, more particularly, to methods and devices for discriminating between various inputs to a multi-touch touch-surface input device.
- touch-surfaces There currently exist many types of input devices for performing operations with an electronic system. These operations often correspond to moving a cursor and/or making selections on a display screen.
- Illustrative electronic systems include tablet, notebook, desktop and server computer systems, personal digital assistants, audio and video control systems, portable music and video players and mobile and satellite telephones.
- touch pad and touch screen systems collectively “touch-surfaces”
- Touch screens typically include a touch panel, a controller and a software driver.
- the touch panel is characteristically an optically clear panel with a touch sensitive surface that is positioned in front of a display screen so that the touch sensitive surface is coextensive with a specified portion of the display screen's viewable area (most often, the entire display area).
- the touch panel registers touch events and sends signals indicative of these events to the controller.
- the controller processes these signals and sends the resulting data to the software driver.
- the software driver in turn, translates the resulting data into events recognizable by the electronic system (e.g., finger movements and selections).
- touch-surfaces now becoming available are capable of simultaneously detecting multiple objects as they approach and/or contact the touch-surface, and detecting object shapes in much more detail. To take advantage of this capability, it is necessary to measure, identify and distinguish between the many kinds of objects that may approach or contact such touch-surfaces simultaneously.
- Prior art touch-surface systems do not provide a robust ability to do this.
- it would be beneficial to provide methods and devices that identify and discriminate multiple simultaneous hover or touch events such as, for example, two or more closely grouped fingers, palm heels from one or more fingers, fingers from thumbs, and fingers from ears and cheeks.
- the invention provides a method to process a proximity image from a multi-touch touch surface.
- the method includes obtaining a dispersion image having a plurality of pixels, determining an irregularity measure value for the dispersion image and controlling the operation of a multi-touch touch-surface device if the irregularity measure value is above a specified threshold.
- the irregularity measure is determined for the entire proximity image.
- an irregularity measure is determined for one or more regions within the image (e.g., patches).
- Device control may be manifested through, for example, changing the device's operating mode (e.g., off to on) or ignoring the identified irregular object so that it does not cause a change in the device's operating state.
- Illustrative multi-touch touch-surface devices include, but are not limited to, tablet computer systems, notebook computer systems, portable music and video systems and mobile telephones. Methods in accordance with any of the described methodologies may be stored in any media that is readable and executable by a programmable control device such as, for example, by a general purpose computer processor.
- FIG. 1 shows, in flowchart form, a multi-touch processing methodology in accordance with one embodiment of the invention.
- FIG. 2 shows, in flowchart form, a patch irregularity calculation in accordance with one embodiment of the invention.
- FIG. 3 shows a plot of empirically determined data illustrating patch minor radii's ability to discriminate between large touch-surface contacts (cheeks, for example) and other touch-surface contacts (fingertips and thumbs, for example).
- FIG. 4 shows a plot of empirically determined data illustrating patch minor radii's ability to discriminate between palm contacts and other touch-surface contacts (e.g., fingertips and thumbs).
- FIG. 5 shows a plot of empirically determined data illustrating a patch irregularity measure's ability to discriminate between ear contacts and other touch-surface contacts (e.g., fingertips, thumbs and cheeks).
- FIG. 6 shows, in flowchart form, far-field operations in accordance with one embodiment of the invention.
- FIG. 7 shows, in block diagram form, a touch-surface device in accordance with one embodiment of the invention.
- touch-surface input devices are capable of simultaneously detecting multiple objects as they approach and/or contact the touch-surface.
- a hand-held multi-touch touch-surface device that may be put into a pocket, purse, or held against the head (e.g., portable music player, portable video player, personal digital assistant or mobile phone), detecting when the device is being clasped on the way into or out of the pocket, against the body, or against the head is very useful for: input rejection (ensuring that touch-surface input signals generated as a result of these actions are not mistaken for normal finger/stylus touches); operational mode transitions (e.g., dimming the device's backlight, putting the device to sleep and waking the device from a low-power state); and, for mobile telephones, answering calls (e.g., when the device is brought near, but not necessarily touching the head) and/or terminating calls (e.g., when the unit is placed into a pocket or purse).
- input rejection ensuring that touch-surface input signals generated as a result of these actions are
- Each sensing element in a two dimensional array of sensing elements (i.e., a touch-surface) generates an output signal indicative of the electric field disturbance (for capacitance sensors), force (for pressure sensors) or optical coupling (for optical sensors) at the sensor element.
- the ensemble of pixel values represents a“proximity image.”
- various embodiments of the invention address the ability to detect and discriminate between touch-surface signals (represented as a proximity image) resulting from, for example, the types of actions identified in the preceding paragraph.
- multi-touch processing methodology 100 begins with the acquisition of proximity image data (block 105 ). Because the acquired data is usually a superposition of information (indicating an object close to or in contact with the touch-surface) fixed offsets (due to circuitry baselines) and noise (e.g., radio frequency interference), an initial adjustment to acquired pixel data may be made to compensate for sensor element baseline activity. For example, on multi-touch device initialization and/or when being brought out of a low-power mode (e.g., sleep), one or more images may be captured. By assuming these initial images include no surface contacts, they may be used to provide the sensor's baseline.
- a low-power mode e.g., sleep
- baseline pixel values may be slowly adjusted over time to compensate for temperature or static charge.
- initial baseline values may need to be adjusted if, in fact, touch-surface contacts were present at start-up.
- a plurality of image samples may be acquired each at a different sensor element driving frequency. For each pixel in these images, the mean or median of subtracted samples (i.e., between the captured baseline and information images) may be combined to create an initial (typically signed) image in accordance with block 105 . For noise that occasionally generates large outlier pixel values (“spiky” noise), other rank-order filters may be useful. As noted in FIG. 1 , proximity image data resulting from operations in accordance with block 105 is denoted [PROX].
- [PROX] image data feeds other processing blocks that may operate sequentially or in parallel with one another (blocks 110 , 115 and 120 ). It has been found that filtering or smoothing a proximity image (block 115 ) prior to segmentation (block 125 ) reduces the number of spurious peaks and thus helps reduce over segmentation.
- each pixel value may be averaged with its nearest neighbor pixels in accordance with a discrete diffusion operation. If this approach is employed, it has been found beneficial to insert a “border” around the captured image so that there is a value with which to average the pixels at the edge of the captured image.
- a one (1) pixel border may be added to the [PROX] image—where each “border” pixel is assigned a value corresponding to the image's “background” (e.g., zero).
- both temporal (e.g., obtaining multiple images over a period of time) and spatial (e.g., averaging neighbor pixels) smoothing operations may be used. Multiple smoothing operations may be beneficial if the captured pixel data is particularly noisy.
- image data resulting from operations in accordance with block 115 is denoted [SMTH].
- [PROX] image pixel values are typically zero or positive in response to an object contacting the touch-surface (aka, a “grounded” object), background noise or objects close to but not touching the touch-surface (aka “ungrounded” objects) may produce an image some of whose pixel values are negative.
- Background noise may be static or vary with circuit temperature, touch-surface moisture, or other factors.
- noisy, negative pixels can cause excessive jitter in centroid and other patch measurements (see discussion below regarding block [ 135 ]).
- [PROX] image pixel values may be confined to a desired, typically positive, range (block 110 ). Subtracting the noise threshold helps reduce centroid jitter induced from pixels that wander around (above and below) the noise threshold in successive image frames. As noted in FIG.
- image data resulting from operations in accordance with block 110 is denoted [CNST].
- all pixel whose values are less than a background noise threshold are set to zero.
- a noise-threshold is subtracted from each pixel value and the result is forced to be non-negative, as shown in Table 1.
- the noise-threshold value is set to between 1 and 3 standard deviations of the noise measured at each pixel and the background-value is set to zero.
- the noise-threshold value is set to between 1 and 3 standard deviations of the noise measured at each pixel and the background-value is set to zero.
- the noise threshold may be set to a specified expected value on a per-pixel basis or a single value may be used for all pixels in an image.
- pixel noise values may be allowed to vary over time such that thermal and environmental effects on sensor element noise may be compensated for.
- Touch-surface contacts typically show up as grouped collections of “active” pixel values, where each region of fleshy contact (e.g. finger, palm, cheek, ear or thigh) is represented by a roughly elliptical patch of pixels.
- image segmentation operations can identify distinct pixel patches that correspond to touch-surface contacts (block 125 ).
- bottom-up, ridge-hiking algorithms may be used to group pixels that are part of the same watershed around each peak pixel—each watershed group or pixel patch corresponds to a touch-surface contact.
- top-down search algorithms may be used to identify pixel patches surrounding each peak pixel, starting from the peak, searching outward and stopping at valleys.
- one-dimensional patches may be culled from the identified patches in that they generally result from isolated noise spikes or failure of an entire row or column of sensor elements and/or associated circuitry.
- multiple detected patches may be merged to produce a reduced number of patches for further processing. Heuristic or empirically determined rules may, for example, be applied to accomplish this. For example, two separately identified patches may be merged when the saddle point along their shared border is not “very deep”—e.g., when the saddle magnitude is more than 60% to 80% of the two patches' peak pixel values. As noted in FIG. 1 , identified patches resulting from operations in accordance with block 125 are denoted [P 1 , P 2 , . . . Pn].
- performing patch parameterization on diffused images may reduce noise from peripheral pixels, but standard spatial filtering processes also cause swelling and distortion of patch shape, cause adjacent patches to spread into one another and other effects that bias centroid and ellipse radii measurements in particular.
- a technique is needed that minimizes the amount of noise from patch periphery pixels without strongly distorting patch shape and ensuing measurements.
- patch peripheral pixel values may be selectively reduced, down-scaled or dampened (block 130 ).
- patch centroid determination may be improved by selectively down-scaling patch peripheral pixels that are fairly weak and whose neighbors are very weak. More specifically, in one embodiment calibrated image pixel values (e.g., in [CNST]) whose corresponding smoothed value (e.g., in [SMTH]) falls within a specified range defined by a lower-limit and an upper-limit are reduced in proportion to where the smoothed value falls within that range. Lower and upper limits are chosen empirically so that only those pixel values that are relatively weak (compared to patch peak values and background noise) are manipulated.
- the lower-limit is set, on a pixel-by-pixel basis, to approximately twice the background noise standard deviation and the upper-limit is set to approximately four times the background noise standard deviation (with the background value typically being zero).
- the lower-limit is set to a value indicative of the “average” or “expected” noise across all pixels in the proximity image.
- the noise value may change dynamically to reflect changing operational conditions (see comments above).
- FIG. 1 an image whose peripheral patch pixels have been dampened in accordance with block 130 is denoted [CNST'].
- peripheral patch pixels are dampened as shown in Table 2.
- Patch peripheral pixel dampening such as described above is equally applicable to touch-surfaces that provide one-dimensional proximity images.
- projection scan touch-surfaces provide an output value (or signal) for each row and column of sensor elements in a touch-surface.
- a “patch” comprises a plurality of values, where each value represents a row or column measurement. The values at the ends of these patches (i.e., the peripheral values) may benefit from noise dampening as described here.
- ear and earlobe may contact the touch-surface sooner or more often than the cheek during calls.
- earlobe patches can be very close in size to finger and thumb patches—but should, nevertheless, not cause spurious finger-button activations during a call.
- a measurement of patch irregularity is defined that does not look for any specific ear (patch) shape, but rather indicates a general roughness, non-roundness or folds in the pixel patch (block 120 ).
- a patches' irregularity measure is above a specified threshold, the contact is identified as an irregular object (e.g., not a cheek, finger or palm), otherwise the patch is identified as not an irregular object (e.g., a cheek, finger or palm).
- patch irregularity determination methodology 120 begins with the computation of a dispersion image (block 200 ).
- the dispersion image (denoted [DISP] in FIG. 2 ) may be any high-pass filtered version of the initial proximity image [PROX].
- the total energy for each patch [P 1 , P 2 , . . . Pn] is computed (block 205 ).
- a patches' total energy may be calculated by summing the square of each pixel value in the patch. This may be expressed mathematically as follows:
- the total energy between adjacent pixels in a patch is then determined (block 210 ). To reduce the effect of energy spikes for pixel patches straddling an edge, the summations below should neglect (i.e., assume a value of zero) contributions from pixels whose neighboring pixels are at the image's border, see EQ. 3. For the same reason, the summations below should ignore contributions from pixels whose neighboring pixels are from a different patch.
- peak patch energy values resulting from operations in accordance with block 215 are denoted [PE 1 , . . . PEn].
- an irregularity measure for each patch is calculated (block 220 ).
- the irregularity measure is defined as the ratio of a patches' spatial energy minus its peak energy to the patches' total energy:
- patch irregularity measure values resulting from operations in accordance with block 220 are denoted [IM 1 , . . . IMn].
- Alternative embodiments might discard any negative pixels values generated during computation of pixel energy values, or take absolute pixel values rather than squaring pixel values when calculating patch energies.
- the irregularity measure may be based on the proximity image as a whole. That is, the entirety of the dispersion image (i.e., all pixels) may be treated as a single “patch” for purposes of generating an irregularity measure value.
- abnormal touch-surface surface conditions may be detected, and responded to, prior to segmentation operations in accordance with block 125 (see FIG. 1 ).
- Illustrative abnormal touch-surface surface conditions include, but are not limited to, liquid (e.g., water or sweat) on the touch-surface or multiple irregular objects in close proximity to or in contact with the touch-surface (e.g., coins and/or keys). When these conditions are detected, it may be beneficial to acquire new sensor element baseline values.
- an irregularity measure may be computed at each of the frequencies. If one or more of the computed irregularity measure values is greater than a specified threshold as discussed above, the sampling frequencies associated with the above-threshold values may be deemed to be affected by an excessive amount of noise and ignored (e.g., radio frequency noise). Periodic determination of frequency-dependent irregularity measures in this manner may be used to detect when such noise sources occur and when they disappear. For example, due to a touch-surface devices operating environment changes.
- an oddly shaped collection of pixels i.e., a patch
- a relatively large set of numbers to define its boundary and signal value at each pixel within the patch.
- One benefit of this approach is that an ellipse is completely described by a relatively small collection of numbers—its center coordinates, major and minor axis lengths, and major axis orientation.
- centroid or center-of-mass computations may be used to parameterize each patch (block 135 ).
- a patches' centroid may be determined using these techniques and the [CNST'] image (see block 130 ).
- the [CNST'] image may be used to generate patch covariance matrices whose Eigenvalues identify a patches' major and minor radii and whose Eigenvectors identify the patches' orientation.
- patch characteristics are also computed:
- a discrimination technique in accordance with one embodiment of the invention uses a patches' signal density parameter (see, for example, EQS. 7 and 8). It has been found that this approach provides a robust means to distinguish objects that contact the touch-surface from those that are held or hovering above the surface—regardless of the object's size.
- the patch signal density parameter is normalized such that a firm fingertip contacting the touch-surface produces a peak value of 1, then a lightly brushing contact is indicated by a patch density value of slightly greater than 0.5 (e.g., half the normalized value) while a hovering object would be indicated by a patch density value equal to or slightly less than 0.5. It will be recognized that what constitutes “slightly greater” or “slightly less” is dependent upon factors such as the type of sensor elements used in the touch-surface and the associated sensor element drive and measure circuitry. Accordingly, while the precise determination of a threshold value based on patch signal density will require some experimentation, it would be will within the purview of an artisan of ordinary skill with benefit of this disclosure.
- fingernail touches also produce patch signal density values generally greater than approximately 0.5. This is because the non-conductive fingernail holds the conductive finger flesh more than approximately 1 millimeter above the touch-surface. Accordingly, a threshold operation based on patch signal density is also a reliable means for discriminating between fleshy fingertip touches and back-of-fingernail touches. This same technique has also been found to reliably determine whether large objects (e.g., cheeks and palms) are hovering or actually in contact with the touch-surface.
- large objects e.g., cheeks and palms
- the various types of touch-surface contacts may be discriminated (block 140 ).
- large objects e.g., cheeks and palms
- irregular objects e.g., ears
- regular objects e.g., fingers, thumbs, cheeks and palms
- finger-clasp actions e.g., when a user claps a multi-touch touch-surface device to put it into or withdraw it from a pocket. Identification of and discrimination between these types of touch-surface inputs permits an associated device to be controlled in a more robust manner.
- detection of a large object may be used to transition the device from one operational state (e.g., off) to another (e.g., on).
- input identified as the result of a large or irregular object which might normally cause a state transition, may be safely ignored if in one or more specified states. For example, if a touch-surface telephone is already in an “on” or “active” state, identification of a large or irregular object may be ignored.
- a contact's minor radius measure provides a robust discriminative measure to accomplish this. If a patches' minor radius exceeds a specified threshold, the contact can reliably be classified as a cheek—as opposed to a finger or thumb, for example. This same measurement can also detect a nearby leg (e.g., thigh) through a few millimeters of fabric (e.g. when a device is inserted in the pocket with its touch-surface facing the body).
- FIG. 3 illustrative empirical data is shown that illustrates the distinction between cheek contacts 300 and other contacts 305 (e.g., fingertips and thumbs) based on patch minor radii. While the exact values for patch contacts may vary from sensor to sensor and population to population, it is clear from FIG. 3 that threshold 310 may be made anywhere between approximately 11 millimeters and approximately 15 millimeters.
- threshold 310 is described by a constant value (i.e., dependent only upon patch minor radius), this is not necessary.
- threshold 310 may be described by a linear or non-linear relationship between multiple parameters such patch minor-radius and patch signal density (see discussion below regarding FIG. 4 ).
- a similar size testing may be performed using a patches' major or geometric mean radius (i.e.,
- the palm versus other object decision threshold 410 may be reduced for contacts with lower signal density because hovering or lightly touching fingers produce lower minor radii than firmly touching fingers, whereas palms tend to produce large minor radii even when hovering. Accordingly, decision threshold 410 may be represented by a straight curve with a small positive slope. While for patch contacts will vary as noted above, it is clear from FIG. 4 that threshold 410 may be made to distinguish palm contacts from other contacts. Using this approach, there is virtually no risk that a hovering palm (a contact that typically produces a patch signal density value similar to that of a touching finger) will mistakenly be interpreted as a cursor move or button activation (e.g., a “click” event).
- a hovering palm a contact that typically produces a patch signal density value similar to that of a touching finger
- Ear and earlobe contacts can generate patches that are roughly the same size as those generated by fingers and thumbs. It has been found, however, that the creases, ridges, and generally rough topography of the ear do produce proximity images unique from fingers and thumbs, at least if the imaging sensor (i.e., touch-surface) covers a significant portion of the ear (i.e. not just the fleshy lobule).
- the irregularity measure described above is one way to characterize contact roughness (see EQ. 5). This permits a robust means to discriminate between contacts due to ears and earlobes from contacts due to fingers, thumbs, cheeks, thighs and palms.
- threshold 510 comprises a linear step-like or spline structure with a first level at an irregularity measure of between approximately 1.0 to 1.2 and a second level at approximately between 1.1 and 1.2.
- a single linear function having a positive slope may be used.
- higher level functions may be used to segregate the various contact types.
- patch contacts may vary from those shown in FIG. 5
- most rough object contacts may be distinguished from most smooth or regular object contacts using the defined irregularity measure—where the exact nature or form of a decision threshold (e.g., threshold 510 ) is dependent upon the precise implementation, operational goals and capabilities of the target multi-touch device.
- successive proximity images are used to track objects as they move across a touch-surface.
- its associated patch(es) may be correlated through overlap calculations. That is, patches identified in successive images that overlap in a specified number of pixels (or fraction of patch pixels) may be interpreted to be caused by the same object.
- the maximum patch minor radius over the life of the tracked contact may be compared to the thresholds discussed above (e.g., thresholds 310 in FIG. 3, 410 in FIGS. 4 and 510 in FIG. 5 ). This approach ensures that a, for example, palm contact does not lose its palm identity should its minor radius temporarily fall below the decision threshold (e.g., 410 ).
- a decision threshold is not a constant value (e.g., 310 ) but rather some curve (e.g., 410 and 510 ), it may be advantageous to apply a density-correction to the instantaneous minor radius prior to the maximum minor radius accumulation operation described here.
- finger-clasps can be detected via any one of the following criteria:
- multi-touch processing methodology may include far-field processing.
- far-field processing refers to the detection and processing associated with bodies (e.g., fingers, palms, cheeks, ears, thighs, . . . ) that are close to (e.g., less than one millimeter to more than a centimeter) but not in contact with the touch-surface.
- bodies e.g., fingers, palms, cheeks, ears, thighs, . . .
- the ability to detect far-field objects may be beneficial in touch-surface devices that, during normal use, are brought into close proximity to a user.
- a telephone that includes touch-surface for user input (e.g., dialing).
- initial far-field processing may be performed after proximity image data is acquired. That is, after operations in accordance with block 105 in FIG. 1 .
- a first step subtracts a small noise factor from the initially acquired proximity image to create a negative-background far-field image (block 600 ):
- Negative Far-Field Image [PROX] ⁇ (Noise Factor) EQ. 9
- the noise factor may be set to between approximately 1 and 2 standard deviations of the average noise measured or expected over the entire image. This will cause most pixels in the resulting negative far-field image to be slightly negative rather than neutral in the absence of any touch-surface contact.
- the negative far-field image resulting from operations in accordance with block 600 is denoted [NFAR].
- each pixel in the [NFAR] image is saturated to the highest level expected from an object hovering a few millimeters from the touch-surface (block 605 ).
- the resulting far-field saturation image (denoted [SFAR] in FIG. 6 ) is generated as shown in Table 3.
- the saturation limit value should be less than the peak pixel value from fingers or thumbs hovering within approximately 1 to 2 millimeters of the touch-surface, but not so low as to cause the resulting [SFAR] image to lose to much information content. While the precise far-field saturation limit value will vary from implementation to implementation (due to differences in sensor element technology and associated circuitry), it has been determined empirically that a suitable value will generally lie between +3 standard deviations and +6 standard deviations of noise associated with the initial far-field image. (Again, this noise may be on a per-pixel, or whole image basis.)
- an infinite impulse response filter technique may be used wherein two or more consecutive [SFAR] images are averaged together.
- an infinite impulse response filter technique may be used to generate a smoothed image. It will be recognized that an infinite impulse response filter generates a weighted running average (or auto-regressive) image.
- an infinite impulse response filter combines the current far-field saturated image (e.g., [SFAR]new) with the immediately prior far-field saturated image (e.g., [SFAR]prior) in a one-third to two-thirds ratio.
- a filtered far-field saturated image generated in accordance with block 610 is denoted [FARF].
- a weighted average of non-linearly scaled background pixel values may be used to generate a scalar far-field indicator value (FAR-FIELD) in accordance with the invention as follows:
- FAR-FIELD ⁇ background ⁇ ⁇ i , j ⁇ ENeg ⁇ ( [ FARF ] [ i ] ⁇ [ j ] ) ⁇ [ LOC [ i ] ⁇ [ j ] ] ⁇ background ⁇ ⁇ i , j ⁇ [ LOC [ i ] ⁇ [ j ] ] EQ . ⁇ 10
- the ENeg( ) function non-linearly amplifies pixel values below a threshold (e.g., zero)
- [LOC] represents a pixel weighting mechanism.
- only proximity image background pixels contribute to the computed FAR-FIELD value. That is, pixels identified as belonging to a patch during image segmentation operations are excluded during far-field measurement operations.
- the ENeg( ) function disproportionately emphasizes the contributions from background pixels as follows:
- ENeg ⁇ ( pixel ⁇ ⁇ value ) ⁇ pixel ⁇ ⁇ value for 0 ⁇ pixel ⁇ ⁇ value ⁇ B 2 ⁇ pixel ⁇ ⁇ value for ( - B ⁇ 2 ) ⁇ pixel ⁇ ⁇ value ⁇ 0 B + ( 3 ⁇ pixel ⁇ ⁇ value ) for pixel ⁇ ⁇ value ⁇ ( - B ⁇ 2 ) EQ . ⁇ 11
- B represents a far-field saturation limit value.
- B is chosen to permit a small number of negative pixels to cancel out a finger or thumb-sized patch of positive pixels. In this way, only a nearly full coverage cheek-sized patch of positive pixels, plus a small remainder of neutral/background pixels, can produce a strongly positive far-field measurement.
- disproportionately emphasizing the contributions from background pixels in accordance with EQs. 10 and 11 permits the FAR-FIELD measurement to be more selective for bodies large enough to positively affect most of a touch-surface's pixel (e.g., cheeks and legs), while not being overly influenced by medium-sized objects (e.g., hovering thumbs). For example, if a hovering thumb causes half of a touch-surface's sensor elements to have a slightly above-background pixel value, disproportionately emphasizing the half that remain below background will keep the measured FAR-FIELD value below zero—indicating no large object is “near” the touch-surface (e.g., within 1 to 3 centimeters).
- background pixels may be linearly combined (e.g., summed).
- [LOC] represents a pixel weighting mechanism.
- selectivity for large bodies may be improved, however, by lowering the weights near the bottom and side edges (for example, to values between 0.1 and 0.9). Doing this can lessen false-positive contributions from a hand whose fingers wrap around the device during (clasping) operations.
- the weights along the top edge may be kept at full strength.
- a FAR-FIELD value greater than a specified threshold indicates a large “near by” object has been detected.
- this information may be used to transition the touch-surface device into a specified mode (e.g., on, off or low power).
- far-field measurements may be combined with other measurements (e.g., the irregularity measure) to provide improved ear detection. For example, when a touch-surface is partly against an ear and also hovering a centimeter or two from a cheek, a weak ear pixel patch may be segmented in accordance with block 125 at the top of the screen.
- the middle and bottom of the touch-surface would only be affected by the cheek's far-field. Even if the FAR-FIELD measurement as taken outside the ear patch is not strong enough to exceed the specified far-field threshold on its own, the FAR-FIELD value can be added or otherwise combined with (weak) patch density and irregularity measure indicators such that the sum or combination surpasses an ear detection threshold.
- one or more proximity sensors may be positioned above the touch-surface's top edge or around, for example, a telephone's receiver opening.
- Illustrative proximity sensors of this type include, but are not limited to, active infrared-reflectance sensors and capacitance-sensitive electrode strips.
- ear ridges may trigger the proximity sensor.
- the earlobe may cause a small pixel patch in the top portion of the touch-surface. Discrimination operations in accordance with block 140 could decide that when a pixel patch at the top of the touch-surface is accompanied by any significant receiver proximity trigger, the pixel patch must be an ear, not a finger.
- the same conditions but with a significant FAR-FIELD value for the lower portion of the touch-surface may be used to trigger detection of an ear at the top of the touch-surface.
- one or more of signal density (see EQs. 7 and 8), patch irregularity (see EQ. 5), FAR-FIELD measurement (see EQ. 10) and proximity sensor input may be combined (e.g., a weighted average) so that ear detection can trigger when multiple indicators are weakly active, or just one indicator is strongly active.
- contact discrimination parameters such as a patches' centroid, minor axis radius, patch irregularity (EQ. 5), patch signal density (EQs. 7 and 8), far-field (EQ.
- proximity sensor input may be (low-pass) filtered to help counteract their often sporadic nature. This may be particularly beneficial if the filters employ adaptive time constants that rise quickly in response to rising input values, but decay more slowing when input values drop and/or are missing.
- a touch-surface device 700 of the type described herein is shown in block diagram form.
- a touch-surface device is any device that receives user input from a multi-touch capable touch-surface component (i.e., an input unit that provides user input in the form of a proximity image).
- a multi-touch capable touch-surface component i.e., an input unit that provides user input in the form of a proximity image.
- Illustrative touch-surface devices include, but are not limited, to tablet computer system, notebook computer systems, portable music and video display devices, personal digital assistants and mobile telephones.
- touch-surface element 705 includes sensor elements and necessary drive and signal acquisition and detection circuitry.
- Memory 710 may be used to retain acquired proximity image information (e.g., [PROX] image data) and by processor 715 for computed image information (e.g., patch characterization parameters).
- Processor 715 represents a computational unit or programmable control device that is capable of using the information generated by touch-surface element 705 to determine various metrics in accordance with FIG. 1 .
- external component 720 represents an entity that uses the generated information. In the illustrative embodiment, external component 720 may obtain information from processor 715 or directly from memory 710 .
- processor 715 could maintain a data structure in memory 710 to retain indication of, for example, large body contact status, large body far-field status, irregular object indication status, proximity sensor status (if utilized), flat finger clasp status and normal finger touch status.
- each status may be indicated by a single Boolean value (i.e., a flag).
- patch peripheral pixel noise is not suppressed (see block 130 ).
- patch peripheral pixel noise suppression is employed but no patch irregularity measure is made (see block 120 ).
- both patch peripheral pixel noise suppression and patch irregularity measures are determined and used.
- patch parameterization operations in accordance with block 135 use the [CNST] image and not the [CNST'] image as discussed above (see Table 2).
- patch parameterization operations in accordance with block 135 do not need to rely on statistical ellipse fitting. They could instead sum patch perimeter pixels and compare the obtained value to all patch pixels or attempt polygonal fits. Further, calibration operations (see Tables 1 and 2) may be delayed until, or made part of, image segmentation operations (block 125 ). In addition, it may be beneficial for purposes of image segmentation to mark pixels that are at, or have been set to, the background level (e.g., during operations in accordance with block 110 ).
- flat finger clasps may be used as a distinct gesture commanding a particular operation like locking the screen, going to sleep, or terminating a telephone call.
- touch-surface element 705 may incorporate memory (e.g., 710 ) and/or processor (e.g., 715 ) functions.
- external component 720 may represent a hardware element (e.g., a host processor) or a software element (e.g., a driver utility).
- a programmable control device may be a single computer processor, a special purpose processor (e.g., a digital signal processor, “DSP”), a plurality of processors coupled by a communications link or a custom designed state machine.
- DSP digital signal processor
- Custom designed state machines may be embodied in a hardware device such as an integrated circuit including, but not limited to, application specific integrated circuits (“ASICs”) or field programmable gate array (“FPGAs”).
- Storage devices suitable for tangibly embodying program instructions include, but are not limited to: magnetic disks (fixed, floppy, and removable) and tape; optical media such as CD-ROMs and digital video disks (“DVDs”); and semiconductor memory devices such as Electrically Programmable Read-Only Memory (“EPROM”), Electrically Erasable Programmable Read-Only Memory (“EEPROM”), Programmable Gate Arrays and flash devices.
- EPROM Electrically Programmable Read-Only Memory
- EEPROM Electrically Erasable Programmable Read-Only Memory
- flash devices such as Electrically Programmable Read-Only Memory (“EPROM”), Electrically Erasable Programmable Read-Only Memory (“EEPROM”), Programmable Gate Arrays and flash devices.
Abstract
Techniques for identifying irregular objects in contact with, or in close proximity to, a touch-surface are described. An irregularity measure is determined based on the regions intrinsic characteristics (e.g., energy content) rather than on the shape (or pattern) of the pixels within the region.
Description
This application is a continuation of U.S. patent application Ser. No. 11/619,490, filed Jan. 3, 2007, the entire disclosure of which is incorporated herein by reference in its entirety for all purposes.
The invention relates generally to data input methods and devices for electronic equipment and, more particularly, to methods and devices for discriminating between various inputs to a multi-touch touch-surface input device.
There currently exist many types of input devices for performing operations with an electronic system. These operations often correspond to moving a cursor and/or making selections on a display screen. Illustrative electronic systems include tablet, notebook, desktop and server computer systems, personal digital assistants, audio and video control systems, portable music and video players and mobile and satellite telephones. The use of touch pad and touch screen systems (collectively “touch-surfaces”) has become increasingly popular in these types of electronic systems because of their ease of use and versatility of operation.
One particular type of touch-surface is the touch screen. Touch screens typically include a touch panel, a controller and a software driver. The touch panel is characteristically an optically clear panel with a touch sensitive surface that is positioned in front of a display screen so that the touch sensitive surface is coextensive with a specified portion of the display screen's viewable area (most often, the entire display area). The touch panel registers touch events and sends signals indicative of these events to the controller. The controller processes these signals and sends the resulting data to the software driver. The software driver, in turn, translates the resulting data into events recognizable by the electronic system (e.g., finger movements and selections).
Unlike earlier input devices, touch-surfaces now becoming available are capable of simultaneously detecting multiple objects as they approach and/or contact the touch-surface, and detecting object shapes in much more detail. To take advantage of this capability, it is necessary to measure, identify and distinguish between the many kinds of objects that may approach or contact such touch-surfaces simultaneously. Prior art touch-surface systems (including their supporting software and/or circuitry) do not provide a robust ability to do this. Thus, it would be beneficial to provide methods and devices that identify and discriminate multiple simultaneous hover or touch events such as, for example, two or more closely grouped fingers, palm heels from one or more fingers, fingers from thumbs, and fingers from ears and cheeks.
In one embodiment the invention provides a method to process a proximity image from a multi-touch touch surface. The method includes obtaining a dispersion image having a plurality of pixels, determining an irregularity measure value for the dispersion image and controlling the operation of a multi-touch touch-surface device if the irregularity measure value is above a specified threshold. In one embodiment, the irregularity measure is determined for the entire proximity image. In another embodiment, an irregularity measure is determined for one or more regions within the image (e.g., patches). Device control may be manifested through, for example, changing the device's operating mode (e.g., off to on) or ignoring the identified irregular object so that it does not cause a change in the device's operating state.
Illustrative multi-touch touch-surface devices include, but are not limited to, tablet computer systems, notebook computer systems, portable music and video systems and mobile telephones. Methods in accordance with any of the described methodologies may be stored in any media that is readable and executable by a programmable control device such as, for example, by a general purpose computer processor.
Methods and devices to detect and discriminate between multiple simultaneous close approaches or touches to a touch-surface are described. The following embodiments are presented to enable any person skilled in the art to make and use the invention as claimed and are provided in the context of mutual capacitance touch-surface devices. Variations using other types of touch-surfaces such as force or optical sensing touch-surfaces will be readily apparent to those skilled in the art. Accordingly, the claims appended hereto are not intended to be limited by the disclosed embodiments, but are to be accorded their widest scope consistent with the principles and features disclosed herein.
As previously noted, recent touch-surface input devices are capable of simultaneously detecting multiple objects as they approach and/or contact the touch-surface. For a hand-held multi-touch touch-surface device that may be put into a pocket, purse, or held against the head (e.g., portable music player, portable video player, personal digital assistant or mobile phone), detecting when the device is being clasped on the way into or out of the pocket, against the body, or against the head is very useful for: input rejection (ensuring that touch-surface input signals generated as a result of these actions are not mistaken for normal finger/stylus touches); operational mode transitions (e.g., dimming the device's backlight, putting the device to sleep and waking the device from a low-power state); and, for mobile telephones, answering calls (e.g., when the device is brought near, but not necessarily touching the head) and/or terminating calls (e.g., when the unit is placed into a pocket or purse).
Each sensing element (aka “pixel”) in a two dimensional array of sensing elements (i.e., a touch-surface) generates an output signal indicative of the electric field disturbance (for capacitance sensors), force (for pressure sensors) or optical coupling (for optical sensors) at the sensor element. The ensemble of pixel values represents a“proximity image.” As described herein, various embodiments of the invention address the ability to detect and discriminate between touch-surface signals (represented as a proximity image) resulting from, for example, the types of actions identified in the preceding paragraph.
Referring to FIG. 1 , multi-touch processing methodology 100 in accordance with one embodiment of the invention begins with the acquisition of proximity image data (block 105). Because the acquired data is usually a superposition of information (indicating an object close to or in contact with the touch-surface) fixed offsets (due to circuitry baselines) and noise (e.g., radio frequency interference), an initial adjustment to acquired pixel data may be made to compensate for sensor element baseline activity. For example, on multi-touch device initialization and/or when being brought out of a low-power mode (e.g., sleep), one or more images may be captured. By assuming these initial images include no surface contacts, they may be used to provide the sensor's baseline. Averaging over multiple sequential images (using, for example, infinite or finite impulse response filters) has been found to provide more accurate baseline values. These baseline values may be subtracted from each subsequently captured image to provide a proximity image for use in ongoing image processing steps. In another embodiment, baseline pixel values may be slowly adjusted over time to compensate for temperature or static charge. In addition, the initial baseline values may need to be adjusted if, in fact, touch-surface contacts were present at start-up. In yet another embodiment, a plurality of image samples may be acquired each at a different sensor element driving frequency. For each pixel in these images, the mean or median of subtracted samples (i.e., between the captured baseline and information images) may be combined to create an initial (typically signed) image in accordance with block 105. For noise that occasionally generates large outlier pixel values (“spiky” noise), other rank-order filters may be useful. As noted in FIG. 1 , proximity image data resulting from operations in accordance with block 105 is denoted [PROX].
Next, [PROX] image data feeds other processing blocks that may operate sequentially or in parallel with one another ( blocks 110, 115 and 120). It has been found that filtering or smoothing a proximity image (block 115) prior to segmentation (block 125) reduces the number of spurious peaks and thus helps reduce over segmentation. In one embodiment of block 115, each pixel value may be averaged with its nearest neighbor pixels in accordance with a discrete diffusion operation. If this approach is employed, it has been found beneficial to insert a “border” around the captured image so that there is a value with which to average the pixels at the edge of the captured image. For example, a one (1) pixel border may be added to the [PROX] image—where each “border” pixel is assigned a value corresponding to the image's “background” (e.g., zero). In another embodiment, both temporal (e.g., obtaining multiple images over a period of time) and spatial (e.g., averaging neighbor pixels) smoothing operations may be used. Multiple smoothing operations may be beneficial if the captured pixel data is particularly noisy. As noted in FIG. 1 , image data resulting from operations in accordance with block 115 is denoted [SMTH].
While [PROX] image pixel values are typically zero or positive in response to an object contacting the touch-surface (aka, a “grounded” object), background noise or objects close to but not touching the touch-surface (aka “ungrounded” objects) may produce an image some of whose pixel values are negative. Background noise may be static or vary with circuit temperature, touch-surface moisture, or other factors. Noisy, negative pixels can cause excessive jitter in centroid and other patch measurements (see discussion below regarding block [135]). To compensate for this, [PROX] image pixel values may be confined to a desired, typically positive, range (block 110). Subtracting the noise threshold helps reduce centroid jitter induced from pixels that wander around (above and below) the noise threshold in successive image frames. As noted in FIG. 1 , image data resulting from operations in accordance with block 110 is denoted [CNST]. In one embodiment, all pixel whose values are less than a background noise threshold are set to zero. In another embodiment, a noise-threshold is subtracted from each pixel value and the result is forced to be non-negative, as shown in Table 1.
TABLE 1 |
Illustrative Pixel Constraint Technique |
On a pixel-by-pixel basis: | |
If [PROX] < (Noise Threshold) |
[CNST] = (Background Value) |
Else |
[CNST] = [PROX] − (Noise Threshold) | ||
In one embodiment, the noise-threshold value is set to between 1 and 3 standard deviations of the noise measured at each pixel and the background-value is set to zero. One skilled in the art will recognize that other values are possible and that the precise choice of values depends, inter alia, on the type of sensor element used, the actual or expected level of pixel noise and the multi-touch device's operational environment. For example, the noise threshold may be set to a specified expected value on a per-pixel basis or a single value may be used for all pixels in an image. In addition, pixel noise values may be allowed to vary over time such that thermal and environmental effects on sensor element noise may be compensated for.
Touch-surface contacts typically show up as grouped collections of “active” pixel values, where each region of fleshy contact (e.g. finger, palm, cheek, ear or thigh) is represented by a roughly elliptical patch of pixels.
By analyzing an image's topography, image segmentation operations can identify distinct pixel patches that correspond to touch-surface contacts (block 125). In one embodiment, bottom-up, ridge-hiking algorithms may be used to group pixels that are part of the same watershed around each peak pixel—each watershed group or pixel patch corresponds to a touch-surface contact. In another embodiment, top-down search algorithms may be used to identify pixel patches surrounding each peak pixel, starting from the peak, searching outward and stopping at valleys. As part of the image segmentation process, one-dimensional patches may be culled from the identified patches in that they generally result from isolated noise spikes or failure of an entire row or column of sensor elements and/or associated circuitry. In addition, because large contacts such as palms and elongated thumbs may produce multiple peaks in a proximity image (due to noise or non-uniform signal saturation, for example), multiple peaks in the image can grow into multiple, split patches. To account for this phenomenon, multiple detected patches may be merged to produce a reduced number of patches for further processing. Heuristic or empirically determined rules may, for example, be applied to accomplish this. For example, two separately identified patches may be merged when the saddle point along their shared border is not “very deep”—e.g., when the saddle magnitude is more than 60% to 80% of the two patches' peak pixel values. As noted in FIG. 1 , identified patches resulting from operations in accordance with block 125 are denoted [P1, P2, . . . Pn].
Analysis shows that noise from pixels on the periphery of a patch, far from the center or peak pixel, can cause more jitter in calculated centroid (center-of-‘mass’) measurements than the same amount of noise from central pixels. This phenomenon applies to other statistically-fitted patch parameters such as major/minor radii and orientation as well. This jitter can be a particularly serious problem for the smooth tracking of hovering objects because hovering objects do not generally induce strong central pixels, leaving the peripheral pixels with even greater influence on the centroid measurement. However, completely leaving these peripheral pixels out of a patches' centroid calculations would discard potentially useful information about the position, size, and shape of the patch. It is further noted that performing patch parameterization on diffused images may reduce noise from peripheral pixels, but standard spatial filtering processes also cause swelling and distortion of patch shape, cause adjacent patches to spread into one another and other effects that bias centroid and ellipse radii measurements in particular. Thus, a technique is needed that minimizes the amount of noise from patch periphery pixels without strongly distorting patch shape and ensuing measurements.
In accordance with one embodiment of the invention, therefore, patch peripheral pixel values may be selectively reduced, down-scaled or dampened (block 130). Generally, patch centroid determination may be improved by selectively down-scaling patch peripheral pixels that are fairly weak and whose neighbors are very weak. More specifically, in one embodiment calibrated image pixel values (e.g., in [CNST]) whose corresponding smoothed value (e.g., in [SMTH]) falls within a specified range defined by a lower-limit and an upper-limit are reduced in proportion to where the smoothed value falls within that range. Lower and upper limits are chosen empirically so that only those pixel values that are relatively weak (compared to patch peak values and background noise) are manipulated. It has been found that: if the lower-limit is set too low, the patch will “bloom” from background pixels that happen to have positive noise; if the lower-limit is set too high, the patches' centroid position will have a spatially periodic bias toward sensor element centers (e.g., capacitive electrode plate centers); if the upper-limit is not sufficiently higher than the lower-limit, periphery dampening will not provide any significant centroid jitter reduction benefits; and if the upper-limit is too high, all patch pixels besides the patches' peak pixel will be affected, again biasing determination of the patches' centroid toward sensor element centers. In accordance with one embodiment of the invention, the lower-limit is set, on a pixel-by-pixel basis, to approximately twice the background noise standard deviation and the upper-limit is set to approximately four times the background noise standard deviation (with the background value typically being zero). In another embodiment, the lower-limit is set to a value indicative of the “average” or “expected” noise across all pixels in the proximity image. In some embodiments, the noise value may change dynamically to reflect changing operational conditions (see comments above). As noted in FIG. 1 , an image whose peripheral patch pixels have been dampened in accordance with block 130 is denoted [CNST']. In one embodiment, peripheral patch pixels are dampened as shown in Table 2.
TABLE 2 |
Illustrative Peripheral Patch Pixel Dampening |
For each pixel in a patch: | |
If [SMTH] < (Lower Limit) |
[CNST′] = (Background Value) |
Else If [SMTH] > (Upper Limit) |
[CNST′] = [CNST] |
Else | ||
|
||
Patch peripheral pixel dampening such as described above is equally applicable to touch-surfaces that provide one-dimensional proximity images. For example, projection scan touch-surfaces provide an output value (or signal) for each row and column of sensor elements in a touch-surface. In these types of touch-surfaces, a “patch” comprises a plurality of values, where each value represents a row or column measurement. The values at the ends of these patches (i.e., the peripheral values) may benefit from noise dampening as described here.
For certain touch-surface input devices such as a telephone, the ear and earlobe may contact the touch-surface sooner or more often than the cheek during calls. Unfortunately, earlobe patches can be very close in size to finger and thumb patches—but should, nevertheless, not cause spurious finger-button activations during a call. In accordance with one embodiment of the invention, a measurement of patch irregularity is defined that does not look for any specific ear (patch) shape, but rather indicates a general roughness, non-roundness or folds in the pixel patch (block 120). That is, if a patches' irregularity measure is above a specified threshold, the contact is identified as an irregular object (e.g., not a cheek, finger or palm), otherwise the patch is identified as not an irregular object (e.g., a cheek, finger or palm).
Referring to FIG. 2 , patch irregularity determination methodology 120 begins with the computation of a dispersion image (block 200). In general, the dispersion image (denoted [DISP] in FIG. 2 ) may be any high-pass filtered version of the initial proximity image [PROX]. In one embodiment, the [DISP] image is generated using a form of unsharp masking as follows:
[DISP]=[PROX]−[SMTH] EQ. 1
Next, the total energy for each patch [P1, P2, . . . Pn] is computed (block 205). In one embodiment, for example, a patches' total energy may be calculated by summing the square of each pixel value in the patch. This may be expressed mathematically as follows:
[DISP]=[PROX]−[SMTH] EQ. 1
Next, the total energy for each patch [P1, P2, . . . Pn] is computed (block 205). In one embodiment, for example, a patches' total energy may be calculated by summing the square of each pixel value in the patch. This may be expressed mathematically as follows:
As noted in
The total energy between adjacent pixels in a patch is then determined (block 210). To reduce the effect of energy spikes for pixel patches straddling an edge, the summations below should neglect (i.e., assume a value of zero) contributions from pixels whose neighboring pixels are at the image's border, see EQ. 3. For the same reason, the summations below should ignore contributions from pixels whose neighboring pixels are from a different patch.
The sum is divided by 4 because each pixel gets counted once for each direction in the proximity image (left, right, up and down). As noted in
As noted in
Finally, an irregularity measure for each patch is calculated (block 220). In one embodiment, the irregularity measure is defined as the ratio of a patches' spatial energy minus its peak energy to the patches' total energy:
As noted in
In another embodiment, the irregularity measure may be based on the proximity image as a whole. That is, the entirety of the dispersion image (i.e., all pixels) may be treated as a single “patch” for purposes of generating an irregularity measure value. One benefit to this approach is that abnormal touch-surface surface conditions may be detected, and responded to, prior to segmentation operations in accordance with block 125 (see FIG. 1 ). Illustrative abnormal touch-surface surface conditions include, but are not limited to, liquid (e.g., water or sweat) on the touch-surface or multiple irregular objects in close proximity to or in contact with the touch-surface (e.g., coins and/or keys). When these conditions are detected, it may be beneficial to acquire new sensor element baseline values. In addition, if multiple touch-surface sensor sampling frequencies are employed an irregularity measure may be computed at each of the frequencies. If one or more of the computed irregularity measure values is greater than a specified threshold as discussed above, the sampling frequencies associated with the above-threshold values may be deemed to be affected by an excessive amount of noise and ignored (e.g., radio frequency noise). Periodic determination of frequency-dependent irregularity measures in this manner may be used to detect when such noise sources occur and when they disappear. For example, due to a touch-surface devices operating environment changes.
In general, an oddly shaped collection of pixels (i.e., a patch) can require a relatively large set of numbers to define its boundary and signal value at each pixel within the patch. To reduce the computational complexity of identifying, distinguishing and tracking touch events, however, it is advantageous to characterize patches identified in accordance with block 125 with as few numbers as practical. Because most patches from flesh contact tend to have an elliptical shape, one approach to patch parameterization is to fit an ellipse to each patch. One benefit of this approach is that an ellipse is completely described by a relatively small collection of numbers—its center coordinates, major and minor axis lengths, and major axis orientation.
Referring again to FIG. 1 , using this approach known centroid or center-of-mass computations may be used to parameterize each patch (block 135). In general, a patches' centroid may be determined using these techniques and the [CNST'] image (see block 130). In addition, the [CNST'] image may be used to generate patch covariance matrices whose Eigenvalues identify a patches' major and minor radii and whose Eigenvectors identify the patches' orientation. For contact discrimination operations (see discussion below regarding block 140), the following patch characteristics are also computed:
In another embodiment, patch signal density may be approximated by:
Prior art techniques to discriminate between objects that actually contact a touch-surface from those that are merely hovering above it have relied upon a patches' total signal parameter (see, for example, EQ. 6). This approach, however, is very dependent upon the size of the object being identified. That is, prior art techniques that threshold on a patches' total signal value generally only work well for objects of a single size. For example, a total patch signal threshold selected to identify a fingertip contact would trigger detection of a thumb or palm when those objects are far above the touch-surface. Such a situation leads to the misidentification of patches (e.g., identifying a patch actually caused by a palm as a thumb).
In contrast, a discrimination technique in accordance with one embodiment of the invention uses a patches' signal density parameter (see, for example, EQS. 7 and 8). It has been found that this approach provides a robust means to distinguish objects that contact the touch-surface from those that are held or hovering above the surface—regardless of the object's size.
If the patch signal density parameter is normalized such that a firm fingertip contacting the touch-surface produces a peak value of 1, then a lightly brushing contact is indicated by a patch density value of slightly greater than 0.5 (e.g., half the normalized value) while a hovering object would be indicated by a patch density value equal to or slightly less than 0.5. It will be recognized that what constitutes “slightly greater” or “slightly less” is dependent upon factors such as the type of sensor elements used in the touch-surface and the associated sensor element drive and measure circuitry. Accordingly, while the precise determination of a threshold value based on patch signal density will require some experimentation, it would be will within the purview of an artisan of ordinary skill with benefit of this disclosure.
It has also been determined that fingernail touches also produce patch signal density values generally greater than approximately 0.5. This is because the non-conductive fingernail holds the conductive finger flesh more than approximately 1 millimeter above the touch-surface. Accordingly, a threshold operation based on patch signal density is also a reliable means for discriminating between fleshy fingertip touches and back-of-fingernail touches. This same technique has also been found to reliably determine whether large objects (e.g., cheeks and palms) are hovering or actually in contact with the touch-surface.
With patch parameterization complete, the various types of touch-surface contacts may be discriminated (block 140). Using the parameters identified above, it is possible to robustly and reliably distinguish large objects (e.g., cheeks and palms) form other objects (e.g., fingers and thumbs), irregular objects (e.g., ears) from regular objects (e.g., fingers, thumbs, cheeks and palms) and finger-clasp actions (e.g., when a user claps a multi-touch touch-surface device to put it into or withdraw it from a pocket). Identification of and discrimination between these types of touch-surface inputs permits an associated device to be controlled in a more robust manner. For example, in one embodiment detection of a large object may be used to transition the device from one operational state (e.g., off) to another (e.g., on). In another embodiment, input identified as the result of a large or irregular object, which might normally cause a state transition, may be safely ignored if in one or more specified states. For example, if a touch-surface telephone is already in an “on” or “active” state, identification of a large or irregular object may be ignored.
As previously noted, it can be advantageous to distinguish large objects (e.g., cheeks and palms) from small objects (e.g., fingertips), regardless of whether the objects are hovering a few millimeters above the touch-surface or are pressed firmly against the surface. It has been found that a contact's minor radius measure provides a robust discriminative measure to accomplish this. If a patches' minor radius exceeds a specified threshold, the contact can reliably be classified as a cheek—as opposed to a finger or thumb, for example. This same measurement can also detect a nearby leg (e.g., thigh) through a few millimeters of fabric (e.g. when a device is inserted in the pocket with its touch-surface facing the body). This measurement has been found to be so robust that if other patches appear on the surface with smaller minor radii (e.g. from an earlobe), they may be safely ignored. Referring to FIG. 3 , illustrative empirical data is shown that illustrates the distinction between cheek contacts 300 and other contacts 305 (e.g., fingertips and thumbs) based on patch minor radii. While the exact values for patch contacts may vary from sensor to sensor and population to population, it is clear from FIG. 3 that threshold 310 may be made anywhere between approximately 11 millimeters and approximately 15 millimeters. (In this and the following data plots, patch signal density values are normalized to 1 for a fully contacting fingertip.) While threshold 310 is described by a constant value (i.e., dependent only upon patch minor radius), this is not necessary. For example, threshold 310 may be described by a linear or non-linear relationship between multiple parameters such patch minor-radius and patch signal density (see discussion below regarding FIG. 4 ).
A similar size testing may be performed using a patches' major or geometric mean radius (i.e.,
the minor-radius discrimination described here has been found to be superior because it is better able to discriminate between thumbs or flattened fingers. (Flattened fingers may produce major radii as large as a cheek major radius, but their minor radii are typically no larger than a normal fingertip touch.)
It will be recognized that distinguishing a palm contact from fingertip or thumb contacts can be especially difficult because the patch radii resulting from a palm contact for people with small hands may approach the patch radii caused by thumb or fingertip contacts for people with large hands. These types of contacts may also be distinguished in accordance with the invention using the patch minor radius parameter. Referring to FIG. 4 , illustrative empirical data is shown that illustrates the distinction between palm contacts 400 and other contacts 405 (e.g., fingertips and thumbs) based on patch minor radii. It has been found that patch signal density values tend to be low for hovering contacts of any size, and saturate at a level independent of object size as the object presses firmly onto the touch-surface. Thus, the palm versus other object decision threshold 410 may be reduced for contacts with lower signal density because hovering or lightly touching fingers produce lower minor radii than firmly touching fingers, whereas palms tend to produce large minor radii even when hovering. Accordingly, decision threshold 410 may be represented by a straight curve with a small positive slope. While for patch contacts will vary as noted above, it is clear from FIG. 4 that threshold 410 may be made to distinguish palm contacts from other contacts. Using this approach, there is virtually no risk that a hovering palm (a contact that typically produces a patch signal density value similar to that of a touching finger) will mistakenly be interpreted as a cursor move or button activation (e.g., a “click” event).
Ear and earlobe contacts can generate patches that are roughly the same size as those generated by fingers and thumbs. It has been found, however, that the creases, ridges, and generally rough topography of the ear do produce proximity images unique from fingers and thumbs, at least if the imaging sensor (i.e., touch-surface) covers a significant portion of the ear (i.e. not just the fleshy lobule). The irregularity measure described above is one way to characterize contact roughness (see EQ. 5). This permits a robust means to discriminate between contacts due to ears and earlobes from contacts due to fingers, thumbs, cheeks, thighs and palms. It has been found that the defined irregularity measure tends to give values between 1.0 to 2.0 for ear and earlobe contacts while regular (e.g., smooth) contacts attributable to fingers, thumbs, palms and cheeks give values less than about 1.0. Referring to FIG. 5 , illustrative empirical data is shown that illustrates the distinction between ear contacts 500 and other contacts 505 (e.g., fingertips, thumbs and cheeks) based on the above defined irregularity measure. In one embodiment, threshold 510 comprises a linear step-like or spline structure with a first level at an irregularity measure of between approximately 1.0 to 1.2 and a second level at approximately between 1.1 and 1.2. In another embodiment, a single linear function having a positive slope may be used. In yet another embodiment, higher level functions may be used to segregate the various contact types. As noted above, while the exact values for patch contacts may vary from those shown in FIG. 5 , it is clear that most rough object contacts may be distinguished from most smooth or regular object contacts using the defined irregularity measure—where the exact nature or form of a decision threshold (e.g., threshold 510) is dependent upon the precise implementation, operational goals and capabilities of the target multi-touch device.
In one embodiment, successive proximity images (aka “frames”) are used to track objects as they move across a touch-surface. For example, as an object is moved across a touch-surface, its associated patch(es) may be correlated through overlap calculations. That is, patches identified in successive images that overlap in a specified number of pixels (or fraction of patch pixels) may be interpreted to be caused by the same object. In such an embodiments, the maximum patch minor radius over the life of the tracked contact may be compared to the thresholds discussed above (e.g., thresholds 310 in FIG. 3, 410 in FIGS. 4 and 510 in FIG. 5 ). This approach ensures that a, for example, palm contact does not lose its palm identity should its minor radius temporarily fall below the decision threshold (e.g., 410). It is further noted that if a decision threshold is not a constant value (e.g., 310) but rather some curve (e.g., 410 and 510), it may be advantageous to apply a density-correction to the instantaneous minor radius prior to the maximum minor radius accumulation operation described here.
When taking a multi-touch device in and out of a pocket, or otherwise generally handling it, users should have the freedom to clasp their hand around it without producing spurious input. Such finger-clasps can be detected via any one of the following criteria:
-
- Identification (via
block 125 inFIG. 1 ) of five, six or more distinct surface contacts. (For a touch-surface the size of a deck of cards, this many fingertips won't normally fit on the surface, but since the phalange joints of each flattened finger may get segmented into more than one contact patch, two or three flattened fingers may generate five or more contact patches.) - Two, three or more contact patches are identified and the major radius of at least two exceed approximately 15 millimeters to 18 millimeters. Since cheeks and other large body parts normally produce just one patch with large major radius, the requirement for two or three large patches prevents this test from triggering on a cheek, leg or chest. Also, the requirement for multiple large major radii prevents this test from triggering on a couple fingertips accompanied by a long thumb laid flat against the surface.
- Identification (via
In another embodiment of the invention, multi-touch processing methodology may include far-field processing. As used herein, far-field processing refers to the detection and processing associated with bodies (e.g., fingers, palms, cheeks, ears, thighs, . . . ) that are close to (e.g., less than one millimeter to more than a centimeter) but not in contact with the touch-surface. The ability to detect far-field objects may be beneficial in touch-surface devices that, during normal use, are brought into close proximity to a user. One example of such a device is a telephone that includes touch-surface for user input (e.g., dialing).
Referring to FIG. 6 , in one embodiment initial far-field processing may be performed after proximity image data is acquired. That is, after operations in accordance with block 105 in FIG. 1 . If the far-field measurement is designed to remain negative in the absence of any object near the touch-surface, and only become positive in the presence of a large object, a first step subtracts a small noise factor from the initially acquired proximity image to create a negative-background far-field image (block 600):
Negative Far-Field Image=[PROX]−(Noise Factor) EQ. 9
In one embodiment, the noise factor may be set to between approximately 1 and 2 standard deviations of the average noise measured or expected over the entire image. This will cause most pixels in the resulting negative far-field image to be slightly negative rather than neutral in the absence of any touch-surface contact. As noted inFIG. 6 , the negative far-field image resulting from operations in accordance with block 600 is denoted [NFAR].
Negative Far-Field Image=[PROX]−(Noise Factor) EQ. 9
In one embodiment, the noise factor may be set to between approximately 1 and 2 standard deviations of the average noise measured or expected over the entire image. This will cause most pixels in the resulting negative far-field image to be slightly negative rather than neutral in the absence of any touch-surface contact. As noted in
Next, each pixel in the [NFAR] image is saturated to the highest level expected from an object hovering a few millimeters from the touch-surface (block 605). In one embodiment, the resulting far-field saturation image (denoted [SFAR] in FIG. 6 ) is generated as shown in Table 3.
TABLE 3 |
Illustrative Far-Field Saturation Operations |
For each pixel in the initial far-field image: | |
If [NFAR] > (Far-Field Saturation Limit) |
[SFAR] = (Far-Field Saturation Limit) |
Else | ||
[SFAR] = [NFAR] | ||
Since a goal of far-field operations is to be sensitive to large numbers of pixels only slightly activated (e.g., having small positive values), without being overwhelmed by a few strongly active pixels (e.g., having large positive values), the saturation limit value should be less than the peak pixel value from fingers or thumbs hovering within approximately 1 to 2 millimeters of the touch-surface, but not so low as to cause the resulting [SFAR] image to lose to much information content. While the precise far-field saturation limit value will vary from implementation to implementation (due to differences in sensor element technology and associated circuitry), it has been determined empirically that a suitable value will generally lie between +3 standard deviations and +6 standard deviations of noise associated with the initial far-field image. (Again, this noise may be on a per-pixel, or whole image basis.)
If the initial proximity image [PROX] contains a significant amount of noise, it may be beneficial to filter the [SFAR] image (block 610). In one embodiment, a finite impulse response filter technique may be used wherein two or more consecutive [SFAR] images are averaged together. In another embodiment, an infinite impulse response filter technique may be used to generate a smoothed image. It will be recognized that an infinite impulse response filter generates a weighted running average (or auto-regressive) image. In one embodiment, for example, an infinite impulse response filter combines the current far-field saturated image (e.g., [SFAR]new) with the immediately prior far-field saturated image (e.g., [SFAR]prior) in a one-third to two-thirds ratio. As noted in FIG. 6 , a filtered far-field saturated image generated in accordance with block 610 is denoted [FARF].
Following image segmentation operations in accordance with block 125 (see FIG. 1 ), a weighted average of non-linearly scaled background pixel values may be used to generate a scalar far-field indicator value (FAR-FIELD) in accordance with the invention as follows:
where the ENeg( ) function non-linearly amplifies pixel values below a threshold (e.g., zero) and [LOC] represents a pixel weighting mechanism. As indicated in EQ. 10, only proximity image background pixels contribute to the computed FAR-FIELD value. That is, pixels identified as belonging to a patch during image segmentation operations are excluded during far-field measurement operations.
In one embodiment, the ENeg( ) function disproportionately emphasizes the contributions from background pixels as follows:
where B represents a far-field saturation limit value. Empirically determined, B is chosen to permit a small number of negative pixels to cancel out a finger or thumb-sized patch of positive pixels. In this way, only a nearly full coverage cheek-sized patch of positive pixels, plus a small remainder of neutral/background pixels, can produce a strongly positive far-field measurement.
While not necessary, disproportionately emphasizing the contributions from background pixels in accordance with EQs. 10 and 11 permits the FAR-FIELD measurement to be more selective for bodies large enough to positively affect most of a touch-surface's pixel (e.g., cheeks and legs), while not being overly influenced by medium-sized objects (e.g., hovering thumbs). For example, if a hovering thumb causes half of a touch-surface's sensor elements to have a slightly above-background pixel value, disproportionately emphasizing the half that remain below background will keep the measured FAR-FIELD value below zero—indicating no large object is “near” the touch-surface (e.g., within 1 to 3 centimeters). In another embodiment, background pixels may be linearly combined (e.g., summed).
As noted above, [LOC] represents a pixel weighting mechanism. In general, there is one value in [LOC] for each pixel present in the touch-surface. If it is desired to consider all touch-surface pixels equally, each value in the [LOC] image may be set to 1.0 (or some similar constant value). For hand-held form-factors selectivity for large bodies may be improved, however, by lowering the weights near the bottom and side edges (for example, to values between 0.1 and 0.9). Doing this can lessen false-positive contributions from a hand whose fingers wrap around the device during (clasping) operations. In mobile phone form-factors, to retain sensitivity to ear and cheek far-fields, the weights along the top edge (where thumbs and fingers are less likely to hover or wrap) may be kept at full strength.
Returning now to FIG. 1 at block 140, when far-field measurements are taken into account during contact discrimination operations, a FAR-FIELD value greater than a specified threshold (e.g., zero) indicates a large “near by” object has been detected. As previously noted, this information may be used to transition the touch-surface device into a specified mode (e.g., on, off or low power). In addition, far-field measurements may be combined with other measurements (e.g., the irregularity measure) to provide improved ear detection. For example, when a touch-surface is partly against an ear and also hovering a centimeter or two from a cheek, a weak ear pixel patch may be segmented in accordance with block 125 at the top of the screen. Meanwhile, the middle and bottom of the touch-surface would only be affected by the cheek's far-field. Even if the FAR-FIELD measurement as taken outside the ear patch is not strong enough to exceed the specified far-field threshold on its own, the FAR-FIELD value can be added or otherwise combined with (weak) patch density and irregularity measure indicators such that the sum or combination surpasses an ear detection threshold.
In addition, one or more proximity sensors may be positioned above the touch-surface's top edge or around, for example, a telephone's receiver opening. Illustrative proximity sensors of this type include, but are not limited to, active infrared-reflectance sensors and capacitance-sensitive electrode strips. In a mobile telephone form-factor, when the device is held such that the receiver is centered on the ear canal, ear ridges may trigger the proximity sensor. Meanwhile the earlobe may cause a small pixel patch in the top portion of the touch-surface. Discrimination operations in accordance with block 140 could decide that when a pixel patch at the top of the touch-surface is accompanied by any significant receiver proximity trigger, the pixel patch must be an ear, not a finger. In another embodiment, the same conditions but with a significant FAR-FIELD value for the lower portion of the touch-surface (indicating a hovering cheek) may be used to trigger detection of an ear at the top of the touch-surface. Generally speaking, one or more of signal density (see EQs. 7 and 8), patch irregularity (see EQ. 5), FAR-FIELD measurement (see EQ. 10) and proximity sensor input may be combined (e.g., a weighted average) so that ear detection can trigger when multiple indicators are weakly active, or just one indicator is strongly active. Finally, it is noted that contact discrimination parameters such as a patches' centroid, minor axis radius, patch irregularity (EQ. 5), patch signal density (EQs. 7 and 8), far-field (EQ. 10) and proximity sensor input (if available) may be (low-pass) filtered to help counteract their often sporadic nature. This may be particularly beneficial if the filters employ adaptive time constants that rise quickly in response to rising input values, but decay more slowing when input values drop and/or are missing.
Referring to FIG. 7 , a touch-surface device 700 of the type described herein is shown in block diagram form. As used herein, a touch-surface device is any device that receives user input from a multi-touch capable touch-surface component (i.e., an input unit that provides user input in the form of a proximity image). Illustrative touch-surface devices include, but are not limited, to tablet computer system, notebook computer systems, portable music and video display devices, personal digital assistants and mobile telephones.
As illustrated, touch-surface element 705 includes sensor elements and necessary drive and signal acquisition and detection circuitry. Memory 710 may be used to retain acquired proximity image information (e.g., [PROX] image data) and by processor 715 for computed image information (e.g., patch characterization parameters). Processor 715 represents a computational unit or programmable control device that is capable of using the information generated by touch-surface element 705 to determine various metrics in accordance with FIG. 1 . In addition, external component 720 represents an entity that uses the generated information. In the illustrative embodiment, external component 720 may obtain information from processor 715 or directly from memory 710. For example, processor 715 could maintain a data structure in memory 710 to retain indication of, for example, large body contact status, large body far-field status, irregular object indication status, proximity sensor status (if utilized), flat finger clasp status and normal finger touch status. In one embodiment, each status may be indicated by a single Boolean value (i.e., a flag).
Various changes in the materials, components, circuit elements, as well as in the details of the illustrated operational methods are possible without departing from the scope of the following claims. It will be recognized, for example, that not all steps identified in FIG. 1 need be performed while others may be combined and still others divided into more refined steps. By way of example, in one embodiment patch peripheral pixel noise is not suppressed (see block 130). In another embodiment, patch peripheral pixel noise suppression is employed but no patch irregularity measure is made (see block 120). In still another embodiment, both patch peripheral pixel noise suppression and patch irregularity measures are determined and used. For embodiments that do not employ peripheral patch pixel noise reduction techniques, patch parameterization operations in accordance with block 135 use the [CNST] image and not the [CNST'] image as discussed above (see Table 2). In addition, patch parameterization operations in accordance with block 135 do not need to rely on statistical ellipse fitting. They could instead sum patch perimeter pixels and compare the obtained value to all patch pixels or attempt polygonal fits. Further, calibration operations (see Tables 1 and 2) may be delayed until, or made part of, image segmentation operations (block 125). In addition, it may be beneficial for purposes of image segmentation to mark pixels that are at, or have been set to, the background level (e.g., during operations in accordance with block 110). It is also noted that because the criteria for identifying a finger clasp are orthogonal to large body contact detection (see discussion above in [00042]), flat finger clasps may be used as a distinct gesture commanding a particular operation like locking the screen, going to sleep, or terminating a telephone call.
With respect to illustrative touch-surface device 700, touch-surface element 705 may incorporate memory (e.g., 710) and/or processor (e.g., 715) functions. In addition, external component 720 may represent a hardware element (e.g., a host processor) or a software element (e.g., a driver utility).
Finally, acts in accordance with FIGS. 1, 2 and 6 may be performed by a programmable control device executing instructions organized into one or more program modules. A programmable control device may be a single computer processor, a special purpose processor (e.g., a digital signal processor, “DSP”), a plurality of processors coupled by a communications link or a custom designed state machine. Custom designed state machines may be embodied in a hardware device such as an integrated circuit including, but not limited to, application specific integrated circuits (“ASICs”) or field programmable gate array (“FPGAs”). Storage devices suitable for tangibly embodying program instructions include, but are not limited to: magnetic disks (fixed, floppy, and removable) and tape; optical media such as CD-ROMs and digital video disks (“DVDs”); and semiconductor memory devices such as Electrically Programmable Read-Only Memory (“EPROM”), Electrically Erasable Programmable Read-Only Memory (“EEPROM”), Programmable Gate Arrays and flash devices.
Claims (37)
1. A method to process a proximity image, comprising:
receiving a proximity image;
smoothing the proximity image to generate a filtered proximity image;
generating a dispersion image by subtracting the filtered proximity image from the proximity image;
determining an irregularity measure value for the dispersion image, the irregularity measure value indicating a degree of irregularity in the shape of the dispersion image; and
controlling an operation of a touch-surface device if the irregularity measure value is above a specified threshold.
2. The method of claim 1 , wherein the specified threshold comprises a constant function.
3. The method of claim 1 , wherein the specified threshold comprises a linear function.
4. The method of claim 1 , wherein the specified threshold comprises a non-linear function.
5. The method of claim 1 , wherein the specified threshold comprises two or more linear functions.
6. The method of claim 1 , wherein the operation of the touch-surface device comprises causing the touch-surface device to obtain baseline touch-surface sensor values.
7. The method of claim 1 , wherein the operation of the touch-surface device comprises:
utilizing multiple sampling frequencies for operation of the touch-surface device;
determining an irregularity measure value corresponding to each of the multiple sampling frequencies;
ignoring a touch-surface sensor sampling frequency during touch-surface device operations when a corresponding determined irregularity measure value is greater than the specified threshold.
8. The method of claim 1 , wherein the operation of a touch-surface device comprises causing the touch-surface device to enter into a low-power state.
9. The method of claim 1 , wherein the operation of a touch-surface device comprises causing the touch-surface device to exit a low-power state.
10. The method of claim 1 , wherein the touch-surface device comprises one of the devices selected from the group of tablet computer system, notebook computer system, portable music player, portable video player, personal digital assistant and mobile telephone.
11. A non-transitory program storage device, readable by a programmable control device, comprising instructions stored thereon for causing the programmable control device to perform the method of claim 1 .
12. A portable electronic device, comprising:
a touch input touch-surface component of the touch-surface device;
means for receiving a proximity image from the touch input touch-surface component; and
processing means for performing the acts of smoothing, generating, determining and controlling in accordance with claim 1 .
13. The device in accordance with claim 12 , wherein the processing means is operative for:
determining a far field image form the proximity image; and
controlling operation of the mobile telephone based on both the determined irregularity measure value and the far field image.
14. The device in accordance with claim 12 , wherein the portable electronic device comprises is selected from the group consisting of a tablet computer system, a hand-held computer system, a portable music player system, a portable video player system and a mobile telephone.
15. A method to identify an irregular input source to a touch-surface device, comprising:
obtaining a proximity image;
generating a dispersion image from the proximity image;
segmenting the proximity image to identify a plurality of patches;
determining an irregularity measure value for each of the plurality of patches based on the dispersion image, the irregularity measure value indicating a degree of irregularity in the space of each of the plurality of patches;
identifying one of the plurality of patches as being associated with an irregular object when the irregularity measure value of the one patch is above a specified threshold;
controlling an operation of the touch-surface device based on the identification of the irregular object;
smoothing the proximity image to generate a filtered proximity image;
wherein the act of generating the dispersion image comprises subtracting the filtered proximity image from the proximity image; and
wherein the act of determining an irregularity measure value for each of the plurality of patches comprises at least determining a total energy value for each of the plurality of patches.
16. The method of claim 15 , wherein the act of determining comprises: determining a spatial energy value for each of the plurality of patches; determining a peak energy value for each of the plurality of patches; and determining an irregularity measure value for each of the plurality of patches by dividing the difference between the spatial energy value and the peak energy value for each patch by the total energy value for each patch.
17. The method of claim 15 , wherein the specified threshold comprises a constant function.
18. The method of claim 15 , wherein the specified threshold comprises a linear function.
19. The method of claim 15 , wherein the specified threshold comprises a non-linear function.
20. The method of claim 15 , wherein the specified threshold comprises two or more linear functions.
21. The method of claim 15 , wherein the irregular patch comprises a patch formed by an ear.
22. The method of claim 15 , wherein the act of identifying comprises distinguishing a patch associated with an irregular object from a patch associated with a large object based on the irregularity measure value for the patch associated with the irregular object being greater than the specified threshold and the irregularity measure value for the patch associated with the large object being less than the specified threshold.
23. The method of claim 22 , wherein the large object comprises a cheek, a chest or a leg.
24. The method of claim 22 , wherein the irregular object comprises an ear, coin, or a key.
25. The method of claim 15 , wherein the act of identifying comprises distinguishing a patch associated with an irregular object from a patch associated with a small regular object based on the irregularity measure value for the patch associated with the irregular object being greater than the specified threshold and the irregularity measure value for the patch associated with the small regular object being less than the specified threshold.
26. The method of claim 25 , wherein the small regular object comprises a finger or a thumb.
27. The method of claim 15 , wherein the plurality of patches have patch pixel values and the method further comprising:
compensating the proximity image for background noise;
filtering the proximity image; and
reducing or down-scalling peripheral patch pixel values for one or more of the plurality of patches based on the compensated proximity image and the filtered proximity image.
28. A non-transitory program storage device, readable by a programmable control device, comprising instructions stored thereon for causing the programmable control device to perform the method of claim 15 .
29. A portable electronic device, comprising.
a touch input touch-surface component;
means for receiving the proximity image from the touch input touch-surface component; and
processing means for performing the acts of generating, segmenting, determining, identifying and controlling in accordance with claim 15 .
30. The device in accordance with claim 29 , wherein the portable electronic device comprises a tablet computer system.
31. The device in accordance with claim 29 , wherein the portable electronic device comprises a hand-held computer system.
32. The device in accordance with claim 29 , wherein the portable electronic device comprises a mobile telephone.
33. The device in accordance with claim 29 , wherein the portable electronic device comprises a portable music player system.
34. The device in accordance with claim 29 , wherein the portable electronic device comprises a portable video player system.
35. The method of claim 1 wherein the irregularity measure indicates a general roughness or non-roundness in the dispersion image.
36. A device having a touch screen, the device operative to process a proximity image, comprising:
means for receiving a proximity image;
means for smoothing the proximity image to generate a filtered proximity image;
means for generating dispersion image from a difference between the proximity image and the filtered proximity image;
means for determining an irregularity measure value for the dispersion image, the irregularity measure value indicating a degree of irregularity in the shape of the dispersion image; and
means for controlling an operation of a touch-surface device if the irregularity measure value is above a specified threshold.
37. A method to process a proximity image to discriminate between ear and earlobe contacts as distinguished from other contacts in a mutual capacitance touch surface mobile telephone device, comprising:
obtaining a proximity image based on a disturbance in an electric field of a two dimensional array of sensing elements of the mutual capacitance touch surface mobile telephone device;
smoothing the proximity image to generate a filtered proximity image;
generating a dispersion image defined by the difference between the proximity image and the filtered proximity image;
segmenting the proximity image to identify a plurality of patches;
determining an irregularity measure value for each of the plurality of patches based on the identified patches and the dispersion image, wherein the irregularity measure value indicates a degree of irregularity in the shape of the dispersion image associated with the ear or earlobe positioned near or in contact with the mutual capacitance touch surface mobile telephone device;
identifying one of the plurality of patches as being associated with the ear or earlobe contact or near contact if the irregularity measure value of the one patch is above a specified threshold; and
controlling the operation of the mutual capacitance touch surface mobile telephone device based on the identification of the ear or earlobe contact.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/584,725 US9411468B2 (en) | 2007-01-03 | 2012-08-13 | Irregular input identification |
US15/231,683 US10025429B2 (en) | 2007-01-03 | 2016-08-08 | Irregular input identification |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/619,490 US8269727B2 (en) | 2007-01-03 | 2007-01-03 | Irregular input identification |
US13/584,725 US9411468B2 (en) | 2007-01-03 | 2012-08-13 | Irregular input identification |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/619,490 Continuation US8269727B2 (en) | 2007-01-03 | 2007-01-03 | Irregular input identification |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/231,683 Continuation US10025429B2 (en) | 2007-01-03 | 2016-08-08 | Irregular input identification |
Publications (2)
Publication Number | Publication Date |
---|---|
US20120306800A1 US20120306800A1 (en) | 2012-12-06 |
US9411468B2 true US9411468B2 (en) | 2016-08-09 |
Family
ID=39485077
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/619,490 Expired - Fee Related US8269727B2 (en) | 2007-01-03 | 2007-01-03 | Irregular input identification |
US13/584,725 Active 2029-03-08 US9411468B2 (en) | 2007-01-03 | 2012-08-13 | Irregular input identification |
US15/231,683 Active 2027-06-11 US10025429B2 (en) | 2007-01-03 | 2016-08-08 | Irregular input identification |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/619,490 Expired - Fee Related US8269727B2 (en) | 2007-01-03 | 2007-01-03 | Irregular input identification |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/231,683 Active 2027-06-11 US10025429B2 (en) | 2007-01-03 | 2016-08-08 | Irregular input identification |
Country Status (2)
Country | Link |
---|---|
US (3) | US8269727B2 (en) |
WO (1) | WO2008085458A2 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160349910A1 (en) * | 2007-01-03 | 2016-12-01 | Apple Inc. | Irregular input identification |
US9691019B1 (en) * | 2017-03-07 | 2017-06-27 | Google Inc. | Depth concatenation using a matrix computation unit |
US10699182B2 (en) | 2017-03-07 | 2020-06-30 | Google Llc | Depth concatenation using a matrix computation unit |
US11954062B2 (en) | 2019-03-15 | 2024-04-09 | Intel Corporation | Dynamic memory reconfiguration |
Families Citing this family (91)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6610917B2 (en) * | 1998-05-15 | 2003-08-26 | Lester F. Ludwig | Activity indication, external source, and processing loop provisions for driven vibrating-element environments |
WO2007127258A2 (en) * | 2006-04-27 | 2007-11-08 | Wms Gaming Inc. | Wagering game with multi-point gesture sensing device |
WO2008045464A2 (en) | 2006-10-10 | 2008-04-17 | Wms Gaming Inc. | Multi-player, multi-touch table for use in wagering game systems |
US8633915B2 (en) | 2007-10-04 | 2014-01-21 | Apple Inc. | Single-layer touch-sensitive display |
US20090174676A1 (en) | 2008-01-04 | 2009-07-09 | Apple Inc. | Motion component dominance factors for motion locking of touch sensor data |
US9372576B2 (en) | 2008-01-04 | 2016-06-21 | Apple Inc. | Image jaggedness filter for determining whether to perform baseline calculations |
US8241912B2 (en) * | 2008-06-26 | 2012-08-14 | Wms Gaming Inc. | Gaming machine having multi-touch sensing device |
KR101572071B1 (en) * | 2009-01-06 | 2015-11-26 | 삼성전자주식회사 | / apparatus and method for controlling on/off of liquid crystal display in a portable terminal |
US8922521B2 (en) | 2009-02-02 | 2014-12-30 | Apple Inc. | Switching circuitry for touch sensitive display |
US8593410B2 (en) | 2009-04-10 | 2013-11-26 | Apple Inc. | Touch sensor panel design |
WO2010145031A1 (en) * | 2009-06-19 | 2010-12-23 | Research In Motion Limited | Portable electronic device with face touch detection |
US8957874B2 (en) | 2009-06-29 | 2015-02-17 | Apple Inc. | Touch sensor panel design |
WO2011048840A1 (en) * | 2009-10-19 | 2011-04-28 | シャープ株式会社 | Input motion analysis method and information processing device |
US8698015B2 (en) * | 2009-12-18 | 2014-04-15 | Intel Corporation | Compensating for multi-touch signal bias drift in touch panels |
US8730309B2 (en) | 2010-02-23 | 2014-05-20 | Microsoft Corporation | Projectors and depth cameras for deviceless augmented reality and interaction |
US8339286B2 (en) | 2010-03-31 | 2012-12-25 | 3M Innovative Properties Company | Baseline update procedure for touch sensitive device |
US8773146B1 (en) | 2010-04-16 | 2014-07-08 | Cypress Semiconductor Corporation | Waterproof scanning of a capacitive sense array |
WO2012027003A1 (en) | 2010-08-23 | 2012-03-01 | Cypress Semiconductor Corporation | Capacitance scanning proximity detection |
US8766936B2 (en) | 2011-03-25 | 2014-07-01 | Honeywell International Inc. | Touch screen and method for providing stable touches |
US8975903B2 (en) | 2011-06-09 | 2015-03-10 | Ford Global Technologies, Llc | Proximity switch having learned sensitivity and method therefor |
US8928336B2 (en) | 2011-06-09 | 2015-01-06 | Ford Global Technologies, Llc | Proximity switch having sensitivity control and method therefor |
US8959459B2 (en) | 2011-06-15 | 2015-02-17 | Wms Gaming Inc. | Gesture sensing enhancement system for a wagering game |
US10004286B2 (en) | 2011-08-08 | 2018-06-26 | Ford Global Technologies, Llc | Glove having conductive ink and method of interacting with proximity sensor |
US20130057515A1 (en) * | 2011-09-07 | 2013-03-07 | Microsoft Corporation | Depth camera as a touch sensor |
US9143126B2 (en) | 2011-09-22 | 2015-09-22 | Ford Global Technologies, Llc | Proximity switch having lockout control for controlling movable panel |
US8629849B2 (en) | 2011-09-23 | 2014-01-14 | Ebay Inc. | Spurious input detection system |
US10112556B2 (en) | 2011-11-03 | 2018-10-30 | Ford Global Technologies, Llc | Proximity switch having wrong touch adaptive learning and method |
US8994228B2 (en) | 2011-11-03 | 2015-03-31 | Ford Global Technologies, Llc | Proximity switch having wrong touch feedback |
US8878438B2 (en) | 2011-11-04 | 2014-11-04 | Ford Global Technologies, Llc | Lamp and proximity switch assembly and method |
US9411472B2 (en) * | 2011-12-08 | 2016-08-09 | Atmel Corporation | Touch sensor with adaptive touch detection thresholding |
US9733707B2 (en) | 2012-03-22 | 2017-08-15 | Honeywell International Inc. | Touch screen display user interface and method for improving touch interface utility on the same employing a rules-based masking system |
US9197206B2 (en) | 2012-04-11 | 2015-11-24 | Ford Global Technologies, Llc | Proximity switch having differential contact surface |
US9287864B2 (en) | 2012-04-11 | 2016-03-15 | Ford Global Technologies, Llc | Proximity switch assembly and calibration method therefor |
US9065447B2 (en) | 2012-04-11 | 2015-06-23 | Ford Global Technologies, Llc | Proximity switch assembly and method having adaptive time delay |
US9219472B2 (en) | 2012-04-11 | 2015-12-22 | Ford Global Technologies, Llc | Proximity switch assembly and activation method using rate monitoring |
US9520875B2 (en) | 2012-04-11 | 2016-12-13 | Ford Global Technologies, Llc | Pliable proximity switch assembly and activation method |
US9568527B2 (en) | 2012-04-11 | 2017-02-14 | Ford Global Technologies, Llc | Proximity switch assembly and activation method having virtual button mode |
US9660644B2 (en) | 2012-04-11 | 2017-05-23 | Ford Global Technologies, Llc | Proximity switch assembly and activation method |
US8933708B2 (en) | 2012-04-11 | 2015-01-13 | Ford Global Technologies, Llc | Proximity switch assembly and activation method with exploration mode |
US9944237B2 (en) | 2012-04-11 | 2018-04-17 | Ford Global Technologies, Llc | Proximity switch assembly with signal drift rejection and method |
US9184745B2 (en) | 2012-04-11 | 2015-11-10 | Ford Global Technologies, Llc | Proximity switch assembly and method of sensing user input based on signal rate of change |
US9559688B2 (en) | 2012-04-11 | 2017-01-31 | Ford Global Technologies, Llc | Proximity switch assembly having pliable surface and depression |
US9831870B2 (en) | 2012-04-11 | 2017-11-28 | Ford Global Technologies, Llc | Proximity switch assembly and method of tuning same |
US9531379B2 (en) | 2012-04-11 | 2016-12-27 | Ford Global Technologies, Llc | Proximity switch assembly having groove between adjacent proximity sensors |
US9329723B2 (en) | 2012-04-16 | 2016-05-03 | Apple Inc. | Reconstruction of original touch image from differential touch image |
US9086732B2 (en) | 2012-05-03 | 2015-07-21 | Wms Gaming Inc. | Gesture fusion |
US9136840B2 (en) | 2012-05-17 | 2015-09-15 | Ford Global Technologies, Llc | Proximity switch assembly having dynamic tuned threshold |
US8981602B2 (en) | 2012-05-29 | 2015-03-17 | Ford Global Technologies, Llc | Proximity switch assembly having non-switch contact and method |
US9337832B2 (en) | 2012-06-06 | 2016-05-10 | Ford Global Technologies, Llc | Proximity switch and method of adjusting sensitivity therefor |
US9641172B2 (en) | 2012-06-27 | 2017-05-02 | Ford Global Technologies, Llc | Proximity switch assembly having varying size electrode fingers |
US9423871B2 (en) | 2012-08-07 | 2016-08-23 | Honeywell International Inc. | System and method for reducing the effects of inadvertent touch on a touch screen controller |
US8922340B2 (en) | 2012-09-11 | 2014-12-30 | Ford Global Technologies, Llc | Proximity switch based door latch release |
US8796575B2 (en) | 2012-10-31 | 2014-08-05 | Ford Global Technologies, Llc | Proximity switch assembly having ground layer |
KR101388699B1 (en) * | 2012-11-22 | 2014-04-24 | 삼성전기주식회사 | Method and apparatus for sensing touch input |
US9128580B2 (en) | 2012-12-07 | 2015-09-08 | Honeywell International Inc. | System and method for interacting with a touch screen interface utilizing an intelligent stencil mask |
US8577644B1 (en) | 2013-03-11 | 2013-11-05 | Cypress Semiconductor Corp. | Hard press rejection |
US9311204B2 (en) | 2013-03-13 | 2016-04-12 | Ford Global Technologies, Llc | Proximity interface development system having replicator and method |
US9298222B2 (en) * | 2013-03-15 | 2016-03-29 | Atmel Corporation | Touch sensor with plastic cover lens |
US9576422B2 (en) | 2013-04-18 | 2017-02-21 | Bally Gaming, Inc. | Systems, methods, and devices for operating wagering game machines with enhanced user interfaces |
US20140380198A1 (en) * | 2013-06-24 | 2014-12-25 | Xiaomi Inc. | Method, device, and terminal apparatus for processing session based on gesture |
US9886141B2 (en) | 2013-08-16 | 2018-02-06 | Apple Inc. | Mutual and self capacitance touch measurements in touch panel |
DE202013008906U1 (en) * | 2013-10-07 | 2013-10-16 | Siemens Enterprise Communications Gmbh & Co. Kg | Portable device |
LU92408B1 (en) * | 2014-03-21 | 2015-09-22 | Olivier Raulot | User gesture recognition |
US9690417B2 (en) * | 2014-05-21 | 2017-06-27 | Apple Inc. | Glove touch detection |
US10936120B2 (en) | 2014-05-22 | 2021-03-02 | Apple Inc. | Panel bootstraping architectures for in-cell self-capacitance |
US10289251B2 (en) | 2014-06-27 | 2019-05-14 | Apple Inc. | Reducing floating ground effects in pixelated self-capacitance touch screens |
US9880655B2 (en) | 2014-09-02 | 2018-01-30 | Apple Inc. | Method of disambiguating water from a finger touch on a touch sensor panel |
WO2016048269A1 (en) | 2014-09-22 | 2016-03-31 | Onamp Research Llc | Ungrounded user signal compensation for pixelated self-capacitance touch sensor panel |
US10038443B2 (en) | 2014-10-20 | 2018-07-31 | Ford Global Technologies, Llc | Directional proximity switch assembly |
CN112379792A (en) | 2014-10-27 | 2021-02-19 | 苹果公司 | Pixelization from capacitive water repellence |
US9552097B2 (en) | 2015-01-28 | 2017-01-24 | Qualcomm Incorporated | Techniques for discerning between intended and unintended gestures on wearable touch-sensitive fabric |
CN107209602B (en) | 2015-02-02 | 2020-05-26 | 苹果公司 | Flexible self-capacitance and mutual capacitance touch sensing system architecture |
US10488992B2 (en) | 2015-03-10 | 2019-11-26 | Apple Inc. | Multi-chip touch architecture for scalability |
US10437384B2 (en) * | 2015-03-13 | 2019-10-08 | Parade Technologies, Ltd. | Water detection and wipe detection algorithms for touchscreen proximity sensing |
US9654103B2 (en) | 2015-03-18 | 2017-05-16 | Ford Global Technologies, Llc | Proximity switch assembly having haptic feedback and method |
US9612685B2 (en) | 2015-04-09 | 2017-04-04 | Microsoft Technology Licensing, Llc | Force-sensitive touch sensor compensation |
US9548733B2 (en) | 2015-05-20 | 2017-01-17 | Ford Global Technologies, Llc | Proximity sensor assembly having interleaved electrode configuration |
US9898148B2 (en) * | 2015-05-27 | 2018-02-20 | Synaptics Incorporated | Capacitive stereoscopic image sensing |
CN105120089B (en) * | 2015-08-17 | 2020-01-03 | 惠州Tcl移动通信有限公司 | Method for mobile terminal to automatically answer call and mobile terminal |
TWI592845B (en) * | 2015-08-28 | 2017-07-21 | 晨星半導體股份有限公司 | Method and associated controller for adaptively adjusting touch-control threshold |
US10365773B2 (en) | 2015-09-30 | 2019-07-30 | Apple Inc. | Flexible scan plan using coarse mutual capacitance and fully-guarded measurements |
US9927917B2 (en) * | 2015-10-29 | 2018-03-27 | Microsoft Technology Licensing, Llc | Model-based touch event location adjustment |
AU2017208277B2 (en) | 2016-09-06 | 2018-12-20 | Apple Inc. | Back of cover touch sensors |
US10642418B2 (en) | 2017-04-20 | 2020-05-05 | Apple Inc. | Finger tracking in wet environment |
CN109767389B (en) * | 2019-01-15 | 2023-06-20 | 四川大学 | Self-adaptive weighted double-norm remote sensing image blind super-resolution reconstruction method based on local and non-local combined prior |
US10963095B1 (en) | 2019-09-27 | 2021-03-30 | Apple Inc. | Glove touch detection |
US11301099B1 (en) * | 2019-09-27 | 2022-04-12 | Apple Inc. | Methods and apparatus for finger detection and separation on a touch sensor panel using machine learning models |
JP2021128497A (en) * | 2020-02-13 | 2021-09-02 | キヤノン株式会社 | Image processing device, image processing method and program |
US11662867B1 (en) | 2020-05-30 | 2023-05-30 | Apple Inc. | Hover detection on a touch sensor panel |
US11775084B2 (en) * | 2021-04-20 | 2023-10-03 | Microsoft Technology Licensing, Llc | Stylus haptic component arming and power consumption |
US11327607B1 (en) * | 2021-08-10 | 2022-05-10 | Novatek Microelectronics Corp. | Touch detection circuit and touch detection method thereof |
Citations (223)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4056754A (en) | 1975-03-01 | 1977-11-01 | Electric Valve Company Limited | Vidicon having switchable electrode connections |
US4219847A (en) | 1978-03-01 | 1980-08-26 | Canadian Patents & Development Limited | Method and apparatus of determining the center of area or centroid of a geometrical area of unspecified shape lying in a larger x-y scan field |
US4290052A (en) | 1979-10-26 | 1981-09-15 | General Electric Company | Capacitive touch entry apparatus having high degree of personal safety |
US4293734A (en) | 1979-02-23 | 1981-10-06 | Peptek, Incorporated | Touch panel system and method |
US4340911A (en) | 1979-02-28 | 1982-07-20 | Fuji Photo Film Co., Ltd. | Image gradation processing method and apparatus for mammogram copying system |
US4444998A (en) | 1981-10-27 | 1984-04-24 | Spectra-Symbol Corporation | Touch controlled membrane for multi axis voltage selection |
GB2139762A (en) | 1983-04-08 | 1984-11-14 | Gavilan Computer Corp | An input device and a method of inputting data to a computer system |
US4526043A (en) | 1983-05-23 | 1985-07-02 | At&T Bell Laboratories | Conformable tactile sensor |
US4550221A (en) | 1983-10-07 | 1985-10-29 | Scott Mabusth | Touch sensitive control device |
US4570193A (en) | 1981-05-08 | 1986-02-11 | Canon Kabushiki Kaisha | Video signal processing device including emphasis and/or de-emphasis circuit |
US4598420A (en) | 1983-12-08 | 1986-07-01 | Mts Systems Corporation | Optical grid analyzer system for automatically determining strain in deformed sheet metal |
US4617563A (en) | 1982-12-28 | 1986-10-14 | Seiko Epson Kabushiki Kaisha | Liquid crystal display device |
US4618989A (en) | 1983-01-21 | 1986-10-21 | Michio Kawata, Director-General of Agency of Industrial Science and Technology | Method and system for detecting elliptical objects |
JPS6230888A (en) | 1985-08-02 | 1987-02-09 | Hitachi Ltd | Heat transmission pipe |
US4678869A (en) | 1985-10-25 | 1987-07-07 | Scriptel Corporation | Position responsive apparatus, system and method having electrographic application |
US4686332A (en) | 1986-06-26 | 1987-08-11 | International Business Machines Corporation | Combined finger touch and stylus detection system for use on the viewing surface of a visual display device |
US4736191A (en) | 1985-08-02 | 1988-04-05 | Karl E. Matzke | Touch activated control method and apparatus |
US4746770A (en) | 1987-02-17 | 1988-05-24 | Sensor Frame Incorporated | Method and apparatus for isolating and manipulating graphic objects on computer video monitor |
US4763356A (en) | 1986-12-11 | 1988-08-09 | AT&T Information Systems, Inc. American Telephone and Telegraph Company | Touch screen form entry system |
US4806709A (en) | 1987-05-26 | 1989-02-21 | Microtouch Systems, Inc. | Method of and apparatus for sensing the location, such as coordinates, of designated points on an electrically sensitive touch-screen surface |
US4850677A (en) | 1987-03-18 | 1989-07-25 | Nec Corporation | Liquid crystal display device having improved electrodes for a multi-tone image |
US4885639A (en) | 1987-06-09 | 1989-12-05 | Sony Corporation | Apparatus for eliminating a noise signal |
US4896363A (en) | 1987-05-28 | 1990-01-23 | Thumbscan, Inc. | Apparatus and method for matching image characteristics such as fingerprint minutiae |
US4899138A (en) | 1987-01-10 | 1990-02-06 | Pioneer Electronic Corporation | Touch panel control device with touch time and finger direction discrimination |
US4906983A (en) | 1985-08-16 | 1990-03-06 | British Aerospace Public Limited Company | Display data memory |
US4914624A (en) | 1988-05-06 | 1990-04-03 | Dunthorn David I | Virtual button for touch screen |
US4952932A (en) | 1987-04-30 | 1990-08-28 | Canon Kabushiki Kaisha | Information input apparatus |
US4954967A (en) | 1983-09-21 | 1990-09-04 | Canon Kabushiki Kaisha | Information processing apparatus |
US4968877A (en) | 1988-09-14 | 1990-11-06 | Sensor Frame Corporation | VideoHarp |
EP0397428A2 (en) | 1989-05-08 | 1990-11-14 | Canon Kabushiki Kaisha | Image processing apparatus |
US5012524A (en) | 1989-02-27 | 1991-04-30 | Motorola, Inc. | Automatic inspection method |
US5073950A (en) | 1989-04-13 | 1991-12-17 | Personnel Identification & Entry Access Control, Inc. | Finger profile identification system |
US5113041A (en) | 1990-12-28 | 1992-05-12 | At&T Bell Laboratories | Information processing |
US5150105A (en) | 1987-08-28 | 1992-09-22 | Thomson Csf | System for the display of color images on matrix display panels |
US5168531A (en) | 1991-06-27 | 1992-12-01 | Digital Equipment Corporation | Real-time recognition of pointing information from video |
US5189732A (en) | 1987-11-18 | 1993-02-23 | Hitachi, Ltd. | Touch panel input apparatus |
US5252951A (en) | 1989-04-28 | 1993-10-12 | International Business Machines Corporation | Graphical user interface with gesture recognition in a multiapplication environment |
US5267327A (en) | 1990-03-30 | 1993-11-30 | Sony Corporation | Apparatus and method for registering the handwriting of a user so it can be translated into block characters |
USRE34476E (en) | 1990-05-14 | 1993-12-14 | Norwood Donald D | Hybrid information management system for handwriting and text |
US5276794A (en) | 1990-09-25 | 1994-01-04 | Grid Systems Corporation | Pop-up keyboard system for entering handwritten data into computer generated forms |
US5293430A (en) | 1991-06-27 | 1994-03-08 | Xerox Corporation | Automatic image segmentation using local area maximum and minimum image signals |
US5305017A (en) | 1989-08-16 | 1994-04-19 | Gerpheide George E | Methods and apparatus for data input |
US5309374A (en) | 1992-08-03 | 1994-05-03 | Iowa State University Research Foundation, Inc. | Acoustic and video imaging system for quality determination of agricultural products |
JPH06161661A (en) | 1992-11-18 | 1994-06-10 | Digital:Kk | Touch pannel input device capable of plural simultaneous inputs and method thereof |
US5327161A (en) | 1989-08-09 | 1994-07-05 | Microtouch Systems, Inc. | System and method for emulating a mouse input device with a touchpad input device |
US5347295A (en) | 1990-10-31 | 1994-09-13 | Go Corporation | Control of a computer through a position-sensed stylus |
US5367453A (en) | 1993-08-02 | 1994-11-22 | Apple Computer, Inc. | Method and apparatus for correcting words |
US5376946A (en) | 1991-07-08 | 1994-12-27 | Mikan; Peter J. | Computer mouse simulator device |
US5386219A (en) | 1991-10-16 | 1995-01-31 | International Business Machines Corp. | Touch overlay for improved touch sensitivity |
JPH07110741A (en) | 1993-10-13 | 1995-04-25 | Digital:Kk | Method and device for detection of pointed coordinate |
US5414227A (en) | 1993-04-29 | 1995-05-09 | International Business Machines Corporation | Stylus tilt detection apparatus for communication with a remote digitizing display |
US5420936A (en) | 1992-10-16 | 1995-05-30 | International Business Machines Corporation | Method and apparatus for accessing touch screen desktop objects via fingerprint recognition |
US5422831A (en) | 1992-08-03 | 1995-06-06 | Iowa State University Research Foundation, Inc. | Acoustic and video imaging system for quality determination of pharmaceutical products |
US5432531A (en) | 1990-12-14 | 1995-07-11 | International Business Machines Corporation | Coordinate processor for a computer system having a pointing device |
US5446480A (en) | 1989-08-17 | 1995-08-29 | Sony Corporation | Input device for entering data for electronic equipments and so forth |
US5457476A (en) | 1992-05-27 | 1995-10-10 | Apple Computer, Inc. | Method for controlling a computerized organizer |
US5459463A (en) | 1990-05-25 | 1995-10-17 | Sextant Avionique | Device for locating an object situated close to a detection area and a transparent keyboard using said device |
US5461655A (en) * | 1992-06-19 | 1995-10-24 | Agfa-Gevaert | Method and apparatus for noise reduction |
US5463388A (en) | 1993-01-29 | 1995-10-31 | At&T Ipm Corp. | Computer mouse or keyboard input device utilizing capacitive sensors |
US5469194A (en) | 1994-05-13 | 1995-11-21 | Apple Computer, Inc. | Apparatus and method for providing different input device orientations of a computer system |
US5479528A (en) | 1991-12-13 | 1995-12-26 | At&T Corp. | Intelligent work surfaces |
US5483261A (en) | 1992-02-14 | 1996-01-09 | Itu Research, Inc. | Graphical input controller and method with rear screen image detection |
US5488204A (en) | 1992-06-08 | 1996-01-30 | Synaptics, Incorporated | Paintbrush stylus for capacitive touch sensor pad |
US5491495A (en) | 1990-11-13 | 1996-02-13 | Wang Laboratories, Inc. | User interface having simulated devices |
US5505072A (en) | 1994-11-15 | 1996-04-09 | Tekscan, Inc. | Scanning circuit for pressure responsive array |
US5513309A (en) | 1993-01-05 | 1996-04-30 | Apple Computer, Inc. | Graphic editor user interface for a pointer-based computer system |
WO1996015464A1 (en) | 1994-11-14 | 1996-05-23 | Moonstone Technology Limited | Capacitive touch detectors |
US5541372A (en) | 1992-06-15 | 1996-07-30 | U.S. Philips Corporation | Force activated touch screen measuring deformation of the front panel |
US5543590A (en) | 1992-06-08 | 1996-08-06 | Synaptics, Incorporated | Object position detector with edge motion feature |
US5543591A (en) | 1992-06-08 | 1996-08-06 | Synaptics, Incorporated | Object position detector with edge motion feature and gesture recognition |
US5543588A (en) | 1992-06-08 | 1996-08-06 | Synaptics, Incorporated | Touch pad driven handheld computing device |
US5546475A (en) | 1994-04-29 | 1996-08-13 | International Business Machines Corporation | Produce recognition system |
US5546334A (en) | 1993-03-29 | 1996-08-13 | Acer Incorporated | Notebook computer system with a separable trackball |
US5548667A (en) | 1991-05-24 | 1996-08-20 | Sony Corporation | Image processing system and method thereof in which three dimensional shape is reproduced from two dimensional image data |
US5552787A (en) | 1995-10-10 | 1996-09-03 | The United States Of America As Represented By The Secretary Of The Navy | Measurement of topography using polarimetric synthetic aperture radar (SAR) |
US5570113A (en) | 1994-06-29 | 1996-10-29 | International Business Machines Corporation | Computer based pen system and method for automatically cancelling unwanted gestures and preventing anomalous signals as inputs to such system |
US5572717A (en) | 1994-04-06 | 1996-11-05 | Altera Corporation | Method and apparatus for assigning and analyzing timing specifications in a computer aided engineering program |
US5583946A (en) | 1993-09-30 | 1996-12-10 | Apple Computer, Inc. | Method and apparatus for recognizing gestures on a computer system |
US5583543A (en) | 1992-11-05 | 1996-12-10 | Sharp Kabushiki Kaisha | Pen input processing apparatus |
US5585823A (en) | 1994-12-30 | 1996-12-17 | Apple Computer, Inc. | Multi-state one-button computer pointing device |
US5588098A (en) | 1991-11-22 | 1996-12-24 | Apple Computer, Inc. | Method and apparatus for direct manipulation of 3-D objects on computer displays |
US5594806A (en) | 1994-06-20 | 1997-01-14 | Personnel Identification & Entry Access Control, Inc. | Knuckle profile indentity verification system |
US5600735A (en) | 1994-05-10 | 1997-02-04 | Motorola, Inc. | Method of recognizing handwritten input |
US5603053A (en) | 1993-05-10 | 1997-02-11 | Apple Computer, Inc. | System for entering data into an active application currently running in the foreground by selecting an input icon in a palette representing input utility |
US5612719A (en) | 1992-12-03 | 1997-03-18 | Apple Computer, Inc. | Gesture sensitive buttons for graphical user interfaces |
US5627567A (en) | 1993-04-27 | 1997-05-06 | Hewlett-Packard Company | Method and apparatus for adaptive touch recognition in a touch sensitive user interface |
US5631976A (en) | 1994-04-29 | 1997-05-20 | International Business Machines Corporation | Object imaging system |
US5638093A (en) | 1993-12-07 | 1997-06-10 | Seiko Epson Corporation | Touch panel input device and control method thereof |
US5648642A (en) | 1992-06-08 | 1997-07-15 | Synaptics, Incorporated | Object position detector |
US5649070A (en) | 1995-02-17 | 1997-07-15 | International Business Machines Corporation | Learning system with prototype replacement |
US5666438A (en) | 1994-07-29 | 1997-09-09 | Apple Computer, Inc. | Method and apparatus for recognizing handwriting of different users of a pen-based computer system |
US5673066A (en) | 1992-04-21 | 1997-09-30 | Alps Electric Co., Ltd. | Coordinate input device |
US5717777A (en) | 1996-01-11 | 1998-02-10 | Dew Engineering And Development Limited | Longest line method and apparatus for fingerprint alignment |
US5729604A (en) | 1996-03-14 | 1998-03-17 | Northern Telecom Limited | Safety switch for communication device |
US5732230A (en) | 1995-05-19 | 1998-03-24 | Richo Company Ltd. | Computer user interface for manipulating image fragments using drag, drop and merge operations |
US5734751A (en) | 1994-07-22 | 1998-03-31 | Nec Corporation | Ellipse-like curve recognition by calculation of distances of points on the curve from FOCI |
US5734742A (en) | 1994-09-19 | 1998-03-31 | Nissan Motor Co., Ltd. | Inspection system and process |
US5748184A (en) | 1996-05-28 | 1998-05-05 | International Business Machines Corporation | Virtual pointing device for touchscreens |
US5751851A (en) | 1994-11-14 | 1998-05-12 | Motorola, Inc. | Method of splitting handwritten input |
US5757478A (en) | 1996-06-03 | 1998-05-26 | Ma; Chris Chen-Hsing | Remote position sensing apparatus and method |
US5764222A (en) | 1996-05-28 | 1998-06-09 | International Business Machines Corporation | Virtual pointing device for touchscreens |
US5764218A (en) | 1995-01-31 | 1998-06-09 | Apple Computer, Inc. | Method and apparatus for contacting a touch-sensitive cursor-controlling input device to generate button values |
US5768492A (en) | 1991-06-17 | 1998-06-16 | Schumer; Alfred L. | Digitizer interface |
US5771342A (en) | 1995-06-05 | 1998-06-23 | Saltire Software | Method and apparatus for dynamically displaying consistently dimensioned two-dimensional drawings |
US5790104A (en) | 1996-06-25 | 1998-08-04 | International Business Machines Corporation | Multiple, moveable, customizable virtual pointing devices |
US5793355A (en) | 1997-03-03 | 1998-08-11 | Compaq Computer Corporation | Portable computer with interchangeable pointing device modules |
US5796389A (en) | 1994-08-22 | 1998-08-18 | International Game Technology | Reduced noise touch screen apparatus and method |
US5797396A (en) | 1995-06-07 | 1998-08-25 | University Of Florida Research Foundation | Automated method for digital image quantitation |
US5798756A (en) | 1994-12-26 | 1998-08-25 | Sharp Kabushiki Kaisha | Coordinate detection device capable of obviating the need of forming closed loop in electrodes thereof |
US5805144A (en) | 1994-12-14 | 1998-09-08 | Dell Usa, L.P. | Mouse pointing device having integrated touchpad |
US5809166A (en) | 1994-11-09 | 1998-09-15 | Canon Kabushiki Kaisha | Character recognition system which cuts between touching characters |
US5808605A (en) | 1996-06-13 | 1998-09-15 | International Business Machines Corporation | Virtual pointing device for touchscreens |
US5812698A (en) | 1995-05-12 | 1998-09-22 | Synaptics, Inc. | Handwriting recognition system and method |
US5812118A (en) | 1996-06-25 | 1998-09-22 | International Business Machines Corporation | Method, apparatus, and memory for creating at least two virtual pointing devices |
US5825675A (en) | 1993-07-29 | 1998-10-20 | Xerox Corporation | Apparatus and configuration method for a small, hand-held computing device |
US5825352A (en) | 1996-01-04 | 1998-10-20 | Logitech, Inc. | Multiple fingers contact sensing method for emulating mouse buttons and mouse operations on a touch sensor pad |
US5831602A (en) | 1996-01-11 | 1998-11-03 | Canon Kabushiki Kaisha | Information processing apparatus, method and computer program product |
US5835079A (en) | 1996-06-13 | 1998-11-10 | International Business Machines Corporation | Virtual pointing device for touchscreens |
US5838302A (en) | 1995-02-24 | 1998-11-17 | Casio Computer Co., Ltd. | Data inputting devices for inputting typed and handwritten data in a mixed manner |
US5852669A (en) | 1994-04-06 | 1998-12-22 | Lucent Technologies Inc. | Automatic face and facial feature location detection for low bit rate model-assisted H.261 compatible coding of video |
US5852434A (en) | 1992-04-03 | 1998-12-22 | Sekendur; Oral F. | Absolute optical position determination |
US5856824A (en) | 1996-06-25 | 1999-01-05 | International Business Machines Corp. | Reshapable pointing device for touchscreens |
US5856822A (en) | 1995-10-27 | 1999-01-05 | 02 Micro, Inc. | Touch-pad digital computer pointing-device |
US5862256A (en) | 1996-06-14 | 1999-01-19 | International Business Machines Corporation | Distinguishing gestures from handwriting in a pen based computer by size discrimination |
US5870771A (en) | 1996-11-15 | 1999-02-09 | Oberg; Larry B. | Computerized system for selecting, adjusting, and previewing framing product combinations for artwork and other items to be framed |
US5870083A (en) | 1996-10-04 | 1999-02-09 | International Business Machines Corporation | Breakaway touchscreen pointing device |
US5872559A (en) | 1996-10-04 | 1999-02-16 | International Business Machines Corporation | Breakaway and re-grow touchscreen pointing device |
US5875311A (en) | 1994-03-18 | 1999-02-23 | International Business Machines Corporation | Computer system with touchpad support in operating system |
US5874948A (en) | 1996-05-28 | 1999-02-23 | International Business Machines Corporation | Virtual pointing device for touchscreens |
US5880411A (en) | 1992-06-08 | 1999-03-09 | Synaptics, Incorporated | Object position detector with edge motion feature and gesture recognition |
US5896126A (en) | 1996-08-29 | 1999-04-20 | International Business Machines Corporation | Selection device for touchscreen systems |
US5898422A (en) | 1996-05-31 | 1999-04-27 | International Business Machines Corporation | Method and system for recognition of pointers |
US5898434A (en) | 1991-05-15 | 1999-04-27 | Apple Computer, Inc. | User interface system having programmable user interface elements |
WO1999038149A1 (en) | 1998-01-26 | 1999-07-29 | Wayne Westerman | Method and apparatus for integrating manual input |
US5933134A (en) | 1996-06-25 | 1999-08-03 | International Business Machines Corporation | Touch screen virtual pointing device which goes into a translucent hibernation state when not in use |
US5936615A (en) | 1996-09-12 | 1999-08-10 | Digital Equipment Corporation | Image-based touchscreen |
US5940065A (en) | 1996-03-15 | 1999-08-17 | Elo Touchsystems, Inc. | Algorithmic compensation system and method therefor for a touch sensor panel |
US5943043A (en) | 1995-11-09 | 1999-08-24 | International Business Machines Corporation | Touch panel "double-touch" input method and detection apparatus |
US5956682A (en) | 1991-06-06 | 1999-09-21 | Lj Laboratories, Llc | Picture frame with associated audio messages and position sensitive or speech recognition device |
US5963190A (en) | 1994-09-26 | 1999-10-05 | Canon Kabushiki Kaisha | Driving method for display device and display apparatus |
US5974558A (en) * | 1994-09-02 | 1999-10-26 | Packard Bell Nec | Resume on pen contact |
US5977950A (en) | 1993-11-29 | 1999-11-02 | Motorola, Inc. | Manually controllable cursor in a virtual image |
US5982302A (en) | 1994-03-07 | 1999-11-09 | Ure; Michael J. | Touch-sensitive keyboard/mouse |
US5987162A (en) | 1996-03-27 | 1999-11-16 | Mitsubishi Denki Kabushiki Kaisha | Image processing method and apparatus for recognizing an arrangement of an object |
US6005963A (en) | 1996-10-23 | 1999-12-21 | International Business Machines Corporation | System and method for determining if a fingerprint image contains an image portion representing a partial fingerprint impression |
US6005959A (en) | 1995-02-17 | 1999-12-21 | International Business Machines Corporation | Produce size recognition system |
US6008800A (en) | 1992-09-18 | 1999-12-28 | Pryor; Timothy R. | Man machine interfaces for entering data into a computer |
US6016355A (en) | 1995-12-15 | 2000-01-18 | Veridicom, Inc. | Capacitive fingerprint acquisition sensor |
US6023313A (en) | 1996-08-21 | 2000-02-08 | Sony Corporation | Liquid crystal display panel manufacturing method |
US6029214A (en) | 1995-11-03 | 2000-02-22 | Apple Computer, Inc. | Input tablet system with user programmable absolute coordinate mode and relative coordinate mode segments |
US6037882A (en) | 1997-09-30 | 2000-03-14 | Levy; David H. | Method and apparatus for inputting data to an electronic system |
US6049329A (en) | 1996-06-04 | 2000-04-11 | International Business Machines Corporartion | Method of and system for facilitating user input into a small GUI window using a stylus |
US6067079A (en) | 1996-06-13 | 2000-05-23 | International Business Machines Corporation | Virtual pointing device for touchscreens |
US6075520A (en) | 1996-11-15 | 2000-06-13 | Rohm Co., Ltd. | Small current detector circuit and locator device using the same |
JP2000163031A (en) | 1998-11-25 | 2000-06-16 | Seiko Epson Corp | Portable information equipment and information storage medium |
GB2344894A (en) | 1998-12-17 | 2000-06-21 | Hewlett Packard Co | Palm pressure rejection for touchscreens |
US6088025A (en) | 1995-08-24 | 2000-07-11 | Matsushita Electric Industrial Co., Ltd. | Terminal device with built-in image sensor |
US6118435A (en) | 1997-04-10 | 2000-09-12 | Idec Izumi Corporation | Display unit with touch panel |
US6121960A (en) | 1996-08-28 | 2000-09-19 | Via, Inc. | Touch screen systems and methods |
US6128014A (en) | 1997-01-10 | 2000-10-03 | Tokyo University Of Agriculture And Technology | Human interactive type display system |
JP2000300543A (en) | 1999-04-21 | 2000-10-31 | Fuji Xerox Co Ltd | Detecting device, inputting device, pointing device, individual identification device and recording medium |
US6147680A (en) | 1997-06-03 | 2000-11-14 | Koa T&T Corporation | Touchpad with interleaved traces |
US6169538B1 (en) | 1998-08-13 | 2001-01-02 | Motorola, Inc. | Method and apparatus for implementing a graphical user interface keyboard and a text buffer on electronic devices |
US6175644B1 (en) | 1998-05-01 | 2001-01-16 | Cognex Corporation | Machine vision system for object feature analysis and validation based on multiple object images |
US6184926B1 (en) | 1996-11-26 | 2001-02-06 | Ncr Corporation | System and method for detecting a human face in uncontrolled environments |
US6188391B1 (en) | 1998-07-09 | 2001-02-13 | Synaptics, Inc. | Two-layer capacitive touchpad and method of making same |
US6222508B1 (en) | 1995-08-25 | 2001-04-24 | Massachusetts Institute Of Technology | VLSI visual display |
US6262717B1 (en) | 1998-07-02 | 2001-07-17 | Cirque Corporation | Kiosk touch pad |
US6271860B1 (en) | 1997-07-30 | 2001-08-07 | David Gross | Method and system for display of an additional dimension |
US6281887B1 (en) | 1999-09-20 | 2001-08-28 | Mitac International Corp. | Touchpad assembly |
US6279170B1 (en) | 1996-12-19 | 2001-08-28 | Victor Chu | Active labels for garments |
US6283858B1 (en) | 1997-02-25 | 2001-09-04 | Bgk International Incorporated | Method for manipulating images |
US20010030637A1 (en) | 1995-04-28 | 2001-10-18 | Hewlett-Packard Company | Electro-optic displays |
US6311162B1 (en) | 1998-07-25 | 2001-10-30 | Ernst F. Reichwein | Interactive symptomatic recording system and methods |
US6310610B1 (en) | 1997-12-04 | 2001-10-30 | Nortel Networks Limited | Intelligent touch display |
US20020008711A1 (en) | 1997-07-18 | 2002-01-24 | Juha Pekka Karjalainen | Apparatus and method for displaying zoomed version of stored image by displaying and shifting absed on pixel overlap |
US6380929B1 (en) | 1996-09-20 | 2002-04-30 | Synaptics, Incorporated | Pen drawing computer input device |
US6392636B1 (en) | 1998-01-22 | 2002-05-21 | Stmicroelectronics, Inc. | Touchpad providing screen cursor/pointer movement control |
US20020080123A1 (en) | 2000-12-26 | 2002-06-27 | International Business Machines Corporation | Method for touchscreen data input |
JP2002342033A (en) | 2001-05-21 | 2002-11-29 | Sony Corp | Non-contact type user input device |
US6492979B1 (en) | 1999-09-07 | 2002-12-10 | Elo Touchsystems, Inc. | Dual sensor touchscreen utilizing projective-capacitive and force touch sensors |
US20020191029A1 (en) | 2001-05-16 | 2002-12-19 | Synaptics, Inc. | Touch screen with user interface enhancement |
US6504530B1 (en) | 1999-09-07 | 2003-01-07 | Elo Touchsystems, Inc. | Touch confirming touchscreen utilizing plural touch sensors |
US6518957B1 (en) | 1999-08-13 | 2003-02-11 | Nokia Mobile Phones Limited | Communications device with touch sensitive screen |
US6552719B2 (en) | 1999-01-07 | 2003-04-22 | Microsoft Corporation | System and method for automatically switching between writing and text input modes |
US6567549B1 (en) | 1996-12-05 | 2003-05-20 | Palmsource Inc. | Method and apparatus for immediate response handwriting recognition system that handles multiple character sets |
US20030117380A1 (en) | 1999-03-16 | 2003-06-26 | International Business Machines Corporation | Method of inputting a numerical value using a touch panel and input apparatus thereof |
US20030161517A1 (en) | 2002-02-22 | 2003-08-28 | Pieter Vuylsteke | Method of normalising a digital signal representation of an image |
US6664982B1 (en) | 1995-10-16 | 2003-12-16 | Nec Corporation | Multi-user on-screen keyboard |
US20040001048A1 (en) | 2002-06-28 | 2004-01-01 | Microsoft Corporation | Method and system for detecting multiple touches on a touch-sensitive screen |
US6690387B2 (en) | 2001-12-28 | 2004-02-10 | Koninklijke Philips Electronics N.V. | Touch-screen image scrolling system and method |
WO2004021261A1 (en) | 2002-08-28 | 2004-03-11 | Philips Intellectual Property & Standards Gmbh | Method and arrangement for assessing the quality of skin print images |
US6727891B2 (en) | 2001-07-03 | 2004-04-27 | Netmor, Ltd. | Input device for personal digital assistants |
US6738154B1 (en) | 1997-01-21 | 2004-05-18 | Xerox Corporation | Locating the position and orientation of multiple objects with a smart platen |
JP2004178547A (en) | 2002-09-30 | 2004-06-24 | Sanyo Electric Co Ltd | Portable electronic device |
US20040119700A1 (en) | 2002-12-20 | 2004-06-24 | Fumiko Ichikawa | Touch sensitive input system |
US6788815B2 (en) | 2000-11-10 | 2004-09-07 | Microsoft Corporation | System and method for accepting disparate types of user input |
US20040178998A1 (en) | 2003-03-14 | 2004-09-16 | Sharp Jeffrey L. | Water tolerant touch sensor |
US6924790B1 (en) | 1995-10-16 | 2005-08-02 | Nec Corporation | Mode switching for pen-based computer systems |
US20050227217A1 (en) | 2004-03-31 | 2005-10-13 | Wilson Andrew D | Template matching on interactive surface |
US20050226505A1 (en) | 2004-03-31 | 2005-10-13 | Wilson Andrew D | Determining connectedness and offset of 3D objects relative to an interactive surface |
US20060026521A1 (en) | 2004-07-30 | 2006-02-02 | Apple Computer, Inc. | Gestures for touch sensitive input devices |
US7015894B2 (en) | 2001-09-28 | 2006-03-21 | Ricoh Company, Ltd. | Information input and output system, method, storage medium, and carrier wave |
US7030862B2 (en) | 2001-08-30 | 2006-04-18 | Canon Kabushiki Kaisha | Image forming apparatus, control method for the same, control program, and storage medium storing the control program |
US7030860B1 (en) | 1999-10-08 | 2006-04-18 | Synaptics Incorporated | Flexible transparent touch sensing system for electronic devices |
US7038659B2 (en) | 2002-04-06 | 2006-05-02 | Janusz Wiktor Rajkowski | Symbol encoding apparatus and method |
US20060097991A1 (en) * | 2004-05-06 | 2006-05-11 | Apple Computer, Inc. | Multipoint touchscreen |
US7053886B2 (en) | 2001-12-27 | 2006-05-30 | Lg.Philips Lcd Co., Ltd. | Liquid crystal panel device having a touch panel and method for driving the same |
WO2006074289A2 (en) | 2005-01-07 | 2006-07-13 | Gesturetek, Inc. | Detecting and tracking objects in images |
US20060197753A1 (en) | 2005-03-04 | 2006-09-07 | Hotelling Steven P | Multi-functional hand-held device |
US20060244735A1 (en) | 2005-04-29 | 2006-11-02 | Microsoft Corporation | System and method for fine cursor positioning using a low resolution imaging touch screen |
US20060285743A1 (en) | 2005-06-20 | 2006-12-21 | Shih-Jong J. Lee | Object based boundary refinement method |
KR100664964B1 (en) | 2005-10-11 | 2007-01-04 | 삼성전자주식회사 | Apparatus and method for operating according touch sensor |
US7274353B2 (en) | 2003-04-02 | 2007-09-25 | Elan Microelectronics Corporation | Capacitive touchpad integrated with key and handwriting functions |
US7283126B2 (en) | 2002-06-12 | 2007-10-16 | Smart Technologies Inc. | System and method for providing gesture suggestions to enhance interpretation of user input |
US7292229B2 (en) | 2002-08-29 | 2007-11-06 | N-Trig Ltd. | Transparent digitiser |
US20080012835A1 (en) * | 2006-07-12 | 2008-01-17 | N-Trig Ltd. | Hover and touch detection for digitizer |
US20080158146A1 (en) | 2007-01-03 | 2008-07-03 | Apple Computer, Inc. | Irregular input identification |
US7428142B1 (en) | 2004-08-25 | 2008-09-23 | Apple Inc. | Lid-closed detector |
US7444163B2 (en) | 2002-09-30 | 2008-10-28 | Sanyo Electric Co., Ltd | Mobile digital devices |
US7499035B2 (en) | 2003-08-21 | 2009-03-03 | Microsoft Corporation | Focus management using in-air points |
US7633076B2 (en) | 2005-09-30 | 2009-12-15 | Apple Inc. | Automated response to and sensing of user activity in portable devices |
US7714846B1 (en) * | 2004-08-26 | 2010-05-11 | Wacom Co., Ltd. | Digital signal processed touchscreen system |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5189711A (en) | 1989-11-24 | 1993-02-23 | Isaac Weiss | Automatic detection of elliptical shapes |
JP3228584B2 (en) | 1993-02-01 | 2001-11-12 | 京セラ株式会社 | Mouse input method on touch panel |
US5485565A (en) | 1993-08-04 | 1996-01-16 | Xerox Corporation | Gestural indicators for selecting graphic objects |
-
2007
- 2007-01-03 US US11/619,490 patent/US8269727B2/en not_active Expired - Fee Related
- 2007-12-21 WO PCT/US2007/026299 patent/WO2008085458A2/en active Application Filing
-
2012
- 2012-08-13 US US13/584,725 patent/US9411468B2/en active Active
-
2016
- 2016-08-08 US US15/231,683 patent/US10025429B2/en active Active
Patent Citations (234)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4056754A (en) | 1975-03-01 | 1977-11-01 | Electric Valve Company Limited | Vidicon having switchable electrode connections |
US4219847A (en) | 1978-03-01 | 1980-08-26 | Canadian Patents & Development Limited | Method and apparatus of determining the center of area or centroid of a geometrical area of unspecified shape lying in a larger x-y scan field |
US4293734A (en) | 1979-02-23 | 1981-10-06 | Peptek, Incorporated | Touch panel system and method |
US4340911A (en) | 1979-02-28 | 1982-07-20 | Fuji Photo Film Co., Ltd. | Image gradation processing method and apparatus for mammogram copying system |
US4290052A (en) | 1979-10-26 | 1981-09-15 | General Electric Company | Capacitive touch entry apparatus having high degree of personal safety |
US4570193A (en) | 1981-05-08 | 1986-02-11 | Canon Kabushiki Kaisha | Video signal processing device including emphasis and/or de-emphasis circuit |
US4444998A (en) | 1981-10-27 | 1984-04-24 | Spectra-Symbol Corporation | Touch controlled membrane for multi axis voltage selection |
US4617563A (en) | 1982-12-28 | 1986-10-14 | Seiko Epson Kabushiki Kaisha | Liquid crystal display device |
US4618989A (en) | 1983-01-21 | 1986-10-21 | Michio Kawata, Director-General of Agency of Industrial Science and Technology | Method and system for detecting elliptical objects |
GB2139762A (en) | 1983-04-08 | 1984-11-14 | Gavilan Computer Corp | An input device and a method of inputting data to a computer system |
US4526043A (en) | 1983-05-23 | 1985-07-02 | At&T Bell Laboratories | Conformable tactile sensor |
US4954967A (en) | 1983-09-21 | 1990-09-04 | Canon Kabushiki Kaisha | Information processing apparatus |
US4550221A (en) | 1983-10-07 | 1985-10-29 | Scott Mabusth | Touch sensitive control device |
US4598420A (en) | 1983-12-08 | 1986-07-01 | Mts Systems Corporation | Optical grid analyzer system for automatically determining strain in deformed sheet metal |
JPS6230888A (en) | 1985-08-02 | 1987-02-09 | Hitachi Ltd | Heat transmission pipe |
US4736191A (en) | 1985-08-02 | 1988-04-05 | Karl E. Matzke | Touch activated control method and apparatus |
US4906983A (en) | 1985-08-16 | 1990-03-06 | British Aerospace Public Limited Company | Display data memory |
US4678869A (en) | 1985-10-25 | 1987-07-07 | Scriptel Corporation | Position responsive apparatus, system and method having electrographic application |
US4686332A (en) | 1986-06-26 | 1987-08-11 | International Business Machines Corporation | Combined finger touch and stylus detection system for use on the viewing surface of a visual display device |
US4763356A (en) | 1986-12-11 | 1988-08-09 | AT&T Information Systems, Inc. American Telephone and Telegraph Company | Touch screen form entry system |
US4899138A (en) | 1987-01-10 | 1990-02-06 | Pioneer Electronic Corporation | Touch panel control device with touch time and finger direction discrimination |
US4746770A (en) | 1987-02-17 | 1988-05-24 | Sensor Frame Incorporated | Method and apparatus for isolating and manipulating graphic objects on computer video monitor |
US4850677A (en) | 1987-03-18 | 1989-07-25 | Nec Corporation | Liquid crystal display device having improved electrodes for a multi-tone image |
US4952932A (en) | 1987-04-30 | 1990-08-28 | Canon Kabushiki Kaisha | Information input apparatus |
US4806709A (en) | 1987-05-26 | 1989-02-21 | Microtouch Systems, Inc. | Method of and apparatus for sensing the location, such as coordinates, of designated points on an electrically sensitive touch-screen surface |
US4896363A (en) | 1987-05-28 | 1990-01-23 | Thumbscan, Inc. | Apparatus and method for matching image characteristics such as fingerprint minutiae |
US4885639A (en) | 1987-06-09 | 1989-12-05 | Sony Corporation | Apparatus for eliminating a noise signal |
US5150105A (en) | 1987-08-28 | 1992-09-22 | Thomson Csf | System for the display of color images on matrix display panels |
US5189732A (en) | 1987-11-18 | 1993-02-23 | Hitachi, Ltd. | Touch panel input apparatus |
US4914624A (en) | 1988-05-06 | 1990-04-03 | Dunthorn David I | Virtual button for touch screen |
US4968877A (en) | 1988-09-14 | 1990-11-06 | Sensor Frame Corporation | VideoHarp |
US5012524A (en) | 1989-02-27 | 1991-04-30 | Motorola, Inc. | Automatic inspection method |
US5073950A (en) | 1989-04-13 | 1991-12-17 | Personnel Identification & Entry Access Control, Inc. | Finger profile identification system |
US5252951A (en) | 1989-04-28 | 1993-10-12 | International Business Machines Corporation | Graphical user interface with gesture recognition in a multiapplication environment |
EP0397428A2 (en) | 1989-05-08 | 1990-11-14 | Canon Kabushiki Kaisha | Image processing apparatus |
US5327161A (en) | 1989-08-09 | 1994-07-05 | Microtouch Systems, Inc. | System and method for emulating a mouse input device with a touchpad input device |
US5305017A (en) | 1989-08-16 | 1994-04-19 | Gerpheide George E | Methods and apparatus for data input |
US5446480A (en) | 1989-08-17 | 1995-08-29 | Sony Corporation | Input device for entering data for electronic equipments and so forth |
US5267327A (en) | 1990-03-30 | 1993-11-30 | Sony Corporation | Apparatus and method for registering the handwriting of a user so it can be translated into block characters |
USRE34476E (en) | 1990-05-14 | 1993-12-14 | Norwood Donald D | Hybrid information management system for handwriting and text |
US5459463A (en) | 1990-05-25 | 1995-10-17 | Sextant Avionique | Device for locating an object situated close to a detection area and a transparent keyboard using said device |
US5276794A (en) | 1990-09-25 | 1994-01-04 | Grid Systems Corporation | Pop-up keyboard system for entering handwritten data into computer generated forms |
US5347295A (en) | 1990-10-31 | 1994-09-13 | Go Corporation | Control of a computer through a position-sensed stylus |
US5491495A (en) | 1990-11-13 | 1996-02-13 | Wang Laboratories, Inc. | User interface having simulated devices |
US5432531A (en) | 1990-12-14 | 1995-07-11 | International Business Machines Corporation | Coordinate processor for a computer system having a pointing device |
US5113041A (en) | 1990-12-28 | 1992-05-12 | At&T Bell Laboratories | Information processing |
US5898434A (en) | 1991-05-15 | 1999-04-27 | Apple Computer, Inc. | User interface system having programmable user interface elements |
US5548667A (en) | 1991-05-24 | 1996-08-20 | Sony Corporation | Image processing system and method thereof in which three dimensional shape is reproduced from two dimensional image data |
US5956682A (en) | 1991-06-06 | 1999-09-21 | Lj Laboratories, Llc | Picture frame with associated audio messages and position sensitive or speech recognition device |
US6353433B1 (en) | 1991-06-17 | 2002-03-05 | Alfred L. Schumer | Digitizer interface |
US5768492A (en) | 1991-06-17 | 1998-06-16 | Schumer; Alfred L. | Digitizer interface |
US5293430A (en) | 1991-06-27 | 1994-03-08 | Xerox Corporation | Automatic image segmentation using local area maximum and minimum image signals |
US5168531A (en) | 1991-06-27 | 1992-12-01 | Digital Equipment Corporation | Real-time recognition of pointing information from video |
US5376946A (en) | 1991-07-08 | 1994-12-27 | Mikan; Peter J. | Computer mouse simulator device |
US5386219A (en) | 1991-10-16 | 1995-01-31 | International Business Machines Corp. | Touch overlay for improved touch sensitivity |
US5588098A (en) | 1991-11-22 | 1996-12-24 | Apple Computer, Inc. | Method and apparatus for direct manipulation of 3-D objects on computer displays |
US5479528A (en) | 1991-12-13 | 1995-12-26 | At&T Corp. | Intelligent work surfaces |
US5483261A (en) | 1992-02-14 | 1996-01-09 | Itu Research, Inc. | Graphical input controller and method with rear screen image detection |
US5852434A (en) | 1992-04-03 | 1998-12-22 | Sekendur; Oral F. | Absolute optical position determination |
US5673066A (en) | 1992-04-21 | 1997-09-30 | Alps Electric Co., Ltd. | Coordinate input device |
US5457476A (en) | 1992-05-27 | 1995-10-10 | Apple Computer, Inc. | Method for controlling a computerized organizer |
US5880411A (en) | 1992-06-08 | 1999-03-09 | Synaptics, Incorporated | Object position detector with edge motion feature and gesture recognition |
US6610936B2 (en) | 1992-06-08 | 2003-08-26 | Synaptics, Inc. | Object position detector with edge motion feature and gesture recognition |
US5648642A (en) | 1992-06-08 | 1997-07-15 | Synaptics, Incorporated | Object position detector |
US5488204A (en) | 1992-06-08 | 1996-01-30 | Synaptics, Incorporated | Paintbrush stylus for capacitive touch sensor pad |
US5543588A (en) | 1992-06-08 | 1996-08-06 | Synaptics, Incorporated | Touch pad driven handheld computing device |
US5543591A (en) | 1992-06-08 | 1996-08-06 | Synaptics, Incorporated | Object position detector with edge motion feature and gesture recognition |
US5543590A (en) | 1992-06-08 | 1996-08-06 | Synaptics, Incorporated | Object position detector with edge motion feature |
US5541372A (en) | 1992-06-15 | 1996-07-30 | U.S. Philips Corporation | Force activated touch screen measuring deformation of the front panel |
US5461655A (en) * | 1992-06-19 | 1995-10-24 | Agfa-Gevaert | Method and apparatus for noise reduction |
US5422831A (en) | 1992-08-03 | 1995-06-06 | Iowa State University Research Foundation, Inc. | Acoustic and video imaging system for quality determination of pharmaceutical products |
US5309374A (en) | 1992-08-03 | 1994-05-03 | Iowa State University Research Foundation, Inc. | Acoustic and video imaging system for quality determination of agricultural products |
US6008800A (en) | 1992-09-18 | 1999-12-28 | Pryor; Timothy R. | Man machine interfaces for entering data into a computer |
US5420936A (en) | 1992-10-16 | 1995-05-30 | International Business Machines Corporation | Method and apparatus for accessing touch screen desktop objects via fingerprint recognition |
US5583543A (en) | 1992-11-05 | 1996-12-10 | Sharp Kabushiki Kaisha | Pen input processing apparatus |
JPH06161661A (en) | 1992-11-18 | 1994-06-10 | Digital:Kk | Touch pannel input device capable of plural simultaneous inputs and method thereof |
US5612719A (en) | 1992-12-03 | 1997-03-18 | Apple Computer, Inc. | Gesture sensitive buttons for graphical user interfaces |
US5513309A (en) | 1993-01-05 | 1996-04-30 | Apple Computer, Inc. | Graphic editor user interface for a pointer-based computer system |
US5463388A (en) | 1993-01-29 | 1995-10-31 | At&T Ipm Corp. | Computer mouse or keyboard input device utilizing capacitive sensors |
US5546334A (en) | 1993-03-29 | 1996-08-13 | Acer Incorporated | Notebook computer system with a separable trackball |
US5627567A (en) | 1993-04-27 | 1997-05-06 | Hewlett-Packard Company | Method and apparatus for adaptive touch recognition in a touch sensitive user interface |
US5414227A (en) | 1993-04-29 | 1995-05-09 | International Business Machines Corporation | Stylus tilt detection apparatus for communication with a remote digitizing display |
US5603053A (en) | 1993-05-10 | 1997-02-11 | Apple Computer, Inc. | System for entering data into an active application currently running in the foreground by selecting an input icon in a palette representing input utility |
US5825675A (en) | 1993-07-29 | 1998-10-20 | Xerox Corporation | Apparatus and configuration method for a small, hand-held computing device |
US5367453A (en) | 1993-08-02 | 1994-11-22 | Apple Computer, Inc. | Method and apparatus for correcting words |
US5583946A (en) | 1993-09-30 | 1996-12-10 | Apple Computer, Inc. | Method and apparatus for recognizing gestures on a computer system |
JPH07110741A (en) | 1993-10-13 | 1995-04-25 | Digital:Kk | Method and device for detection of pointed coordinate |
US5977950A (en) | 1993-11-29 | 1999-11-02 | Motorola, Inc. | Manually controllable cursor in a virtual image |
US5638093A (en) | 1993-12-07 | 1997-06-10 | Seiko Epson Corporation | Touch panel input device and control method thereof |
US5982302A (en) | 1994-03-07 | 1999-11-09 | Ure; Michael J. | Touch-sensitive keyboard/mouse |
US5875311A (en) | 1994-03-18 | 1999-02-23 | International Business Machines Corporation | Computer system with touchpad support in operating system |
US5852669A (en) | 1994-04-06 | 1998-12-22 | Lucent Technologies Inc. | Automatic face and facial feature location detection for low bit rate model-assisted H.261 compatible coding of video |
US5572717A (en) | 1994-04-06 | 1996-11-05 | Altera Corporation | Method and apparatus for assigning and analyzing timing specifications in a computer aided engineering program |
US5631976A (en) | 1994-04-29 | 1997-05-20 | International Business Machines Corporation | Object imaging system |
US5546475A (en) | 1994-04-29 | 1996-08-13 | International Business Machines Corporation | Produce recognition system |
US5600735A (en) | 1994-05-10 | 1997-02-04 | Motorola, Inc. | Method of recognizing handwritten input |
US5469194A (en) | 1994-05-13 | 1995-11-21 | Apple Computer, Inc. | Apparatus and method for providing different input device orientations of a computer system |
US5594806A (en) | 1994-06-20 | 1997-01-14 | Personnel Identification & Entry Access Control, Inc. | Knuckle profile indentity verification system |
US5570113A (en) | 1994-06-29 | 1996-10-29 | International Business Machines Corporation | Computer based pen system and method for automatically cancelling unwanted gestures and preventing anomalous signals as inputs to such system |
US5734751A (en) | 1994-07-22 | 1998-03-31 | Nec Corporation | Ellipse-like curve recognition by calculation of distances of points on the curve from FOCI |
US5666438A (en) | 1994-07-29 | 1997-09-09 | Apple Computer, Inc. | Method and apparatus for recognizing handwriting of different users of a pen-based computer system |
US5796389A (en) | 1994-08-22 | 1998-08-18 | International Game Technology | Reduced noise touch screen apparatus and method |
US5974558A (en) * | 1994-09-02 | 1999-10-26 | Packard Bell Nec | Resume on pen contact |
US5734742A (en) | 1994-09-19 | 1998-03-31 | Nissan Motor Co., Ltd. | Inspection system and process |
US5963190A (en) | 1994-09-26 | 1999-10-05 | Canon Kabushiki Kaisha | Driving method for display device and display apparatus |
US5809166A (en) | 1994-11-09 | 1998-09-15 | Canon Kabushiki Kaisha | Character recognition system which cuts between touching characters |
WO1996015464A1 (en) | 1994-11-14 | 1996-05-23 | Moonstone Technology Limited | Capacitive touch detectors |
US5751851A (en) | 1994-11-14 | 1998-05-12 | Motorola, Inc. | Method of splitting handwritten input |
US5505072A (en) | 1994-11-15 | 1996-04-09 | Tekscan, Inc. | Scanning circuit for pressure responsive array |
US5805144A (en) | 1994-12-14 | 1998-09-08 | Dell Usa, L.P. | Mouse pointing device having integrated touchpad |
US5798756A (en) | 1994-12-26 | 1998-08-25 | Sharp Kabushiki Kaisha | Coordinate detection device capable of obviating the need of forming closed loop in electrodes thereof |
US5585823A (en) | 1994-12-30 | 1996-12-17 | Apple Computer, Inc. | Multi-state one-button computer pointing device |
US5764218A (en) | 1995-01-31 | 1998-06-09 | Apple Computer, Inc. | Method and apparatus for contacting a touch-sensitive cursor-controlling input device to generate button values |
US6005959A (en) | 1995-02-17 | 1999-12-21 | International Business Machines Corporation | Produce size recognition system |
US6310964B1 (en) | 1995-02-17 | 2001-10-30 | International Business Machines Corp. | Produce size recognition system |
US5649070A (en) | 1995-02-17 | 1997-07-15 | International Business Machines Corporation | Learning system with prototype replacement |
US5838302A (en) | 1995-02-24 | 1998-11-17 | Casio Computer Co., Ltd. | Data inputting devices for inputting typed and handwritten data in a mixed manner |
US20010030637A1 (en) | 1995-04-28 | 2001-10-18 | Hewlett-Packard Company | Electro-optic displays |
US5812698A (en) | 1995-05-12 | 1998-09-22 | Synaptics, Inc. | Handwriting recognition system and method |
US5732230A (en) | 1995-05-19 | 1998-03-24 | Richo Company Ltd. | Computer user interface for manipulating image fragments using drag, drop and merge operations |
US5771342A (en) | 1995-06-05 | 1998-06-23 | Saltire Software | Method and apparatus for dynamically displaying consistently dimensioned two-dimensional drawings |
US5797396A (en) | 1995-06-07 | 1998-08-25 | University Of Florida Research Foundation | Automated method for digital image quantitation |
US6088025A (en) | 1995-08-24 | 2000-07-11 | Matsushita Electric Industrial Co., Ltd. | Terminal device with built-in image sensor |
US6222508B1 (en) | 1995-08-25 | 2001-04-24 | Massachusetts Institute Of Technology | VLSI visual display |
US5552787A (en) | 1995-10-10 | 1996-09-03 | The United States Of America As Represented By The Secretary Of The Navy | Measurement of topography using polarimetric synthetic aperture radar (SAR) |
US6664982B1 (en) | 1995-10-16 | 2003-12-16 | Nec Corporation | Multi-user on-screen keyboard |
US6924790B1 (en) | 1995-10-16 | 2005-08-02 | Nec Corporation | Mode switching for pen-based computer systems |
US5856822A (en) | 1995-10-27 | 1999-01-05 | 02 Micro, Inc. | Touch-pad digital computer pointing-device |
US6029214A (en) | 1995-11-03 | 2000-02-22 | Apple Computer, Inc. | Input tablet system with user programmable absolute coordinate mode and relative coordinate mode segments |
US5943043A (en) | 1995-11-09 | 1999-08-24 | International Business Machines Corporation | Touch panel "double-touch" input method and detection apparatus |
US6016355A (en) | 1995-12-15 | 2000-01-18 | Veridicom, Inc. | Capacitive fingerprint acquisition sensor |
US5825352A (en) | 1996-01-04 | 1998-10-20 | Logitech, Inc. | Multiple fingers contact sensing method for emulating mouse buttons and mouse operations on a touch sensor pad |
US5831602A (en) | 1996-01-11 | 1998-11-03 | Canon Kabushiki Kaisha | Information processing apparatus, method and computer program product |
US5717777A (en) | 1996-01-11 | 1998-02-10 | Dew Engineering And Development Limited | Longest line method and apparatus for fingerprint alignment |
US5729604A (en) | 1996-03-14 | 1998-03-17 | Northern Telecom Limited | Safety switch for communication device |
US5940065A (en) | 1996-03-15 | 1999-08-17 | Elo Touchsystems, Inc. | Algorithmic compensation system and method therefor for a touch sensor panel |
US5987162A (en) | 1996-03-27 | 1999-11-16 | Mitsubishi Denki Kabushiki Kaisha | Image processing method and apparatus for recognizing an arrangement of an object |
US5764222A (en) | 1996-05-28 | 1998-06-09 | International Business Machines Corporation | Virtual pointing device for touchscreens |
US5748184A (en) | 1996-05-28 | 1998-05-05 | International Business Machines Corporation | Virtual pointing device for touchscreens |
US5874948A (en) | 1996-05-28 | 1999-02-23 | International Business Machines Corporation | Virtual pointing device for touchscreens |
US5898422A (en) | 1996-05-31 | 1999-04-27 | International Business Machines Corporation | Method and system for recognition of pointers |
US5757478A (en) | 1996-06-03 | 1998-05-26 | Ma; Chris Chen-Hsing | Remote position sensing apparatus and method |
US6049329A (en) | 1996-06-04 | 2000-04-11 | International Business Machines Corporartion | Method of and system for facilitating user input into a small GUI window using a stylus |
US6067079A (en) | 1996-06-13 | 2000-05-23 | International Business Machines Corporation | Virtual pointing device for touchscreens |
US5835079A (en) | 1996-06-13 | 1998-11-10 | International Business Machines Corporation | Virtual pointing device for touchscreens |
US5808605A (en) | 1996-06-13 | 1998-09-15 | International Business Machines Corporation | Virtual pointing device for touchscreens |
US5862256A (en) | 1996-06-14 | 1999-01-19 | International Business Machines Corporation | Distinguishing gestures from handwriting in a pen based computer by size discrimination |
US5790104A (en) | 1996-06-25 | 1998-08-04 | International Business Machines Corporation | Multiple, moveable, customizable virtual pointing devices |
US5856824A (en) | 1996-06-25 | 1999-01-05 | International Business Machines Corp. | Reshapable pointing device for touchscreens |
US5812118A (en) | 1996-06-25 | 1998-09-22 | International Business Machines Corporation | Method, apparatus, and memory for creating at least two virtual pointing devices |
US5933134A (en) | 1996-06-25 | 1999-08-03 | International Business Machines Corporation | Touch screen virtual pointing device which goes into a translucent hibernation state when not in use |
US6023313A (en) | 1996-08-21 | 2000-02-08 | Sony Corporation | Liquid crystal display panel manufacturing method |
US6121960A (en) | 1996-08-28 | 2000-09-19 | Via, Inc. | Touch screen systems and methods |
US5896126A (en) | 1996-08-29 | 1999-04-20 | International Business Machines Corporation | Selection device for touchscreen systems |
US5936615A (en) | 1996-09-12 | 1999-08-10 | Digital Equipment Corporation | Image-based touchscreen |
US6380929B1 (en) | 1996-09-20 | 2002-04-30 | Synaptics, Incorporated | Pen drawing computer input device |
US5870083A (en) | 1996-10-04 | 1999-02-09 | International Business Machines Corporation | Breakaway touchscreen pointing device |
US5872559A (en) | 1996-10-04 | 1999-02-16 | International Business Machines Corporation | Breakaway and re-grow touchscreen pointing device |
US6005963A (en) | 1996-10-23 | 1999-12-21 | International Business Machines Corporation | System and method for determining if a fingerprint image contains an image portion representing a partial fingerprint impression |
US6075520A (en) | 1996-11-15 | 2000-06-13 | Rohm Co., Ltd. | Small current detector circuit and locator device using the same |
US5870771A (en) | 1996-11-15 | 1999-02-09 | Oberg; Larry B. | Computerized system for selecting, adjusting, and previewing framing product combinations for artwork and other items to be framed |
US6184926B1 (en) | 1996-11-26 | 2001-02-06 | Ncr Corporation | System and method for detecting a human face in uncontrolled environments |
US6567549B1 (en) | 1996-12-05 | 2003-05-20 | Palmsource Inc. | Method and apparatus for immediate response handwriting recognition system that handles multiple character sets |
US6279170B1 (en) | 1996-12-19 | 2001-08-28 | Victor Chu | Active labels for garments |
US6128014A (en) | 1997-01-10 | 2000-10-03 | Tokyo University Of Agriculture And Technology | Human interactive type display system |
US6738154B1 (en) | 1997-01-21 | 2004-05-18 | Xerox Corporation | Locating the position and orientation of multiple objects with a smart platen |
US6283858B1 (en) | 1997-02-25 | 2001-09-04 | Bgk International Incorporated | Method for manipulating images |
US5793355A (en) | 1997-03-03 | 1998-08-11 | Compaq Computer Corporation | Portable computer with interchangeable pointing device modules |
US6118435A (en) | 1997-04-10 | 2000-09-12 | Idec Izumi Corporation | Display unit with touch panel |
US6147680A (en) | 1997-06-03 | 2000-11-14 | Koa T&T Corporation | Touchpad with interleaved traces |
US20020008711A1 (en) | 1997-07-18 | 2002-01-24 | Juha Pekka Karjalainen | Apparatus and method for displaying zoomed version of stored image by displaying and shifting absed on pixel overlap |
US6271860B1 (en) | 1997-07-30 | 2001-08-07 | David Gross | Method and system for display of an additional dimension |
US6037882A (en) | 1997-09-30 | 2000-03-14 | Levy; David H. | Method and apparatus for inputting data to an electronic system |
US6310610B1 (en) | 1997-12-04 | 2001-10-30 | Nortel Networks Limited | Intelligent touch display |
US6392636B1 (en) | 1998-01-22 | 2002-05-21 | Stmicroelectronics, Inc. | Touchpad providing screen cursor/pointer movement control |
WO1999038149A1 (en) | 1998-01-26 | 1999-07-29 | Wayne Westerman | Method and apparatus for integrating manual input |
US20020015024A1 (en) * | 1998-01-26 | 2002-02-07 | University Of Delaware | Method and apparatus for integrating manual input |
US20050104867A1 (en) | 1998-01-26 | 2005-05-19 | University Of Delaware | Method and apparatus for integrating manual input |
US7339580B2 (en) | 1998-01-26 | 2008-03-04 | Apple Inc. | Method and apparatus for integrating manual input |
US6323846B1 (en) | 1998-01-26 | 2001-11-27 | University Of Delaware | Method and apparatus for integrating manual input |
US6175644B1 (en) | 1998-05-01 | 2001-01-16 | Cognex Corporation | Machine vision system for object feature analysis and validation based on multiple object images |
US6262717B1 (en) | 1998-07-02 | 2001-07-17 | Cirque Corporation | Kiosk touch pad |
US6188391B1 (en) | 1998-07-09 | 2001-02-13 | Synaptics, Inc. | Two-layer capacitive touchpad and method of making same |
US6311162B1 (en) | 1998-07-25 | 2001-10-30 | Ernst F. Reichwein | Interactive symptomatic recording system and methods |
US6169538B1 (en) | 1998-08-13 | 2001-01-02 | Motorola, Inc. | Method and apparatus for implementing a graphical user interface keyboard and a text buffer on electronic devices |
JP2000163031A (en) | 1998-11-25 | 2000-06-16 | Seiko Epson Corp | Portable information equipment and information storage medium |
GB2344894A (en) | 1998-12-17 | 2000-06-21 | Hewlett Packard Co | Palm pressure rejection for touchscreens |
US6552719B2 (en) | 1999-01-07 | 2003-04-22 | Microsoft Corporation | System and method for automatically switching between writing and text input modes |
US20030117380A1 (en) | 1999-03-16 | 2003-06-26 | International Business Machines Corporation | Method of inputting a numerical value using a touch panel and input apparatus thereof |
US6657614B1 (en) | 1999-04-21 | 2003-12-02 | Fuji Xerox Co., Ltd. | Detecting apparatus, input apparatus, pointing device, individual identification apparatus, and recording medium |
JP2000300543A (en) | 1999-04-21 | 2000-10-31 | Fuji Xerox Co Ltd | Detecting device, inputting device, pointing device, individual identification device and recording medium |
US6518957B1 (en) | 1999-08-13 | 2003-02-11 | Nokia Mobile Phones Limited | Communications device with touch sensitive screen |
US6504530B1 (en) | 1999-09-07 | 2003-01-07 | Elo Touchsystems, Inc. | Touch confirming touchscreen utilizing plural touch sensors |
US6492979B1 (en) | 1999-09-07 | 2002-12-10 | Elo Touchsystems, Inc. | Dual sensor touchscreen utilizing projective-capacitive and force touch sensors |
US6281887B1 (en) | 1999-09-20 | 2001-08-28 | Mitac International Corp. | Touchpad assembly |
US7030860B1 (en) | 1999-10-08 | 2006-04-18 | Synaptics Incorporated | Flexible transparent touch sensing system for electronic devices |
US6788815B2 (en) | 2000-11-10 | 2004-09-07 | Microsoft Corporation | System and method for accepting disparate types of user input |
US20020080123A1 (en) | 2000-12-26 | 2002-06-27 | International Business Machines Corporation | Method for touchscreen data input |
US20020191029A1 (en) | 2001-05-16 | 2002-12-19 | Synaptics, Inc. | Touch screen with user interface enhancement |
JP2002342033A (en) | 2001-05-21 | 2002-11-29 | Sony Corp | Non-contact type user input device |
US6727891B2 (en) | 2001-07-03 | 2004-04-27 | Netmor, Ltd. | Input device for personal digital assistants |
US7030862B2 (en) | 2001-08-30 | 2006-04-18 | Canon Kabushiki Kaisha | Image forming apparatus, control method for the same, control program, and storage medium storing the control program |
US7015894B2 (en) | 2001-09-28 | 2006-03-21 | Ricoh Company, Ltd. | Information input and output system, method, storage medium, and carrier wave |
US7053886B2 (en) | 2001-12-27 | 2006-05-30 | Lg.Philips Lcd Co., Ltd. | Liquid crystal panel device having a touch panel and method for driving the same |
US7184064B2 (en) | 2001-12-28 | 2007-02-27 | Koninklijke Philips Electronics N.V. | Touch-screen image scrolling system and method |
US6690387B2 (en) | 2001-12-28 | 2004-02-10 | Koninklijke Philips Electronics N.V. | Touch-screen image scrolling system and method |
US20030161517A1 (en) | 2002-02-22 | 2003-08-28 | Pieter Vuylsteke | Method of normalising a digital signal representation of an image |
US7038659B2 (en) | 2002-04-06 | 2006-05-02 | Janusz Wiktor Rajkowski | Symbol encoding apparatus and method |
US7283126B2 (en) | 2002-06-12 | 2007-10-16 | Smart Technologies Inc. | System and method for providing gesture suggestions to enhance interpretation of user input |
US20040001048A1 (en) | 2002-06-28 | 2004-01-01 | Microsoft Corporation | Method and system for detecting multiple touches on a touch-sensitive screen |
WO2004021261A1 (en) | 2002-08-28 | 2004-03-11 | Philips Intellectual Property & Standards Gmbh | Method and arrangement for assessing the quality of skin print images |
US7292229B2 (en) | 2002-08-29 | 2007-11-06 | N-Trig Ltd. | Transparent digitiser |
JP2004178547A (en) | 2002-09-30 | 2004-06-24 | Sanyo Electric Co Ltd | Portable electronic device |
US7444163B2 (en) | 2002-09-30 | 2008-10-28 | Sanyo Electric Co., Ltd | Mobile digital devices |
US20040119700A1 (en) | 2002-12-20 | 2004-06-24 | Fumiko Ichikawa | Touch sensitive input system |
US20040178998A1 (en) | 2003-03-14 | 2004-09-16 | Sharp Jeffrey L. | Water tolerant touch sensor |
US7274353B2 (en) | 2003-04-02 | 2007-09-25 | Elan Microelectronics Corporation | Capacitive touchpad integrated with key and handwriting functions |
US7499035B2 (en) | 2003-08-21 | 2009-03-03 | Microsoft Corporation | Focus management using in-air points |
US20050226505A1 (en) | 2004-03-31 | 2005-10-13 | Wilson Andrew D | Determining connectedness and offset of 3D objects relative to an interactive surface |
US20050227217A1 (en) | 2004-03-31 | 2005-10-13 | Wilson Andrew D | Template matching on interactive surface |
US20060097991A1 (en) * | 2004-05-06 | 2006-05-11 | Apple Computer, Inc. | Multipoint touchscreen |
US20060026521A1 (en) | 2004-07-30 | 2006-02-02 | Apple Computer, Inc. | Gestures for touch sensitive input devices |
US7428142B1 (en) | 2004-08-25 | 2008-09-23 | Apple Inc. | Lid-closed detector |
US7714846B1 (en) * | 2004-08-26 | 2010-05-11 | Wacom Co., Ltd. | Digital signal processed touchscreen system |
WO2006074289A2 (en) | 2005-01-07 | 2006-07-13 | Gesturetek, Inc. | Detecting and tracking objects in images |
US20060197753A1 (en) | 2005-03-04 | 2006-09-07 | Hotelling Steven P | Multi-functional hand-held device |
US20060244735A1 (en) | 2005-04-29 | 2006-11-02 | Microsoft Corporation | System and method for fine cursor positioning using a low resolution imaging touch screen |
US20060285743A1 (en) | 2005-06-20 | 2006-12-21 | Shih-Jong J. Lee | Object based boundary refinement method |
US7633076B2 (en) | 2005-09-30 | 2009-12-15 | Apple Inc. | Automated response to and sensing of user activity in portable devices |
US20070083372A1 (en) | 2005-10-11 | 2007-04-12 | Samsung Electronics Co., Ltd. | Apparatus and method for controlling portable device |
KR100664964B1 (en) | 2005-10-11 | 2007-01-04 | 삼성전자주식회사 | Apparatus and method for operating according touch sensor |
US20080012835A1 (en) * | 2006-07-12 | 2008-01-17 | N-Trig Ltd. | Hover and touch detection for digitizer |
US20080158146A1 (en) | 2007-01-03 | 2008-07-03 | Apple Computer, Inc. | Irregular input identification |
US8269727B2 (en) * | 2007-01-03 | 2012-09-18 | Apple Inc. | Irregular input identification |
Non-Patent Citations (116)
Title |
---|
Adams, H.G. et al. (Jul. 1985). "Automated Radiographic Report Generation Using Barcode Technology," American Journal of Roentgenology 145(1):177-180. |
Agrawal, R. et al. (Jul. 1986). "An Overview of Tactile Sensing," Center for Research on Integrated Manufacturing: Robot Systems Division, The University of Michigan, 47 pages. |
Anonymous. (1989). "Concorde Version 3.0: Turns a PC into a Graphics Workstation," News and Views: Tektronix, Inc. Bioinformatics, 5(1):63-75. |
Anonymous. (Apr. 13, 2004). Proximity Sensor Demo Kit, User Guide, Version 0.62-Preliminary, Integration Associates, Inc., 14 pages. |
Anonymous. (May 8, 1992). "The Sensor Frame Graphic Manipulator," NASA Phase II Final Report, 28 pages. |
Apte, A. et al. (Nov. 1993). "Recognizing Multistroke Geometric Shapes: An Experimental Evaluation," Association for Computing Machinery UIST '93, Atlanta, GA, Nov. 3-5, 1993, 121-128. |
Bolle, R.M. et al. (1996) "Veggie Vision: A Producce Recognition System," IEEE pp. 244-251. |
Brown, E. et al. (1990). "Windows on Tablets As A Means of Achieving Vitrual Input Devices," Interact '90, pp. 675-681. |
Buxton, W. (1986). "There's More to Interaction Than Meets the Eye: Some Issues in Manual Input," User Centered System Design, New Perspectives on Human-Computer Interaction, pp. 319-337. |
Buxton, W. et al. (1986). "A Study in Two-Handed Input," Proceedings of CHI '86, pp. 321-326. |
Buxton, W. et al. (Jul. 1985). "Issues and Techniques in Touch-Sensitive Tablet Input," Association for Computing Machinery SIGGRAPH '85, San Francisco, CA, Jul. 22-26, 1985, 19(3):215-224. |
Buxton, W.A.S. (Mar./Apr. 1994). "Two-Handed Document Navigation," Xerox Disclosure Journal 19(2):103-108. |
Charlebois, M. et al. (1996). "Curvature Based Shape Estimation Using Tactile Sensing," Proceedings 1996 IEEE International Conference on Robotics and Automation, six pages. |
Charlebois, M.A. (Dec. 1996). "Exploring the Shape of Objects with Curved Surfaces Using Tactile Sensing," A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Applied Science, Simon Fraser University, Burnaby, British Columbia, Canada, 108 pages. |
Cirque Corporation. (1994). GlipePoint® Portable Touch Surface Round Mouse Post PS/2 Version User's Guide, Guide Version 1.0, 16 pages. |
Colosi, J. ed. (Dec. 15, 1984). Computer News, Hewlett-Packard 10(4):1-27. |
Dario, P. et al. (1986). "A Sensorised Scenario for Basic Investigation on Active Touch," in Robot Sensors vol. 2- Tactile and Non-Vision, Pugh, A. ed., IFS Publications: UK, pp. 237-245. |
Davies, E.R. (1990). Machine Vision: Theory, Algorithms, Practicalities, Academic Press, Inc.: San Diego, CA, 575 pages. |
Davies, E.R. (1990). Machine Vision: Theory, Algorithms, Practicalities, Academic Press, Inc.: San Diego, CA, pp. xi-xxi (Table of Contents Only.). |
Davies, E.R. (1997). "Boundary Pattern Analysis," Chapter 7 in Machine Vision: Theory Algorithms Practicalities, 2nd Edition, Academic Press, Inc.: San Diego, CA, pp. 171-191. |
Davies, E.R. (1997). "Ellipse Detection," Chapter 11 in Machine Vision: Theory Algorithms Practicalities, 2nd Edition, Academic Press, Inc.: San Diego, CA, pp. 271-290. |
Davies, E.R. (1997). "Image Acquisition," Chapter 23 in Machine Vision: Theory Algorithms Practicalities, 2nd Edition, Academic Press, Inc.: San Diego, CA, pp. 583-601. |
Davies, E.R. (Aug. 1987). "Lateral Histograms for Efficient Object Location: Speed Versus Ambiguity," Pattern Recognition Letters 6(3):189-198. |
Desai, A. et al. (1994). "On the Localization of Objects Using an FSR Pressure Pad Transducer," IEEE pp. 953-957. |
Desai, A.M. (Nov. 1994). "Interpretation of Tactile Data From an FSR Pressure Pad Transducer Using Image Processing Techniques," A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Applied Science, Simon Fraser University, Burnaby, British Columbia, Canada, 157 pages. |
Dvorak, J. (Apr. 1992). "Whatever Happened to the Gavilan Mobile Computer?," Computer Shopper 12(4):668, Dialog® search result, nine pages. |
Elo Touchsystems, Inc. (1993). SmartSet(TM) Touchscreen Controller Family Technical Reference Manual, Manual Version 1.0, 139 pages. |
Elo Touchsystems, Inc. (1993). SmartSet™ Touchscreen Controller Family Technical Reference Manual, Manual Version 1.0, 139 pages. |
Elographics, Inc. (1988). E271-140 AccuTouch® Serial Touchscreen Controller, User's Manual Version 2.0, 24 pages. |
Elographics, Inc. (1989). "All the Building Blocks You Need to Quickly Develop Touchscreen Applications," eight pages. |
Elographics, Inc. (1989). "Application Development Tools," four pages. |
Elographics, Inc. (1990). ELODEV(TM) Touchscreen Driver Program Version 1.4, Installation Guide and Programmer's Reference Manual, Manual Version 2.0c, 119 pages. |
Elographics, Inc. (1990). ELODEV™ Touchscreen Driver Program Version 1.4, Installation Guide and Programmer's Reference Manual, Manual Version 2.0c, 119 pages. |
Elographics, Inc. (1990). MonitorMouse(TM) and MonitorMouse for Windows(TM) 3.0, Version 1.2, User's Guide, Manual Version 1.2, 33 pages. |
Elographics, Inc. (1990). MonitorMouse™ and MonitorMouse for Windows™ 3.0, Version 1.2, User's Guide, Manual Version 1.2, 33 pages. |
Elographics, Inc. (1993). "In Touch with Excellence," seven pages. |
Elographics, Inc. (1993). TouchBack(TM) Version 1.2 Programmer's Reference Manual, Manual Version 2.0a, 70 pages. |
Elographics, Inc. (1993). TouchBack™ Version 1.2 Programmer's Reference Manual, Manual Version 2.0a, 70 pages. |
Everett, H.R. (1995). Sensors for Mobile Robots: Theory and Application, AK Peters, Ltd.: Wellesley, MA, pp. v-x (Table of Contents Only.). |
Fearing, R.S. (1988). "Tactile Sensing, Perception and Shape Interpretation," A Dissertation Submitted to the Department of Electrical Engineering and the Committee on Graduate Studies of Stanford University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy, presented Dec. 1987, 160 pages. |
Fearing, R.S. (Jun. 1990). "Tactile Sensing Mechanisms," Robotics Research 9(3):3-23. |
Final Office Action mailed Jan. 31, 2012, for U.S. Appl, No. 11/619,490, filed Jan. 3, 2007, 23 pages. |
Final Office Action mailed Mar. 24, 2011, for U.S. Appl. No. 11/619,490, filed Jan. 3, 2007, 17 pages. |
Gavilan Computer Corporation. (Jan. 23, 1984). Gavilan(TM) Mobile Computer: MS-DOS User's Guide, Part No: 620-0010-02, 276 pages. |
Gavilan Computer Corporation. (Jan. 23, 1984). Gavilan™ Mobile Computer: MS-DOS User's Guide, Part No: 620-0010-02, 276 pages. |
Geman, S. et al. (Nov. 1984). "Stochastic Relaxation, Gibbs Distributions, and The Bayesian Restoration of Images," IEEE Transactions on Pattern Analysis and Machine Intelligence 6(6):721-741. |
Ghani, N. (Oct. 1988). "Visual and Tactile Senses in Collaboration," The International Journal of Sensing for Industry pp. 210-215. |
Golub, G. et al. (1983). "Measuring Vectors, Matrices, Subspaces, and Linear System Sensitivty," Chapter 2 in Matrix Computations, Johns Hopkins University Press: Baltimore, MD, pp. 10-30. |
Goodisman, A. (May 10, 1991). "A Stylus-Based User Interface for Text: Entry and Editing," Thesis (B.S./M.S.) MIT Department of Electrical Engineering and Computer Science, pp. 1-100. |
Hackwood, S. et al. (1986). "Torque-Sensitive Tactile Array for Robotics," in Robot Sensors vol. 2-Tactile and Non-Vision, Pugh, A. ed., IFS Publications: UK, pp. 123-131. |
Harmon, L.D. (Summer 1982). "Automated Tactile Sensing," The International Journal of Robotics Research 1(2):3-32. |
Harrison, B.L. et al. (Apr. 18, 1998). "Squeeze Me, Hold Me, Tilt Me! An Exploration of Manipulative User Interfaces," CHI 98, Conference Proceedings, Los Angeles, CA, Apr. 18 -23, 1998, pp. 17-24. |
Holland, S. (Jul. 1989). "Artificial Intelligence, Education and Music: The Use of Artificial Intelligence to Encourage and Facilitate Music Composition by Novices," CITE Report No. 88, Centre for Information Technology in Education, 309 pages. |
Hurst, M.P. (Jul. 1995). "Configurator: A Configuration Program for Satellite Dish Hardware," Thesis (M.S.) MIT Department of Electrical Engineering and Computer Science, pp. 1-74. |
International Search Report mailed Jul. 10, 2008, for PCT Application No. PCT/US2007/026299, filed Dec. 21, 2007, four pages. |
Jayawant, B.V. et al. (1986). "Robot Tactile Sensing: a New Array Sensor," in Robot Sensors vol. 2-Tactile and Non-Vision, Pugh, A. ed., IFS Publications: UK, pp. 199-205. |
Kahn, R.E. et al. (Mar. 1988). "An Open Architecture for a Digital Library System and a Plan for Its Development," in The Digital Library Project: The World of Knowbots, Corporation for National Research Initiatives, 1:1-48. |
Kirk, D.E. (1970). "The Method of Steepest Descent," Chapter 6.2 in Optimal Control Theory: An Introduction, Prentice-Hall, Inc.: Englewood Cliffs, NJ, pp. 331-343. |
Kjeldsen, F.C.M. (1997). "Visual Interpretation of Hand Gestures as a Practical Interface Modality," Thesis (Ph.D.) Columbia University, Graduate School of Arts, pp. 1-198. |
Konrad, J. (Jun. 1989). "Bayesian Estimation of Motion Fields from Image Sequences," Thesis, McGill University, Department of Electrical Engineering, pp. 1-237. |
Krueger, M. et al. (Jun. 10, 1988). "Videoplace, Responsive Environment, 1972-1990," located at http://www.youtube.com/watch?v=dmmxVA5xhuo, last visited Aug. 5, 2011, two pages. |
Kurtenbach, G.P. (1983). "The Design and Evaluation of Marking Menus," A Thesis Submitted in Conformity with the Requirements of the Degree of Doctor of Philosophy, University of Toronto, Toronto, Canada, 201 pages. |
Lee, S. (Oct. 1984). "A Fast Multiple-Touch-Sensitive Input Device," a Thesis Submitted in Conformity with the Requirements for the Degree of Master of Applied Science in the Department of Electrical Engineering, University of Toronto, 115 pages. |
Lee, S.K. et al. (Apr. 1985). "A Multi-Touch Three Dimensional Touch-Sensitive Tablet," Proceedings of CHI: ACM Conference on Human Factors in Computing Systems, pp. 21-25. |
Leonard, J.J. et al. (Jun. 1991). "Mobile Robot Localization by Tracking Geometric Beacons," IEEE Transaction on Robotics and Automation 7(3):376-382. |
Leung, A. et al. (1994). "Application of Adaptive Neural Network to Localization of Objects Using Pressure Array Transducer," IEEE pp. 2114-2119. |
Liu, H. et al. (Jun. 1993). "Development of a New Kind of Robot Slipping Sensor," Proceedings of the 1993 American Control Conference, Jun. 2-4, 1993, San Francisco, CA, pp. 756-757. |
Logitech, Fremont, Inc. (Aug. 2, 1998). P2.1 Touchpad, Firmware Engineering Specification FWS-761588-0000, Revision A, 25 pages. |
Logitech, Inc. (Jun. 24, 1997). TP3(TM)-Standard Touch Pad Product Specification, Document ID: PS-761311-0000-USA, 27 pages. |
Logitech, Inc. (Jun. 24, 1997). TP3™-Standard Touch Pad Product Specification, Document ID: PS-761311-0000-USA, 27 pages. |
Mehta, N. (Oct. 1982). "A Flexible Human Macine Interface," A Thesis Submitted in Conformity with the Requirements for the Degree of Master of Applied Science in the University of Toronto, 87 pages. |
Microtouch Sysyems, Inc. (1993). TouchWare for DOS, Windows and NT, User's Guide, Revision 3.0 #19-207, 33 pages. |
Microtouch Sysyems, Inc. (1993). Windows Touch Driver, User's Guide, Revision 2.0 #9006601, 26 pages. |
Miller, D. et al. (Feb./Mar. 1994). "Combined Source-Channel Vector Quantization Using Deterministic Annealing,"IEEE Transactions on Communications 42 (2/3/4):347-356. |
Moroney, D.T. (Mar. 1992). "Use of an Optical Multichannel Analyzer for Reflectivity Measurements," Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Science in Astronautical Engineering, Naval Postgraduate School, Monterrey, CA, 187 pages. |
Mui, L. (Aug. 22, 1995). "A Statistical Multi-Experts Approach to Image Classification and Segmentation," Thesis (B.S./M.Eng.) MIT Department of Electrical Engineering and Computer Science, pp. 1-177 (submitted in two parts). |
Nicholls, H.R. et al. (Jun. 1989). "A Survey of Robot Tactile Sensing Technology," The International Journal of Robotics Research 8(3):3-30. |
Non-Final Office Action mailed Nov. 22, 2010, for U.S. Appl. No. 11/619,490, filed Jan. 3, 2007, 17 pages. |
Non-Final Office Action mailed Oct. 1, 2009, for U.S. Appl. No. 11/619,490, filed Jan. 3, 2007, 35 pages. |
Non-Final Office Action mailed Sep. 29, 2011, for U.S. Appl. No. 11/619,490, filed Jan. 3, 2007, 22 pages. |
Notice of Allowance mailed May 15, 2012, for U.S. Appl. No. 11/619,490, filed Jan. 3, 2007, 12 pages. |
Nowlin, W.C. (Apr. 1991). "Experimental Results on Bayesian Algorithms for Interpreting Compliant Tactile Sensing Data," Proceedings of the 1991 IEEE International Conference on Robotics and Automation, Sacramento, CA, pp. 378-383. |
Ogawa, H. (1979). "Preprocessing for Chinese Character Recognition and Global Classification of Handwritten Chinese Characters," Pattern Recognition, 11(1):1-7. |
Poggio, T. et al. (1990). "The MIT Vision Machine," Chapter 42 in Artificial Intelligence, MIT Press: Cambridge, MA, pp. 493-529. |
Pugh, A. ed. (1986). International Trends in Manufacturing Technology: Robot Sensors vol. 2-Tactile and Non-Vision, 3 pages (Table of Contents Only). |
Quantum Research Group Ltd. (1997). QT9701B2 Datasheet, 30 pages. |
Rebman, J. et al. (1986). "A Tactile Sensor with Electrooptical Transduction," in Robot Sensors vol. 2-Tactile and Non-Vision, Pugh, A. ed., IFS Publications: UK, pp. 145-155. |
Robertson, B.E. et al. (1986). "Tactile Sensor System for Robotics," in International Trends in Manufacturing Technology: Robot Sensors vol. 2-Tactile and Non-Vision, Pugh, A. ed., IFS Publications: UK, pp. 89-97. |
Rubine, D. (Jul. 1991). "Specifying Gestures by Example," Computer Graphics 25(4):329-337. |
Rubine, D. (Sep. 1988). "The VideoHarp," Proceedings of the 14th International Computer Music Conference, Cologne, FR, Sep. 20-25, 1988, 11 pages. |
Rubine, D. et al. (1990) "Programmable Finger-Tracking Instrument Controllers," Computer Music Journal 14(1):26-41. |
Rubine, D.H. (Dec. 1991). "The Automatic Recognition of Gestures," CMU-CS-91-202, Submitted in Partial Fulfillment of the Requirements of the Degree of Doctor of Philosophy in Computer Science at Carnegie Mellon University, 285 pages. |
Rubine, D.H. (May 1992). "Combining Gestures and Direct Manipulation," CHI '92, pp. 659-660. |
Rusaw, S. et al. (May 1998). "Determining Polygon Orientation Using Model Based Force Interpretation," Proceedings of the 1998 IEEE International Conference on Robotics and Automation, Leuven, Belgium, pp. 544-549. |
Sato, N. et al. (1986). "A Method for Three-Dimensional Part Identification by Tactile Transducer," in Robot Sensors vol. 2-Tactile and Non-Vision, Pugh, A. ed., IFS Publications: UK, pp. 133-143. |
Sears, A. et al. (Jan. 23, 1989). "High Precision Touchscreens: Design Strategies and Comparisons With a Mouse," Human-Computer Interaction Laboratory, University of MD, pp. 1-23. |
Sensor Frame Corporation. (May 8, 1990). "The Sensor Frame Graphic Manipulator Final Report," Sensor Frame Corporation: NASA-CR-194243, 28 pages. |
Sethi, M. (1997). "Generating Clouds in Computer Graphics," A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of M.Sc. in Computer Science, 154 pgs. |
Siegel, D.M. et al. (1987). "Performance Analysis of a Tactile Sensor," IEEE, pp. 1493-1499. |
Son, J.S. et al. (1996). "Comparison of Contact Sensor Localization Abilities During Manipulation," Robotics and Autonomous System 17 pp. 217-233. |
Speeter, T.H. (Nov. 1988). "Flexible, Piezoresitive Touch Sensing Array," Optics Illumination and Imaging Sensing for Machine Vision III, SPIE 1005:31-43. |
Stansfield, S.A. (Mar. 1990). "Haptic Perception With an Articulated, Sensate Robot Hand," SANDIA Report: SAND90-0085-UC-406, 37 pages. |
Stap, J.W. (Jun. 29, 1992). "Automatic 3-D Measurement of Workpieces on a Ship Repair Yard," M.Sc. Thesis, 132 pages. |
Stauffer, R.N., ed. (Jun. 1983). "Progress in Tactile Sensor Development," Robotics Today pp. 43-49. |
Sugiyama, S. et al. (Mar. 1990). "Tactile Image Detection Using a 1K-Element Silicon Pressure Sensor Array," Sensors and Actuators, A22(Nos. 1-2):397-400. |
Summagraphics Corporation. (1987). Four-Button Cursor for the Summagraphics® Bit Pad® Plus, one page. |
Summagraphics Corporation. (Sep. 1989). CR™ 1212 Graphics Tablet Technical Reference, Publication #84-2001-002, 53 pages. |
Synaptics + Pilotfish (Nov. 25, 2006). "Onyx," Internet Citation, three pages. |
Synaptics, Inc. (1999). The TouchPad: A Revolutionary Human Interface Device, four pages. |
Van Brussel, H. et al. (Jun. 1986). "A High Resolution Tactile Sensor for Part Recognition," Proceedings of the 6th International Conference on Robot Vision and Sensory Controls, Paris, France, Jun. 3-5, 1986, pp. 49-59. |
Velasco, V.B. Jr. (Aug. 1997). "A Methodology for Computer-Assisted Gripper Customization Using Rapid Prototyping Technology," Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Ph.D., 199 pages. |
Want, R. et al. (Mar. 1995). "The ParcTab Ubiquitous Computing Experiment," Technical Report CSL-95-1, Xerox, 44 pages. |
Westerman, W. (Spring 1999). "Hand Tracking, Finger Identification, and Chordic Manipulation on a Multi-Touch Surface," A Dissertation Submitted to the Faculty of the University of Delaware in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Electrical Engineering, 364 pages. |
Wolfeld, J.A. (Aug. 1981). "Real Time Control of a Robot Tactile Sensor," Thesis Presented in Partial Fulfillment of the Requirements for the Degree of Master of Science in Engineering for Graduate Work in Computer and Information Science, 68 pages. |
Yaniger, S.L. (Apr. 1991). "Force Sensing Resistors: A Review of the Technology," Elector International Conference Record, Apr. 16-18, 1991, New York, NY pp. 666-668. |
Yao, M. Y-S et al. (Mar. 1, 2002). "Quantifying Quality: A Case Study in Fingerprints," Proceedings IEEE Conference on AudioID, IEEE, New York, NY, six pages. |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160349910A1 (en) * | 2007-01-03 | 2016-12-01 | Apple Inc. | Irregular input identification |
US10025429B2 (en) * | 2007-01-03 | 2018-07-17 | Apple Inc. | Irregular input identification |
US9691019B1 (en) * | 2017-03-07 | 2017-06-27 | Google Inc. | Depth concatenation using a matrix computation unit |
US10699182B2 (en) | 2017-03-07 | 2020-06-30 | Google Llc | Depth concatenation using a matrix computation unit |
US10896367B2 (en) | 2017-03-07 | 2021-01-19 | Google Llc | Depth concatenation using a matrix computation unit |
US11954062B2 (en) | 2019-03-15 | 2024-04-09 | Intel Corporation | Dynamic memory reconfiguration |
Also Published As
Publication number | Publication date |
---|---|
WO2008085458A3 (en) | 2008-09-18 |
US20080158146A1 (en) | 2008-07-03 |
US20120306800A1 (en) | 2012-12-06 |
US10025429B2 (en) | 2018-07-17 |
US8269727B2 (en) | 2012-09-18 |
US20160349910A1 (en) | 2016-12-01 |
WO2008085458A2 (en) | 2008-07-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10025429B2 (en) | Irregular input identification | |
US9778807B2 (en) | Multi-touch input discrimination | |
US7643010B2 (en) | Peripheral pixel noise reduction | |
US8542210B2 (en) | Multi-touch input discrimination | |
US7876310B2 (en) | Far-field input identification |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |