WO2012144984A1 - Touch screen selection - Google Patents

Touch screen selection Download PDF

Info

Publication number
WO2012144984A1
WO2012144984A1 PCT/US2011/032968 US2011032968W WO2012144984A1 WO 2012144984 A1 WO2012144984 A1 WO 2012144984A1 US 2011032968 W US2011032968 W US 2011032968W WO 2012144984 A1 WO2012144984 A1 WO 2012144984A1
Authority
WO
WIPO (PCT)
Prior art keywords
touch
selection
input
area
elements
Prior art date
Application number
PCT/US2011/032968
Other languages
French (fr)
Inventor
Jerome Huck
Original Assignee
Hewlett-Packard Development Company, L.P.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hewlett-Packard Development Company, L.P. filed Critical Hewlett-Packard Development Company, L.P.
Priority to CN201180070231.4A priority Critical patent/CN103492981A/en
Priority to US14/112,063 priority patent/US9519369B2/en
Priority to PCT/US2011/032968 priority patent/WO2012144984A1/en
Priority to EP11863932.7A priority patent/EP2699986B1/en
Publication of WO2012144984A1 publication Critical patent/WO2012144984A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842Selection of displayed objects or displayed text elements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/0412Digitisers structurally integrated in a display
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048Indexing scheme relating to G06F3/048
    • G06F2203/04805Virtual magnifying lens, i.e. window or frame movable on top of displayed information to enlarge it for better reading or selection

Definitions

  • FIG. 1 is a block diagram of a computing device utilizing a user interface for touch screen selection, according to one example
  • FIGs. 2A and 2B are block diagrams of devices for providing a touch screen user interface, according to various examples
  • FIG. 3 is a flowchart of a method for selecting user interface elements, according to one example
  • FIG. 4 is a flowchart of a method for performing enhanced tasks via magnified user interface area, according to one example
  • FIG. 5 is a flowchart of a method for determining a magnification level of an area associated with a touch selection input, according to one example
  • FIG. 6 is a flowchart of a method for incrementally magnifying a user interface area to determine a touch selection input, according to one example
  • FIGs. 7A - 7F are block diagrams of example user interfaces for touch screen selection via auto magnification, according to various examples.
  • FIGs. 8A - 8C are block diagrams of example user interfaces for touch screen selection via auto magnification, according to various examples.
  • Touch screen devices are growing as a mechanism to provide content to users.
  • the number of individuals utilizing touch screen devices, such as smart phones and tablet computing devices is continually growing.
  • Various user interfaces are utilized to provide content via these touch screen devices.
  • many user interfaces utilized on such devices may be inadequate to provide a complete user experience.
  • a user's finger is a common input for touch screen devices; however a finger is sometimes too large to unambiguously select a user interface element presented on a touch screen display.
  • Touch screens in mobile phones and other devices were created to be portable, with smaller screens. As such, the touch screens on these devices can be too small to recognize an intended selection of a user interface element.
  • touch screen selection of one or more selection elements such as soft keypads, links, text boxes, selection boxes, content creation elements, etc. can be a problem when a finger touch spans multiple elements or are close to other elements, which makes the device unsure about which selection element was intended.
  • a further challenge in touch screen selection is to provide an enhanced selection user interface that is intuitive for a user.
  • various embodiments disclosed herein relate to enabling touch screen selection of interface elements via auto magnification.
  • the auto magnification can be utilized when an ambiguity in a touch selection of interface elements is determined.
  • the device then magnifies an area associated with a touch location based on the determination that there is ambiguity in what the selection refers to.
  • Auto magnification can be utilized to provide a seamless user experience to a user. Magnifying a particular area directly presents the same user interface to the user. As such, the user can easily use the magnified area in the same way as the previously presented user interface. Thus, the user may be able to quickly determine that the same touch mechanism can be used to select the element presented on the screen. The user can then touch the magnified area of the screen to select the element. When the element is selected, a task based on the selection can be performed.
  • Tasks can vary based on a type of selection element.
  • a task for a hyperlink can be to follow the link in a web browser.
  • a task for a list selection element can be to expand a list associated with the list selection element.
  • the auto magnification can be set to automatically de-magnify the area to return the user interface to the previous view before expanding the list.
  • FIG. 1 is a block diagram of a computing device utilizing a user interface for touch screen selection, according to one example.
  • the computing device 100 includes, for example, a processor 1 10, and a machine-readable storage medium 120 including instructions 122, 124, 126, 128, 130 for touch screen selection based on automatic magnification.
  • Computing device 100 may be, for example, a notebook computer, a slate or tablet computing device, a portable reading device, a wireless email device, a mobile phone, or any other computing device with a touch-enabled screen.
  • Processor 1 10 may be, at least one central processing unit (CPU), at least one semiconductor-based microprocessor, at least one graphics processing unit (GPU), other hardware devices suitable for retrieval and execution of instructions stored in machine-readable storage medium 120, or combinations thereof.
  • the processor 1 10 may include multiple cores on a chip, include multiple cores across multiple chips, multiple cores across multiple devices (e.g., if the computing device 100 includes multiple node devices), or combinations thereof.
  • Processor 1 10 may fetch, decode, and execute instructions 122 - 130 to implement touch selection based on auto magnification, for example via the processes of FIGs. 3 - 6.
  • processor 1 10 may include at least one integrated circuit (IC), other control logic, other electronic circuits, or combinations thereof that include a number of electronic components for performing the functionality of instructions 122 - 130.
  • IC integrated circuit
  • Machine-readable storage medium 120 may be any electronic, magnetic, optical, or other physical storage device that contains or stores executable instructions.
  • machine-readable storage medium may be, for example, Random Access Memory (RAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a storage drive, a Compact Disc Read Only Memory (CD-ROM), and the like.
  • RAM Random Access Memory
  • EEPROM Electrically Erasable Programmable Read-Only Memory
  • CD-ROM Compact Disc Read Only Memory
  • the machine- readable storage medium 120 can be non-transitory.
  • machine-readable storage medium 120 may be encoded with a series of executable instructions for selecting touch elements on a touch-enabled display.
  • a touch element is a portion of a presented user interface on a touch-enabled display that is activated when a touch input is determined to be associated portion.
  • touch elements include links (e.g., web links to other web pages, links embedded in icons or images, etc.), menu items (e.g., menu bars, file menus, toolbars, etc.), fields (e.g., a search input area, a text area, etc.), and movable user interface components (e.g., a window, a pointer, a drawing tool, etc.).
  • links e.g., web links to other web pages, links embedded in icons or images, etc.
  • menu items e.g., menu bars, file menus, toolbars, etc.
  • fields e.g., a search input area, a text area, etc.
  • movable user interface components e.g., a window, a pointer, a drawing tool, etc.
  • the computing device 100 can utilize a touch-enabled display to present graphical information.
  • a touch-enabled display is a visual display that can detect the presence and location of a touch with an area of the display.
  • Various technologies can be utilized to implement the display, for example, resistive technology, capacitive technology, infrared technology, etc.
  • the processor 1 10 can cause presentation of graphical information by transmitting the information to the display.
  • the computing device 100 can provide user interfaces on the touch- enabled display by executing programs (e.g., an operating system, an application, etc.).
  • display instructions 122 can be executed to cause display of a plurality of touch elements on the touch-enabled display.
  • the touch elements may be presented on the touch-enabled display as part of a user interface being presented by one or more of the programs.
  • the touch elements can correspond to portions of the display that can be activated. Some touch elements may be displayed close together.
  • An input can be received by the processor 1 10 indicating that a portion of the screen has been touched.
  • the portion can be considered a blot area.
  • a blot area is the part of the touch-enabled display that is activated by the touch.
  • the input instructions 124 are executed by the processor 1 10 to determine an action to take based on the input.
  • the input can be received as a geometric set of points or a description of an area of the display touched.
  • the processor 1 10 can map the blot area to locations of touch elements being displayed. As such, the processor 1 10 can determine if the blot area corresponds to one or more of the touch elements. In certain cases, this determination is unambiguous (e.g., only one touch element is in the blot area). In other cases, the determination may be ambiguous.
  • the processor 1 10 can execute ambiguity instructions 126 to determine whether the touch input is ambiguous in selecting one of the touch elements. As such, the processor 1 10 determines what touch elements are nearby the blot area. If no touch element or only one touch element is within the blot area, the input is unambiguous. If multiple touch elements can be associated with the blot area, the processor 1 10 determines a confidence level of how likely the touch is to correspond to a particular touch element. Different touch elements can have different criteria to determine the confidence level.
  • a first touch element is completely encompassed by the blot area while the other touch elements are minimally overlapping with the blot area.
  • the processor 1 10 can determine that the first touch element is selected unambiguously.
  • a first touch element and a second touch element are completely or partially encompassed by the blot area.
  • the processor 1 10 determines that there is an ambiguity as to which touch element was selected. Such ambiguity can be determined based on more than two touch elements. As such, the processor 1 10 determines that the blot area is ambiguous in determining one of the touch elements. Further, the processor 1 10 may determine that a set of the touch elements, which can include the first touch element and the second touch element, may be selected based on the blot area. For example, the set can include touch elements that have a certain confidence level that they may be selected by the touch input.
  • zoom instructions 128 can be utilized to determine to zoom in onto a region of interest surrounding the blot area. This can occur automatically based on the ambiguity determination.
  • the zooming can fill the entire display or can fill a portion of the display (e.g., as a callout). Further, the zoomed in region can maintain scaled spacing of the region.
  • the zooming in can be a magnification by a particular scale (e.g., a 2X zoom, a 1 .5X zoom, a 4X zoom, etc.) and retain scaled spatial relationships during the zoom.
  • the maintenance of the spatial relationships and the zooming in of the region provides a large surface for the user to select the touch element. Further, the zoomed in region provides an intuitive user interface to the user because the user need not learn a new manner to interact with the region. That is because the user interface of the region interacts in the same manner as the non-zoomed in user interface.
  • the scale of the zooming can be determined based on spacing information between touch elements, the size of the touch elements, the size of the blot, or combinations thereof.
  • the scale can be based on a function utilized to zoom in to a level where the blot area can unambiguously select one of the touch elements for which the ambiguity was detected.
  • the zooming can be executed in a manner in which the size of one of the touch elements (e.g., the smallest touch element, the largest touch element, etc.) is at least a certain percentage of the size of the blot.
  • the zoom instructions 128 can cause the processor 1 10 to examine the nearby touch elements to attempt to spread them apart in the zoomed in image to allow a subsequent user touch (creating a new blot) to uniquely specify the intended touch element.
  • the zooming can be based on a user profile.
  • the user profile can determine individual parameters that can be utilized in determining the zoom level.
  • the user profile can include information about blot sizes generated by the user, which may include blot sizes for one or more fingers used by the user. With this approach, a processed blot size associated with the user and/or other tendencies of the user can be used to determine the zoom level.
  • the computing device 100 may receive another touch input to select a touch element via the touch-enabled display.
  • the processor 1 10 can determine another blot area associated with the other touch input.
  • Selection instructions 130 can then be utilized to determine one of the touch elements based on the other blot area.
  • the other touch input can also go through the ambiguity determination. In this manner, if the other touch input is determined to be ambiguous, a portion of the region can be zoomed in on.
  • FIGs. 2A and 2B are block diagrams of devices for providing a touch screen user interface, according to various examples.
  • Devices 200a, 200b include modules that can be utilized to magnify a region of interest to select touch elements.
  • the respective devices 200a, 200b may be a notebook computer, a tablet computing device, a portable reading device, a wireless device, a mobile phone, a server or any other device that may be utilized to present touch screen user interface elements.
  • the devices 200a, 200b can include a touch-enabled display, such as a touch screen 210 to present information, such as multiple touch elements, to a user.
  • the touch screen 210 can be connected via one or more interfaces (e.g., via a cable or a wireless medium).
  • a processor such as a CPU, a GPU, or a microprocessor suitable for retrieval and execution of instructions and/or electronic circuits can be configured to perform the functionality of any of the modules 212 - 218 described below.
  • the devices 200a, 200b can include some of the modules (e.g., modules 212 - 218), the touch screen 210, and/or additional components. Further, in certain embodiments, some modules can be utilized to implement functionality of other modules described below.
  • devices 200a, 200b may include a series of modules 212 - 218 for utilizing a touch screen user interface during use of one or more applications executing on the devices 200a, 200b.
  • Each of the modules 212— 218 may include, for example, hardware devices including electronic circuitry for implementing the functionality described below.
  • each module may be implemented as a series of instructions encoded on a machine-readable storage medium of respective devices 200a, 200b and executable by a processor. It should be noted that, in some embodiments, some modules 212 - 218 are implemented as hardware devices, while other modules are implemented as executable instructions.
  • the touch screen 210 can be utilized to present interfaces to users. These interfaces can be part of a program (e.g., an operating system, an application, a game, etc.) executed by a processor 230 and stored in memory 232 of the device 200b.
  • the touch screen 210 can further be utilized as an input source.
  • the input detection module 212 can determine touch input from the touch screen 210.
  • the input detection module 212 can further utilize a coordinate system to associate a touch input via the touch screen 210 with a portion of the screen.
  • the portion of the screen can be considered a blot area.
  • the touch input can be associated with one or more points of the screen.
  • a selection module 214 attempts to determine what the touch input is attempting to activate and/or utilize.
  • the touch elements can be options for what can be activated by the touch input.
  • the selection module 214 and/or the ambiguity module 216 can determine whether the touch input is ambiguous in selecting one of the touch elements.
  • the touch input can be considered ambiguous if more than one of the touch elements could be selected. Thus, if ambiguous, the touch input can be considered to be open to being associated with more than one of the touch elements. If the input is unambiguous, the selection module 214 can select the associated touch element.
  • the magnification module 218 is utilized to magnify a region of interest of the touch screen 210.
  • the region may include the touch elements. These touch elements can be the touch elements that may be associated with the touch input. Further, the magnified region can maintain a scaled spatial orientation of the region.
  • the magnification module 218 can zoom in on the region by a particular scale (e.g., a predefined scale, a scale dynamically determined, etc.).
  • a dynamic scale can be determined based on spacing information between touch elements, the size of the touch elements, the size of a blot associated with the touch input, or combinations thereof.
  • the magnification can be based on a user profile.
  • the magnification module determines a scaling of the magnification based on a blot area and sizing information about at least two of the touch elements.
  • the ambiguity module 216 determines a blot area associated with the touch input. Further, the ambiguity module 216 can determine that the blot area covers at least two of the touch elements. The ambiguity module 216 can then determine sizing information about the at least two touch elements. This sizing information and the information describing the blot area can be utilized to dynamically determine the scale.
  • the user may utilize that area as a user interface. As such, the user can select the intended touch element.
  • the input detection module 212 detects another touch input via the touch screen 210 in the magnified area. This input is then provided to the selection module 214 and/or ambiguity module 216.
  • the ambiguity module 216 can process the information and determine whether the other touch input is ambiguous. If the other touch input is determined to be ambiguous, the magnification module 218 can be utilized to further magnify another region of interest in the magnified region. If the other touch input is determined to be unambiguous, the selection module 214 selects a touch element based on the other touch input.
  • the original touch input and the other touch input can be a direct touch of at least one of the touch elements.
  • the user may utilize the same approach of selecting the selected touch element. This approach is simple and intuitive.
  • the magnification module 218 can de-magnify the region and a task associated with the selection can be performed.
  • an input device 240 can include a sensor, a keyboard, a mouse, a remote, a keypad, a microphone, or the like. Sensors can be used to implement various technologies, such as infrared technology, camera technology, touch screen technology, etc. Further, the device 200b may include devices utilized for input and output (not shown), such as a networking interface (e.g., Ethernet), a Universal Serial Bus (USB) connection, etc.
  • a networking interface e.g., Ethernet
  • USB Universal Serial Bus
  • the input/output interfaces 234 can additionally be utilized to drive output to an output device 242.
  • One of the modules 212 - 218, the processor 230, or another component of the device 200b can be utilized to send a signal via the input/output interfaces 234 to output devices 242. Examples of output devices include speakers, other display devices, amplifiers, etc.
  • the ambiguity module 216 determines a blot area associated with the original touch input.
  • the touch elements in this scenario can be associated with a drawing tool that includes a pointer that is smaller than the blot area. This may be utilized in a painting tool or pencil tool used by the user.
  • a user profile, an application profile associated with the drawing tool, or other information can be processed to determine a magnification level to magnify a region of interest based on the original touch input.
  • the magnification module 218 magnifies the region to a level where the pointer is within a threshold ratio of the blot area.
  • FIG. 3 is a flowchart of a method for selecting user interface elements, according to one example. Although execution of method 300 is described below with reference to device 200b, other suitable components for execution of method 300 can be utilized (e.g., computing device 100, device 200a, etc.). Additionally, the components for executing the method 300 may be spread among multiple devices. Method 300 may be implemented in the form of executable instructions stored on a machine-readable storage medium, such as storage medium 120, and/or in the form of electronic circuitry.
  • a machine-readable storage medium such as storage medium 120
  • Method 300 may start at 302 and proceed to 304, where device 200 may display a plurality of selection elements on a touch-enabled display such as the touch screen 210.
  • a selection element is a user interface element that may be chosen or activated by an input.
  • a selection element can be a link, a text input box, a soft key, a list, menu items on a tool bar, drawing space to be utilized by a drawing tool, a content box, a search box, or the like.
  • Example user interfaces and selection elements are provided in FIGs. 7A - 7F and FIGs. 8A - 8C.
  • a user may then touch the touch-enabled display to select one of the selection elements.
  • the input detection module 212 receives a touch selection input via the touch-enabled display (306).
  • the input detection module 212 can then determine where the touch selection input is located, for example, by associating one or more coordinates with the touch selection input.
  • the ambiguity module 216 determines whether the touch selection is ambiguous.
  • the touch selection input can be considered ambiguous if the touch selection input is not clear as to selecting a selection element. As noted above, this can occur if it is determined that more than one selection element may be selected by the input. If there is no ambiguity and only one selection element is chosen, the element can be selected and tasks can be processed based on the selection.
  • the method 300 proceeds to 310, where the magnification module 218 magnifies an area associated with the touch selection input.
  • Magnification can be considered a direct magnification and equal scaling of the image presented at the display.
  • the area can include, at least in part, the selection elements.
  • the area includes a set of the selection elements associated with the ambiguity determination, that is, the selection elements that may be selected by the touch input. In certain scenarios, the set may be considered the selection elements that may be associated with the input, which cause the ambiguity.
  • the magnification of the area can maintain the spatial orientation of the area. As such, the scaling during the magnification is equal for the spaces between selection elements as well as the magnification of the selection elements.
  • the magnification of the area can further be presented on a full screen area of the touch-enabled display or a portion of the display. For example, a portion of the display can be viewed as a callout from the location of the touch selection input.
  • the input detection module 212 receives another touch selection input at the area via the touch-enabled display.
  • the input detection module 212 determines where the other input is located, for example, by associating one or more coordinates with the other input.
  • the original touch selection input and the other touch selection input can correspond to a direct touch of at least one of the selection elements.
  • the selection module 214 selects one of the selection elements based on the other touch input to the magnified area.
  • the ambiguity module 216 can determine whether the selection of the one selection element is ambiguous. If the selection is ambiguous, the processes of FIG. 6 can be utilized to generate an unambiguous selection. If the selection is unambiguous, a task associated with the selection element can be performed. Then, at 316, the method 300 stops.
  • FIG. 4 is a flowchart of a method for performing enhanced tasks via magnified user interface area, according to one example.
  • execution of method 400 is described below with reference to device 200b, other suitable components for execution of method 400 can be utilized (e.g., computing device 100, device 200a, etc.). Additionally, the components for executing the method 400 may be spread among multiple devices. Moreover, method 400 may be implemented in the form of executable instructions stored on a machine-readable storage medium, such as storage medium 120, and/or in the form of electronic circuitry.
  • Method 400 may begin after a magnified area is presented to a user and the device 200b receives another input at the magnified area.
  • Method 400 may start at 402 and proceed to 404, where device 200b determines that a selection of a selection element is unambiguous. The determination can be made using various methods, for example, determining a confidence level that the input is associated with the selection element. If the determined confidence level meets a predetermined threshold confidence level, the selection element can be determined to be unambiguous. In other scenarios, other criteria can be utilized in determining the ambiguity of the selection.
  • an enhanced task is performed based on the selection 406.
  • the enhanced task is to perform an action associated with the selection element and de-magnifying the magnified area to the area.
  • the de- magnification can occur before or after the task is completed.
  • activation of a list can be a task to perform based on the selection of the magnified selection element. Instead of performing the opening of the list in the magnified area, the area can be de-magnified and then the opened list can be presented. In this manner, the user can view the opened list in the scaling that the user was originally viewing.
  • An example of opening a list is described below in the discussion of FIG. 7E.
  • the task can be to execute a link to a webpage.
  • the enhanced task can include saving a de-magnified version of the original user interface to a memory and then executing the link.
  • the link can be presented without magnification. Further, if the user utilizes a back selection element to move back to the original user interface, the saved de-magnified version of the user original user interface can be presented. Then, at 408, the method 400 stops.
  • FIG. 5 is a flowchart of a method for determining a magnification level of an area associated with a touch selection input, according to one example.
  • execution of method 500 is described below with reference to computing device 100, other suitable components for execution of method 500 can be utilized (e.g., device 200a, device 200b, etc.). Additionally, the components for executing the method 500 may be spread among multiple devices.
  • Method 500 may be implemented in the form of executable instructions stored on a machine-readable storage medium, such as storage medium 120, and/or in the form of electronic circuitry.
  • Method 500 can occur after a touch input is received and determined to be ambiguous.
  • Method 500 may start at 502 and proceed to 504, where computing device 100 may determine a portion of the touch-enabled display activated by the touch selection input.
  • the portion can be, for example, a blot area. Further, the portion can be associated with coordinates of the display.
  • the computing device 100 can determine information about selection elements corresponding to the portion.
  • these selection elements can be a set of elements that may be determined to cause the ambiguity because more than one of the selection elements may be the desired target of the user.
  • the information can include sizing information about the selection elements. Further, the information can include spacing information associated with the selection elements.
  • a size of the portion can be determined. The size of the portion and/or selection elements can be associated with a length, width, radius, area, or the like associated with the elements.
  • the computing device 100 can determine a magnification level based on the size of the portion, the sizing information, the spacing information, or a combination thereof.
  • the magnification can be to a level such that the size of the portion (e.g., blot area) is at least a certain percentage of the size of one or more particular elements.
  • the size of the one or more particular elements as well as the space size between elements can be compared to the portion to determine the magnification level.
  • the magnification level can be optimized in such a manner that the size of the particular elements and/or the size of the space between elements can be used so that another input the size of the portion can be used to uniquely activate one of the elements.
  • blot area size information used in the calculation can be determined from a user profile or other collected information.
  • the method 500 then proceeds to 512, where the method 500 stops.
  • FIG. 6 is a flowchart of a method for incrementally magnifying a user interface area to determine a touch selection input, according to one example.
  • execution of method 600 is described below with reference to device 200b, other suitable components for execution of method 600 can be utilized (e.g., device 200a, computing device 100, etc.). Additionally, the components for executing the method 600 may be spread among multiple devices.
  • Method 600 may be implemented in the form of executable instructions stored on a machine- readable storage medium, such as storage medium 120, and/or in the form of electronic circuitry.
  • Method 600 may start in 602, where device 200b can be at a state (e.g., at 310 or 312 of method 300) in which a user interface is magnified to an area in response to a received ambiguous touch selection input. Another touch selection input is received to select one of a plurality of selection elements presented in the area. The method 600 can then proceed to 604, where device 200b is caused to determine that the other touch selection input is ambiguous in selecting the one selection element.
  • a state e.g., at 310 or 312 of method 300
  • Another touch selection input is received to select one of a plurality of selection elements presented in the area.
  • the method 600 can then proceed to 604, where device 200b is caused to determine that the other touch selection input is ambiguous in selecting the one selection element.
  • the magnification module 218 further magnifies the user interface to a portion of the magnified area based on the other touch selection input received.
  • the portion can be a region nearby a blot area caused by the other touch selection input.
  • the portion can include the touch selection elements that were the cause of the ambiguity determination.
  • the portion can include the one selection element and other selection elements that are within range of the touch selection input.
  • the inclusion of the selection elements can include partial selection elements.
  • to include a selection element in the magnified portion means to include at least a selectable part of the selection element. The entire selection element may also be included.
  • the user can view the magnified portion to select the one selection element.
  • the device 200b receives a third touch selection input (at 608). This input can be via the touch screen 210 and be at the magnified portion.
  • the selection module 214 can select the one selection element by associating the third touch selection input with the one selection element (at 610). As such, the selection of the one touch selection element can be based, at least in part, on the third touch selection input.
  • the method 600 can repeat anytime a touch selection input is deemed to be ambiguous. Then, at 612, the method 600 stops.
  • FIGs. 7A - 7F are block diagrams of example user interfaces for touch screen selection via auto magnification, according to various examples.
  • the user interfaces can be associated with a device.
  • User interface 700 shows multiple touch elements 702a, 702b, 702c, 704 as well as other user interface components. For example, picture 706 may be presented, but not be selectable.
  • a user of the device can attempt to select, via a touch-enabled display, one of the touch elements.
  • the touch is represented by a blot area 718.
  • the blot area looks like a circle in FIG. 7B, however, the blot area 718 can be represented by one or more points, a shape like the circle, or another representation. Further, the blot area need not be presented to a user. Because the blot area 718 could be associated with Item A 702a or Item B 702b, the blot area 718 can be considered ambiguous.
  • a magnification module of the device magnifies an area associated with the selection to allow for an unambiguous selection.
  • User interface 720 of FIG. 7C shows a magnified region of interest based on the ambiguity decision and the blot area 718.
  • Item A 722a, Item B 722b, and Item C 722c are displayed in the magnified region. The user can then enter another touch input to select one of the items unambiguously.
  • User interface 730 of FIG. 7D shows a blot area 738 associated with the touch input. The blot area 738 unambiguously selects Item B 722b. The selection of Item B 722b triggers an activation of a task associated with Item B 722b.
  • the user interface 740 of FIG. 7E shows Items A - C 742a - 742c as well as icon 744 and Items B1 746a and B2 746b.
  • the user interface no longer includes the magnification and can automatically go back to the previous state before the activation of the task.
  • the task includes the presentation of Items B1 746a and B2 746b.
  • the task can be performed simultaneously with the demagnification and/or before the demagnification.
  • FIG. 7F includes user interface 750 which shows the Items 742a - 742c, 746a, 746b as well as the icon 744.
  • another touch input is detected.
  • Another blot area 748 is determined in associated with the touch input.
  • the device can determine that the selection of touch elements is ambiguous.
  • the icon 744 is not a touch element. As such, a magnification of the touch elements can focus on Item B 742b and Item B1 746a.
  • the icon 744 is a touch element. In this example, the icon 744 can be a focus of the magnified area.
  • FIGs. 8A - 8C are block diagrams of example user interfaces for touch screen selection via auto magnification, according to various examples.
  • User interface 800 of FIG. 8A shows a drawing application including a toolbar on the left.
  • the toolbar can include a pencil tool 802, a rectangle tool 804, an ellipse tool 806, and an eraser tool 808.
  • the pencil tool 802 is selected.
  • a drawing area 808 is presented in the user interface 800.
  • a user can touch the drawing area 810 to draw utilizing the selected pencil tool 802.
  • a blot area 812 associated with the touch is shown.
  • the drawing application can receive the input and determine that area the user wishes to draw to is ambiguous because of the available options of pixel space to use the pencil tool on in the blot area 812.
  • FIG. 8B shows user interface 820 that includes a magnified region 822 to allow for unambiguous drawing using the tool.
  • the ambiguity can be determined based on a ratio of the size of a blot area 824 of another touch to the size of a tip of the pencil tool.
  • the user can draw using the pencil tool in the magnified area 822.
  • the magnified area 822 is returned to its previous state, as shown in user interface 840 of FIG. 8C.
  • the drawing 842 can also be added to the drawing area 810.
  • the use of other tools, for example, the eraser tool 808, may not invoke a magnification. This can be because the eraser tool has a sufficiently big pointer that when in use the user touches are determined to be unambiguous. As previously noted, the ambiguity determination can be based on many factors.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Example embodiments disclosed herein relate to touch screen selection. A device displays a plurality of touch elements on a touch-enabled display. The device receives a touch input via the touch-enabled display. Further, the device determines an ambiguity based on the touch selection input. The device magnifies an area associated with the touch selection input based on the ambiguity determination. Moreover, the area includes a set of the touch elements associated with the ambiguity determination. The device receives another touch input at the area via the touch-enabled display. Then, the device determines one of the touch elements based on the other touch input.

Description

TOUCH SCREEN SELECTION
BACKGROUND
[0001 ] Service providers and device manufacturers are challenged to deliver quality and value to consumers, for example by providing innovative user interfaces to customers. The use of touch screen devices is growing around the world. The development of such touch screen devices generates various technical challenges for manufacturers and service providers. For example, user interfaces that are useful in desktop computing situations can be inadequate for touch screen devices.
BRIEF DESCRIPTION OF THE DRAWINGS
[0002] The following detailed description references the drawings, wherein:
[0003] FIG. 1 is a block diagram of a computing device utilizing a user interface for touch screen selection, according to one example;
[0004] FIGs. 2A and 2B are block diagrams of devices for providing a touch screen user interface, according to various examples;
[0005] FIG. 3 is a flowchart of a method for selecting user interface elements, according to one example;
[0006] FIG. 4 is a flowchart of a method for performing enhanced tasks via magnified user interface area, according to one example;
[0007] FIG. 5 is a flowchart of a method for determining a magnification level of an area associated with a touch selection input, according to one example;
[0008] FIG. 6 is a flowchart of a method for incrementally magnifying a user interface area to determine a touch selection input, according to one example;
[0009] FIGs. 7A - 7F are block diagrams of example user interfaces for touch screen selection via auto magnification, according to various examples; and [0010] FIGs. 8A - 8C are block diagrams of example user interfaces for touch screen selection via auto magnification, according to various examples.
DETAILED DESCRIPTION
[001 1 ] Touch screen devices are growing as a mechanism to provide content to users. The number of individuals utilizing touch screen devices, such as smart phones and tablet computing devices is continually growing. Various user interfaces are utilized to provide content via these touch screen devices. However, many user interfaces utilized on such devices may be inadequate to provide a complete user experience.
[0012] For example, a user's finger is a common input for touch screen devices; however a finger is sometimes too large to unambiguously select a user interface element presented on a touch screen display. Touch screens in mobile phones and other devices were created to be portable, with smaller screens. As such, the touch screens on these devices can be too small to recognize an intended selection of a user interface element. For example, touch screen selection of one or more selection elements such as soft keypads, links, text boxes, selection boxes, content creation elements, etc. can be a problem when a finger touch spans multiple elements or are close to other elements, which makes the device unsure about which selection element was intended. A further challenge in touch screen selection is to provide an enhanced selection user interface that is intuitive for a user.
[0013] Accordingly, various embodiments disclosed herein relate to enabling touch screen selection of interface elements via auto magnification. The auto magnification can be utilized when an ambiguity in a touch selection of interface elements is determined. The device then magnifies an area associated with a touch location based on the determination that there is ambiguity in what the selection refers to.
[0014] Auto magnification can be utilized to provide a seamless user experience to a user. Magnifying a particular area directly presents the same user interface to the user. As such, the user can easily use the magnified area in the same way as the previously presented user interface. Thus, the user may be able to quickly determine that the same touch mechanism can be used to select the element presented on the screen. The user can then touch the magnified area of the screen to select the element. When the element is selected, a task based on the selection can be performed.
[0015] Tasks can vary based on a type of selection element. In one example, a task for a hyperlink can be to follow the link in a web browser. In another example, a task for a list selection element can be to expand a list associated with the list selection element. In this example, the auto magnification can be set to automatically de-magnify the area to return the user interface to the previous view before expanding the list.
[0016] Referring now to the drawings, FIG. 1 is a block diagram of a computing device utilizing a user interface for touch screen selection, according to one example. The computing device 100 includes, for example, a processor 1 10, and a machine-readable storage medium 120 including instructions 122, 124, 126, 128, 130 for touch screen selection based on automatic magnification. Computing device 100 may be, for example, a notebook computer, a slate or tablet computing device, a portable reading device, a wireless email device, a mobile phone, or any other computing device with a touch-enabled screen.
[0017] Processor 1 10 may be, at least one central processing unit (CPU), at least one semiconductor-based microprocessor, at least one graphics processing unit (GPU), other hardware devices suitable for retrieval and execution of instructions stored in machine-readable storage medium 120, or combinations thereof. For example, the processor 1 10 may include multiple cores on a chip, include multiple cores across multiple chips, multiple cores across multiple devices (e.g., if the computing device 100 includes multiple node devices), or combinations thereof. Processor 1 10 may fetch, decode, and execute instructions 122 - 130 to implement touch selection based on auto magnification, for example via the processes of FIGs. 3 - 6. As an alternative or in addition to retrieving and executing instructions, processor 1 10 may include at least one integrated circuit (IC), other control logic, other electronic circuits, or combinations thereof that include a number of electronic components for performing the functionality of instructions 122 - 130.
[0018] Machine-readable storage medium 120 may be any electronic, magnetic, optical, or other physical storage device that contains or stores executable instructions. Thus, machine-readable storage medium may be, for example, Random Access Memory (RAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a storage drive, a Compact Disc Read Only Memory (CD-ROM), and the like. As such, the machine- readable storage medium 120 can be non-transitory. As described in detail below, machine-readable storage medium 120 may be encoded with a series of executable instructions for selecting touch elements on a touch-enabled display. In certain embodiments, a touch element is a portion of a presented user interface on a touch-enabled display that is activated when a touch input is determined to be associated portion. Examples of touch elements include links (e.g., web links to other web pages, links embedded in icons or images, etc.), menu items (e.g., menu bars, file menus, toolbars, etc.), fields (e.g., a search input area, a text area, etc.), and movable user interface components (e.g., a window, a pointer, a drawing tool, etc.).
[0019] The computing device 100 can utilize a touch-enabled display to present graphical information. In certain embodiments, a touch-enabled display is a visual display that can detect the presence and location of a touch with an area of the display. Various technologies can be utilized to implement the display, for example, resistive technology, capacitive technology, infrared technology, etc. The processor 1 10 can cause presentation of graphical information by transmitting the information to the display.
[0020] The computing device 100 can provide user interfaces on the touch- enabled display by executing programs (e.g., an operating system, an application, etc.). As such, display instructions 122 can be executed to cause display of a plurality of touch elements on the touch-enabled display. The touch elements may be presented on the touch-enabled display as part of a user interface being presented by one or more of the programs. As previously noted, the touch elements can correspond to portions of the display that can be activated. Some touch elements may be displayed close together.
[0021 ] An input can be received by the processor 1 10 indicating that a portion of the screen has been touched. The portion can be considered a blot area. In certain embodiments, a blot area is the part of the touch-enabled display that is activated by the touch. The input instructions 124 are executed by the processor 1 10 to determine an action to take based on the input. The input can be received as a geometric set of points or a description of an area of the display touched. The processor 1 10 can map the blot area to locations of touch elements being displayed. As such, the processor 1 10 can determine if the blot area corresponds to one or more of the touch elements. In certain cases, this determination is unambiguous (e.g., only one touch element is in the blot area). In other cases, the determination may be ambiguous.
[0022] The processor 1 10 can execute ambiguity instructions 126 to determine whether the touch input is ambiguous in selecting one of the touch elements. As such, the processor 1 10 determines what touch elements are nearby the blot area. If no touch element or only one touch element is within the blot area, the input is unambiguous. If multiple touch elements can be associated with the blot area, the processor 1 10 determines a confidence level of how likely the touch is to correspond to a particular touch element. Different touch elements can have different criteria to determine the confidence level.
[0023] In one example, a first touch element is completely encompassed by the blot area while the other touch elements are minimally overlapping with the blot area. The processor 1 10 can determine that the first touch element is selected unambiguously.
[0024] In another example, a first touch element and a second touch element are completely or partially encompassed by the blot area. The processor 1 10 determines that there is an ambiguity as to which touch element was selected. Such ambiguity can be determined based on more than two touch elements. As such, the processor 1 10 determines that the blot area is ambiguous in determining one of the touch elements. Further, the processor 1 10 may determine that a set of the touch elements, which can include the first touch element and the second touch element, may be selected based on the blot area. For example, the set can include touch elements that have a certain confidence level that they may be selected by the touch input.
[0025] If the processor 1 10 determines that there is an ambiguity, the processor 1 10 executes zoom instructions 128. Zoom instructions 128 can be utilized to determine to zoom in onto a region of interest surrounding the blot area. This can occur automatically based on the ambiguity determination. The zooming can fill the entire display or can fill a portion of the display (e.g., as a callout). Further, the zoomed in region can maintain scaled spacing of the region. For example, the zooming in can be a magnification by a particular scale (e.g., a 2X zoom, a 1 .5X zoom, a 4X zoom, etc.) and retain scaled spatial relationships during the zoom. The maintenance of the spatial relationships and the zooming in of the region provides a large surface for the user to select the touch element. Further, the zoomed in region provides an intuitive user interface to the user because the user need not learn a new manner to interact with the region. That is because the user interface of the region interacts in the same manner as the non-zoomed in user interface.
[0026] In certain scenarios, the scale of the zooming can be determined based on spacing information between touch elements, the size of the touch elements, the size of the blot, or combinations thereof. The scale can be based on a function utilized to zoom in to a level where the blot area can unambiguously select one of the touch elements for which the ambiguity was detected. For example, the zooming can be executed in a manner in which the size of one of the touch elements (e.g., the smallest touch element, the largest touch element, etc.) is at least a certain percentage of the size of the blot. As such, the zoom instructions 128 can cause the processor 1 10 to examine the nearby touch elements to attempt to spread them apart in the zoomed in image to allow a subsequent user touch (creating a new blot) to uniquely specify the intended touch element. Moreover, the zooming can be based on a user profile. The user profile can determine individual parameters that can be utilized in determining the zoom level. For example, the user profile can include information about blot sizes generated by the user, which may include blot sizes for one or more fingers used by the user. With this approach, a processed blot size associated with the user and/or other tendencies of the user can be used to determine the zoom level.
[0027] Once the region is zoomed in on, the computing device 100 may receive another touch input to select a touch element via the touch-enabled display. The processor 1 10 can determine another blot area associated with the other touch input. Selection instructions 130 can then be utilized to determine one of the touch elements based on the other blot area. The other touch input can also go through the ambiguity determination. In this manner, if the other touch input is determined to be ambiguous, a portion of the region can be zoomed in on.
[0028] FIGs. 2A and 2B are block diagrams of devices for providing a touch screen user interface, according to various examples. Devices 200a, 200b include modules that can be utilized to magnify a region of interest to select touch elements. The respective devices 200a, 200b may be a notebook computer, a tablet computing device, a portable reading device, a wireless device, a mobile phone, a server or any other device that may be utilized to present touch screen user interface elements. The devices 200a, 200b can include a touch-enabled display, such as a touch screen 210 to present information, such as multiple touch elements, to a user. The touch screen 210 can be connected via one or more interfaces (e.g., via a cable or a wireless medium). A processor, such as a CPU, a GPU, or a microprocessor suitable for retrieval and execution of instructions and/or electronic circuits can be configured to perform the functionality of any of the modules 212 - 218 described below. In some embodiments, the devices 200a, 200b can include some of the modules (e.g., modules 212 - 218), the touch screen 210, and/or additional components. Further, in certain embodiments, some modules can be utilized to implement functionality of other modules described below.
[0029] As detailed below, devices 200a, 200b may include a series of modules 212 - 218 for utilizing a touch screen user interface during use of one or more applications executing on the devices 200a, 200b. Each of the modules 212— 218 may include, for example, hardware devices including electronic circuitry for implementing the functionality described below. In addition or as an alternative, each module may be implemented as a series of instructions encoded on a machine-readable storage medium of respective devices 200a, 200b and executable by a processor. It should be noted that, in some embodiments, some modules 212 - 218 are implemented as hardware devices, while other modules are implemented as executable instructions.
[0030] The touch screen 210 can be utilized to present interfaces to users. These interfaces can be part of a program (e.g., an operating system, an application, a game, etc.) executed by a processor 230 and stored in memory 232 of the device 200b. The touch screen 210 can further be utilized as an input source. The input detection module 212 can determine touch input from the touch screen 210. The input detection module 212 can further utilize a coordinate system to associate a touch input via the touch screen 210 with a portion of the screen. The portion of the screen can be considered a blot area. As such, the touch input can be associated with one or more points of the screen.
[0031 ] A selection module 214 attempts to determine what the touch input is attempting to activate and/or utilize. In certain scenarios, the touch elements can be options for what can be activated by the touch input. When touch input is received, the selection module 214 and/or the ambiguity module 216 can determine whether the touch input is ambiguous in selecting one of the touch elements. The touch input can be considered ambiguous if more than one of the touch elements could be selected. Thus, if ambiguous, the touch input can be considered to be open to being associated with more than one of the touch elements. If the input is unambiguous, the selection module 214 can select the associated touch element.
[0032] If the input is determined to be ambiguous as to the selection of one of the touch elements, the magnification module 218 is utilized to magnify a region of interest of the touch screen 210. The region may include the touch elements. These touch elements can be the touch elements that may be associated with the touch input. Further, the magnified region can maintain a scaled spatial orientation of the region. The magnification module 218 can zoom in on the region by a particular scale (e.g., a predefined scale, a scale dynamically determined, etc.). A dynamic scale can be determined based on spacing information between touch elements, the size of the touch elements, the size of a blot associated with the touch input, or combinations thereof. Moreover, the magnification can be based on a user profile.
[0033] In one embodiment, the magnification module determines a scaling of the magnification based on a blot area and sizing information about at least two of the touch elements. As such, the ambiguity module 216 determines a blot area associated with the touch input. Further, the ambiguity module 216 can determine that the blot area covers at least two of the touch elements. The ambiguity module 216 can then determine sizing information about the at least two touch elements. This sizing information and the information describing the blot area can be utilized to dynamically determine the scale.
[0034] Once the magnified region is presented, the user may utilize that area as a user interface. As such, the user can select the intended touch element. The input detection module 212 then detects another touch input via the touch screen 210 in the magnified area. This input is then provided to the selection module 214 and/or ambiguity module 216. The ambiguity module 216 can process the information and determine whether the other touch input is ambiguous. If the other touch input is determined to be ambiguous, the magnification module 218 can be utilized to further magnify another region of interest in the magnified region. If the other touch input is determined to be unambiguous, the selection module 214 selects a touch element based on the other touch input. The original touch input and the other touch input can be a direct touch of at least one of the touch elements. As such, the user may utilize the same approach of selecting the selected touch element. This approach is simple and intuitive. Once the selection is complete, the magnification module 218 can de-magnify the region and a task associated with the selection can be performed.
[0035] In certain scenarios, an input device 240 can include a sensor, a keyboard, a mouse, a remote, a keypad, a microphone, or the like. Sensors can be used to implement various technologies, such as infrared technology, camera technology, touch screen technology, etc. Further, the device 200b may include devices utilized for input and output (not shown), such as a networking interface (e.g., Ethernet), a Universal Serial Bus (USB) connection, etc.
[0036] The input/output interfaces 234 can additionally be utilized to drive output to an output device 242. One of the modules 212 - 218, the processor 230, or another component of the device 200b can be utilized to send a signal via the input/output interfaces 234 to output devices 242. Examples of output devices include speakers, other display devices, amplifiers, etc.
[0037] In one embodiment, the ambiguity module 216 determines a blot area associated with the original touch input. The touch elements in this scenario can be associated with a drawing tool that includes a pointer that is smaller than the blot area. This may be utilized in a painting tool or pencil tool used by the user. In this scenario, a user profile, an application profile associated with the drawing tool, or other information can be processed to determine a magnification level to magnify a region of interest based on the original touch input. In one example, the magnification module 218 magnifies the region to a level where the pointer is within a threshold ratio of the blot area. With this approach, if the pointer (e.g., a paint brush or pencil end) is within a certain percentage of the size of the blot area, other determinations can be made by the processor 230 to determine which touch element to select. For example, the touch element to select can be in the center of the blot area created. [0038] FIG. 3 is a flowchart of a method for selecting user interface elements, according to one example. Although execution of method 300 is described below with reference to device 200b, other suitable components for execution of method 300 can be utilized (e.g., computing device 100, device 200a, etc.). Additionally, the components for executing the method 300 may be spread among multiple devices. Method 300 may be implemented in the form of executable instructions stored on a machine-readable storage medium, such as storage medium 120, and/or in the form of electronic circuitry.
[0039] Method 300 may start at 302 and proceed to 304, where device 200 may display a plurality of selection elements on a touch-enabled display such as the touch screen 210. In certain embodiments, a selection element is a user interface element that may be chosen or activated by an input. By way of example, a selection element can be a link, a text input box, a soft key, a list, menu items on a tool bar, drawing space to be utilized by a drawing tool, a content box, a search box, or the like. Example user interfaces and selection elements are provided in FIGs. 7A - 7F and FIGs. 8A - 8C.
[0040] A user may then touch the touch-enabled display to select one of the selection elements. The input detection module 212 receives a touch selection input via the touch-enabled display (306). The input detection module 212 can then determine where the touch selection input is located, for example, by associating one or more coordinates with the touch selection input.
[0041 ] Then, at 308, the ambiguity module 216 determines whether the touch selection is ambiguous. The touch selection input can be considered ambiguous if the touch selection input is not clear as to selecting a selection element. As noted above, this can occur if it is determined that more than one selection element may be selected by the input. If there is no ambiguity and only one selection element is chosen, the element can be selected and tasks can be processed based on the selection.
[0042] If the touch selection input is determined to be ambiguous, then the method 300 proceeds to 310, where the magnification module 218 magnifies an area associated with the touch selection input. Magnification can be considered a direct magnification and equal scaling of the image presented at the display. The area can include, at least in part, the selection elements. The area includes a set of the selection elements associated with the ambiguity determination, that is, the selection elements that may be selected by the touch input. In certain scenarios, the set may be considered the selection elements that may be associated with the input, which cause the ambiguity. Further, the magnification of the area can maintain the spatial orientation of the area. As such, the scaling during the magnification is equal for the spaces between selection elements as well as the magnification of the selection elements. The magnification of the area can further be presented on a full screen area of the touch-enabled display or a portion of the display. For example, a portion of the display can be viewed as a callout from the location of the touch selection input.
[0043] Once the magnified area is displayed, a user can intuitively touch the magnified area to select the user's desired selection element via the display. As such, at 312, the input detection module 212 receives another touch selection input at the area via the touch-enabled display. The input detection module 212 then determines where the other input is located, for example, by associating one or more coordinates with the other input. Further, the original touch selection input and the other touch selection input can correspond to a direct touch of at least one of the selection elements.
[0044] At 314, the selection module 214 selects one of the selection elements based on the other touch input to the magnified area. In certain scenarios, the ambiguity module 216 can determine whether the selection of the one selection element is ambiguous. If the selection is ambiguous, the processes of FIG. 6 can be utilized to generate an unambiguous selection. If the selection is unambiguous, a task associated with the selection element can be performed. Then, at 316, the method 300 stops.
[0045] FIG. 4 is a flowchart of a method for performing enhanced tasks via magnified user interface area, according to one example. Although execution of method 400 is described below with reference to device 200b, other suitable components for execution of method 400 can be utilized (e.g., computing device 100, device 200a, etc.). Additionally, the components for executing the method 400 may be spread among multiple devices. Moreover, method 400 may be implemented in the form of executable instructions stored on a machine-readable storage medium, such as storage medium 120, and/or in the form of electronic circuitry.
[0046] Method 400 may begin after a magnified area is presented to a user and the device 200b receives another input at the magnified area. Method 400 may start at 402 and proceed to 404, where device 200b determines that a selection of a selection element is unambiguous. The determination can be made using various methods, for example, determining a confidence level that the input is associated with the selection element. If the determined confidence level meets a predetermined threshold confidence level, the selection element can be determined to be unambiguous. In other scenarios, other criteria can be utilized in determining the ambiguity of the selection.
[0047] At 406, an enhanced task is performed based on the selection 406. In one example, the enhanced task is to perform an action associated with the selection element and de-magnifying the magnified area to the area. The de- magnification can occur before or after the task is completed.
[0048] In one example, activation of a list can be a task to perform based on the selection of the magnified selection element. Instead of performing the opening of the list in the magnified area, the area can be de-magnified and then the opened list can be presented. In this manner, the user can view the opened list in the scaling that the user was originally viewing. An example of opening a list is described below in the discussion of FIG. 7E.
[0049] In another example, the task can be to execute a link to a webpage. In this example, the enhanced task can include saving a de-magnified version of the original user interface to a memory and then executing the link. The link can be presented without magnification. Further, if the user utilizes a back selection element to move back to the original user interface, the saved de-magnified version of the user original user interface can be presented. Then, at 408, the method 400 stops.
[0050] FIG. 5 is a flowchart of a method for determining a magnification level of an area associated with a touch selection input, according to one example. Although execution of method 500 is described below with reference to computing device 100, other suitable components for execution of method 500 can be utilized (e.g., device 200a, device 200b, etc.). Additionally, the components for executing the method 500 may be spread among multiple devices. Method 500 may be implemented in the form of executable instructions stored on a machine-readable storage medium, such as storage medium 120, and/or in the form of electronic circuitry.
[0051 ] Method 500 can occur after a touch input is received and determined to be ambiguous. Method 500 may start at 502 and proceed to 504, where computing device 100 may determine a portion of the touch-enabled display activated by the touch selection input. The portion can be, for example, a blot area. Further, the portion can be associated with coordinates of the display.
[0052] Then, at 506, the computing device 100 can determine information about selection elements corresponding to the portion. For example, these selection elements can be a set of elements that may be determined to cause the ambiguity because more than one of the selection elements may be the desired target of the user. The information can include sizing information about the selection elements. Further, the information can include spacing information associated with the selection elements. At 508, a size of the portion can be determined. The size of the portion and/or selection elements can be associated with a length, width, radius, area, or the like associated with the elements.
[0053] Then, at 510, the computing device 100 can determine a magnification level based on the size of the portion, the sizing information, the spacing information, or a combination thereof. For example, the magnification can be to a level such that the size of the portion (e.g., blot area) is at least a certain percentage of the size of one or more particular elements. In another example, the size of the one or more particular elements as well as the space size between elements can be compared to the portion to determine the magnification level. The magnification level can be optimized in such a manner that the size of the particular elements and/or the size of the space between elements can be used so that another input the size of the portion can be used to uniquely activate one of the elements. In some scenarios, blot area size information used in the calculation can be determined from a user profile or other collected information. The method 500 then proceeds to 512, where the method 500 stops.
[0054] FIG. 6 is a flowchart of a method for incrementally magnifying a user interface area to determine a touch selection input, according to one example. Although execution of method 600 is described below with reference to device 200b, other suitable components for execution of method 600 can be utilized (e.g., device 200a, computing device 100, etc.). Additionally, the components for executing the method 600 may be spread among multiple devices. Method 600 may be implemented in the form of executable instructions stored on a machine- readable storage medium, such as storage medium 120, and/or in the form of electronic circuitry.
[0055] Method 600 may start in 602, where device 200b can be at a state (e.g., at 310 or 312 of method 300) in which a user interface is magnified to an area in response to a received ambiguous touch selection input. Another touch selection input is received to select one of a plurality of selection elements presented in the area. The method 600 can then proceed to 604, where device 200b is caused to determine that the other touch selection input is ambiguous in selecting the one selection element.
[0056] At 606, based on the ambiguity determination, the magnification module 218 further magnifies the user interface to a portion of the magnified area based on the other touch selection input received. The portion can be a region nearby a blot area caused by the other touch selection input. Further, the portion can include the touch selection elements that were the cause of the ambiguity determination. As such, the portion can include the one selection element and other selection elements that are within range of the touch selection input. The inclusion of the selection elements can include partial selection elements. As such, to include a selection element in the magnified portion means to include at least a selectable part of the selection element. The entire selection element may also be included.
[0057] The user can view the magnified portion to select the one selection element. As such, the device 200b receives a third touch selection input (at 608). This input can be via the touch screen 210 and be at the magnified portion. The selection module 214 can select the one selection element by associating the third touch selection input with the one selection element (at 610). As such, the selection of the one touch selection element can be based, at least in part, on the third touch selection input. The method 600 can repeat anytime a touch selection input is deemed to be ambiguous. Then, at 612, the method 600 stops.
[0058] FIGs. 7A - 7F are block diagrams of example user interfaces for touch screen selection via auto magnification, according to various examples. The user interfaces can be associated with a device. User interface 700 shows multiple touch elements 702a, 702b, 702c, 704 as well as other user interface components. For example, picture 706 may be presented, but not be selectable.
[0059] A user of the device can attempt to select, via a touch-enabled display, one of the touch elements. Here, the touch is represented by a blot area 718. The blot area looks like a circle in FIG. 7B, however, the blot area 718 can be represented by one or more points, a shape like the circle, or another representation. Further, the blot area need not be presented to a user. Because the blot area 718 could be associated with Item A 702a or Item B 702b, the blot area 718 can be considered ambiguous. A magnification module of the device magnifies an area associated with the selection to allow for an unambiguous selection.
[0060] User interface 720 of FIG. 7C shows a magnified region of interest based on the ambiguity decision and the blot area 718. Item A 722a, Item B 722b, and Item C 722c are displayed in the magnified region. The user can then enter another touch input to select one of the items unambiguously. User interface 730 of FIG. 7D shows a blot area 738 associated with the touch input. The blot area 738 unambiguously selects Item B 722b. The selection of Item B 722b triggers an activation of a task associated with Item B 722b.
[0061 ] The user interface 740 of FIG. 7E shows Items A - C 742a - 742c as well as icon 744 and Items B1 746a and B2 746b. The user interface no longer includes the magnification and can automatically go back to the previous state before the activation of the task. In this scenario, the task includes the presentation of Items B1 746a and B2 746b. In certain scenarios, the task can be performed simultaneously with the demagnification and/or before the demagnification.
[0062] FIG. 7F includes user interface 750 which shows the Items 742a - 742c, 746a, 746b as well as the icon 744. Here, another touch input is detected. Another blot area 748 is determined in associated with the touch input. In this case, the device can determine that the selection of touch elements is ambiguous. In one example, the icon 744 is not a touch element. As such, a magnification of the touch elements can focus on Item B 742b and Item B1 746a. In another example, the icon 744 is a touch element. In this example, the icon 744 can be a focus of the magnified area.
[0063] FIGs. 8A - 8C are block diagrams of example user interfaces for touch screen selection via auto magnification, according to various examples. User interface 800 of FIG. 8A shows a drawing application including a toolbar on the left. The toolbar can include a pencil tool 802, a rectangle tool 804, an ellipse tool 806, and an eraser tool 808. The pencil tool 802 is selected. A drawing area 808 is presented in the user interface 800. A user can touch the drawing area 810 to draw utilizing the selected pencil tool 802. A blot area 812 associated with the touch is shown.
[0064] The drawing application can receive the input and determine that area the user wishes to draw to is ambiguous because of the available options of pixel space to use the pencil tool on in the blot area 812. As such, FIG. 8B shows user interface 820 that includes a magnified region 822 to allow for unambiguous drawing using the tool. The ambiguity can be determined based on a ratio of the size of a blot area 824 of another touch to the size of a tip of the pencil tool. As such, the user can draw using the pencil tool in the magnified area 822. When the user is done (e.g., determined by letting go of the touch used to generate the drawing) the magnified area 822 is returned to its previous state, as shown in user interface 840 of FIG. 8C. The drawing 842 can also be added to the drawing area 810. The use of other tools, for example, the eraser tool 808, may not invoke a magnification. This can be because the eraser tool has a sufficiently big pointer that when in use the user touches are determined to be unambiguous. As previously noted, the ambiguity determination can be based on many factors.

Claims

CLAIMS What is claimed is:
1 . A method comprising:
displaying a plurality of selection elements on a touch-enabled display;
receiving a touch selection input via the touch-enabled display;
determining an ambiguity based on the touch selection input;
magnifying an area associated with the touch selection input based on the ambiguity determination, wherein the area includes a set of the selection elements associated with the ambiguity determination;
receiving another touch selection input at the area via the touch-enabled display; and
selecting one of the selection elements based on the other touch selection input.
2. The method of claim 1 , wherein the touch selection input and the other touch selection input correspond to a direct touch of at least one of the selection elements.
3. The method of claim 1 , wherein the magnification of the area maintains a scaled spatial orientation of the area.
4. The method of claim 1 , wherein the magnification of the area displays the area to a full screen of the touch-enabled display.
5. The method of claim 1 , further comprising:
determining that the selection of the one selection element is unambiguous; and de-magnifying the area.
6. The method of claim 1 , further comprising:
determining a portion of the touch-enabled display activated by the touch selection input;
determining sizing information about the selection elements that correspond to the portion, wherein the magnification is determined based on a size of the portion and the sizing information.
7. The method of claim 1 , wherein the selection elements include at least one of: a link, a soft key, a drawing tool, a content box, and a search box.
8. The method of claim 1 , further comprising:
determining that the other touch selection input is ambiguous in selecting the one selection element;
further magnifying to a portion of the area based on the other touch selection input;
receiving a third touch selection input at the portion; and
selecting the one selection element further based on the third touch selection input.
9. A device comprising:
a touch screen to display a plurality of touch elements;
an input detection module to detect a touch input via the touch screen;
a selection module;
an ambiguity module to determine that the touch input is ambiguous in selecting one of the touch elements; and
a magnification module to magnify a region of interest of the touch screen associated with the touch input, wherein the region includes the touch elements, and wherein the magnified region maintains scaled spatial orientation of the region;
wherein the input detection module detects another touch input in the magnified region, and
wherein the selection module selects the one touch element based on the other touch input.
10. The device of claim 9, wherein the touch input and the other touch input are a direct touch of at least one of the touch elements.
1 1 The device of claim 9, wherein the ambiguity module determines a blot area associated with the touch input,
wherein the touch elements are associated with a drawing tool that includes a pointer that is smaller than the blot area, and
wherein the magnification module magnifies the region to a level where the pointer is within a threshold ratio of the blot area.
12. The device of claim 9,
wherein the ambiguity module determines a blot area associated with the touch input and that the blot area covers at least two of the touch elements, wherein the ambiguity module determines sizing information about the at least two touch elements, and
wherein the magnification module determines a scaling of the magnification of the region based on the blot area and the sizing information.
13. A non-transitory machine-readable storage medium storing instructions that, if executed by a processor of a device, cause the processor to:
cause display of a plurality of touch elements on a touch-enabled display;
determine a blot area associated with an input received via the touch-enabled display;
determine that the blot area is ambiguous to determine one of the touch elements;
zoom in on a region of interest surrounding the blot area based on the ambiguity determination, wherein the zoomed in region maintains scaled spacing of the region;
determine another blot area associated with another input received via the touch-enabled display; and
determine one of the touch elements based on the other blot area.
14. The non-transitory machine-readable storage medium of claim 13, further comprising instructions that, if executed by the processor, cause the processor to: determine a set of the touch elements that may be selected based on the blot area; and
determine spacing information of the set,
wherein a scale for the zoom based on a size of the blot area and the spacing information.
15. The non-transitory machine-readable storage medium of claim 14, wherein the zoom is further based on a user profile.
PCT/US2011/032968 2011-04-19 2011-04-19 Touch screen selection WO2012144984A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
CN201180070231.4A CN103492981A (en) 2011-04-19 2011-04-19 Touch screen selection
US14/112,063 US9519369B2 (en) 2011-04-19 2011-04-19 Touch screen selection
PCT/US2011/032968 WO2012144984A1 (en) 2011-04-19 2011-04-19 Touch screen selection
EP11863932.7A EP2699986B1 (en) 2011-04-19 2011-04-19 Touch screen selection

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2011/032968 WO2012144984A1 (en) 2011-04-19 2011-04-19 Touch screen selection

Publications (1)

Publication Number Publication Date
WO2012144984A1 true WO2012144984A1 (en) 2012-10-26

Family

ID=47041841

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2011/032968 WO2012144984A1 (en) 2011-04-19 2011-04-19 Touch screen selection

Country Status (4)

Country Link
US (1) US9519369B2 (en)
EP (1) EP2699986B1 (en)
CN (1) CN103492981A (en)
WO (1) WO2012144984A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2999310A1 (en) * 2012-12-12 2014-06-13 Oberthur Technologies Electronic device i.e. smart mobile phone, has touch input on input interface, and execution unit executing function while taking display characteristics of interface element into account so as to select input of interface element
EP2800020A1 (en) * 2013-04-30 2014-11-05 Dassault Systèmes A computer-implemented method for manipulating three-dimensional modeled objects of an assembly in a three-dimensional scene.
WO2015088882A1 (en) * 2013-12-09 2015-06-18 Microsoft Technology Licensing, Llc Resolving ambiguous touches to a touch screen interface
EP2984548A4 (en) * 2013-04-10 2016-11-30 Google Inc System and method for disambiguating item selection
US9996235B2 (en) 2015-10-15 2018-06-12 International Business Machines Corporation Display control of an image on a display screen

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5962085B2 (en) * 2012-03-15 2016-08-03 ソニー株式会社 Display control apparatus, control method thereof, and program
JP6155835B2 (en) * 2013-05-16 2017-07-05 株式会社リコー Information processing apparatus, information processing method, and program
US9753641B2 (en) * 2015-03-26 2017-09-05 Kyocera Document Solutions Inc. Display input device, control method for display input device, and computer-readable storage medium for storing control program for display input device
US9910557B2 (en) * 2015-04-17 2018-03-06 Xerox Corporation Intent-based graphical user interface window control
US10275436B2 (en) * 2015-06-01 2019-04-30 Apple Inc. Zoom enhancements to facilitate the use of touch screen devices
CN104898889A (en) * 2015-06-26 2015-09-09 小米科技有限责任公司 Character determining method and device
US9554273B1 (en) * 2015-09-04 2017-01-24 International Business Machines Corporation User identification on a touchscreen device
US10082954B2 (en) * 2015-09-04 2018-09-25 International Business Machines Corporation Challenge generation for verifying users of computing devices

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060059437A1 (en) * 2004-09-14 2006-03-16 Conklin Kenneth E Iii Interactive pointing guide
US20070198950A1 (en) 2006-02-17 2007-08-23 Microsoft Corporation Method and system for improving interaction with a user interface
WO2008070815A1 (en) 2006-12-07 2008-06-12 Microsoft Corporation Operating touch screen interfaces
US20090231285A1 (en) * 2008-03-11 2009-09-17 Microsoft Corporation Interpreting ambiguous inputs on a touch-screen
US20100077304A1 (en) * 2008-09-19 2010-03-25 Microsoft Corporation Virtual Magnification with Interactive Panning
US20100257478A1 (en) * 1999-05-27 2010-10-07 Longe Michael R Virtual keyboard system with automatic correction

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7760187B2 (en) 2004-07-30 2010-07-20 Apple Inc. Visual expander
JP2005044026A (en) * 2003-07-24 2005-02-17 Fujitsu Ltd Instruction execution method, instruction execution program and instruction execution device
US7626599B2 (en) * 2005-07-12 2009-12-01 Microsoft Corporation Context map in computer display magnification
TWI328185B (en) * 2006-04-19 2010-08-01 Lg Electronics Inc Touch screen device for potable terminal and method of displaying and selecting menus thereon
KR20090024541A (en) 2007-09-04 2009-03-09 삼성전자주식회사 Method for selecting hyperlink and mobile communication terminal using the same
RU2495477C2 (en) 2008-09-10 2013-10-10 ОПЕРА СОФТВЭА ЭйСиЭй Method and apparatus for selecting object on display screen
CN101430632A (en) 2008-12-19 2009-05-13 深圳华为通信技术有限公司 Touch screen input method and apparatus, and communication terminal
US8423897B2 (en) * 2010-01-28 2013-04-16 Randy Allan Rendahl Onscreen keyboard assistance method and system
CN101814002A (en) 2010-03-24 2010-08-25 华为终端有限公司 Virtual keyboard and input method using virtual keyboard
CN101968711A (en) * 2010-09-29 2011-02-09 北京播思软件技术有限公司 Method for accurately inputting characters based on touch screen

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100257478A1 (en) * 1999-05-27 2010-10-07 Longe Michael R Virtual keyboard system with automatic correction
US20060059437A1 (en) * 2004-09-14 2006-03-16 Conklin Kenneth E Iii Interactive pointing guide
US20070198950A1 (en) 2006-02-17 2007-08-23 Microsoft Corporation Method and system for improving interaction with a user interface
WO2008070815A1 (en) 2006-12-07 2008-06-12 Microsoft Corporation Operating touch screen interfaces
US20090231285A1 (en) * 2008-03-11 2009-09-17 Microsoft Corporation Interpreting ambiguous inputs on a touch-screen
US20100077304A1 (en) * 2008-09-19 2010-03-25 Microsoft Corporation Virtual Magnification with Interactive Panning

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP2699986A4 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2999310A1 (en) * 2012-12-12 2014-06-13 Oberthur Technologies Electronic device i.e. smart mobile phone, has touch input on input interface, and execution unit executing function while taking display characteristics of interface element into account so as to select input of interface element
EP2984548A4 (en) * 2013-04-10 2016-11-30 Google Inc System and method for disambiguating item selection
EP2800020A1 (en) * 2013-04-30 2014-11-05 Dassault Systèmes A computer-implemented method for manipulating three-dimensional modeled objects of an assembly in a three-dimensional scene.
CN104133928A (en) * 2013-04-30 2014-11-05 达索***公司 Computer-implemented method for manipulating three-dimensional modeled objects of an assembly in a three-dimensional scene
US9710131B2 (en) 2013-04-30 2017-07-18 Dassault Systemes Computer-implemented method for manipulating three-dimensional modeled objects of an assembly in a three-dimensional scene
WO2015088882A1 (en) * 2013-12-09 2015-06-18 Microsoft Technology Licensing, Llc Resolving ambiguous touches to a touch screen interface
US9996235B2 (en) 2015-10-15 2018-06-12 International Business Machines Corporation Display control of an image on a display screen
US10318133B2 (en) 2015-10-15 2019-06-11 International Business Machines Corporation Display control of an image on a display screen
US10572127B2 (en) 2015-10-15 2020-02-25 International Business Machines Corporation Display control of an image on a display screen
US10768799B2 (en) 2015-10-15 2020-09-08 International Business Machines Corporation Display control of an image on a display screen

Also Published As

Publication number Publication date
US9519369B2 (en) 2016-12-13
EP2699986B1 (en) 2019-03-20
US20140049499A1 (en) 2014-02-20
EP2699986A4 (en) 2014-12-17
CN103492981A (en) 2014-01-01
EP2699986A1 (en) 2014-02-26

Similar Documents

Publication Publication Date Title
US9519369B2 (en) Touch screen selection
US10489047B2 (en) Text processing method and device
US8291348B2 (en) Computing device and method for selecting display regions responsive to non-discrete directional input actions and intelligent content analysis
EP2715499B1 (en) Invisible control
US10627990B2 (en) Map information display device, map information display method, and map information display program
US8823749B2 (en) User interface methods providing continuous zoom functionality
US9552154B2 (en) Device and method for providing a user interface
US8413075B2 (en) Gesture movies
US20130047100A1 (en) Link Disambiguation For Touch Screens
EP2664986A2 (en) Method and electronic device thereof for processing function corresponding to multi-touch
EP2706449B1 (en) Method for changing object position and electronic device thereof
US20150186004A1 (en) Multimode gesture processing
US20140082559A1 (en) Control area for facilitating user input
KR20160094442A (en) Clicked object magnifying method and apparatus based on floating touch
US9052767B2 (en) Information terminal device and touch panel display method
US20110316887A1 (en) Electronic device with a touch screen and touch operation control method utilized thereby
CN108108417B (en) Cross-platform adaptive control interaction method, system, equipment and storage medium
CN113268182A (en) Application icon management method and electronic equipment
US10019127B2 (en) Remote display area including input lenses each depicting a region of a graphical user interface
JP5585239B2 (en) Information processing apparatus, control program, recording medium, and display control apparatus
US10261675B2 (en) Method and apparatus for displaying screen in device having touch screen
CN115480664A (en) Touch response method and device, electronic equipment and storage medium
US9134894B2 (en) Electronic device, storage medium and method for selecting objects of the electronic device
CN103902166A (en) Display method and electronic device
JP2012203791A (en) Input device and input method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11863932

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2011863932

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 14112063

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE