WO2012001432A1 - System and method for control of functions and data entry for electronic devices, and computer program for implementing said system and method - Google Patents

System and method for control of functions and data entry for electronic devices, and computer program for implementing said system and method Download PDF

Info

Publication number
WO2012001432A1
WO2012001432A1 PCT/HR2010/000019 HR2010000019W WO2012001432A1 WO 2012001432 A1 WO2012001432 A1 WO 2012001432A1 HR 2010000019 W HR2010000019 W HR 2010000019W WO 2012001432 A1 WO2012001432 A1 WO 2012001432A1
Authority
WO
WIPO (PCT)
Prior art keywords
keys
fact
functions
signs
sign
Prior art date
Application number
PCT/HR2010/000019
Other languages
French (fr)
Inventor
Anto Spajic
Original Assignee
Anto Spajic
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Anto Spajic filed Critical Anto Spajic
Priority to PCT/HR2010/000019 priority Critical patent/WO2012001432A1/en
Publication of WO2012001432A1 publication Critical patent/WO2012001432A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/023Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
    • G06F3/0233Character input methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/0202Constructional details or processes of manufacture of the input device
    • G06F3/0216Arrangements for ergonomically adjusting the disposition of keys of a keyboard
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/023Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
    • G06F3/0233Character input methods
    • G06F3/0236Character input methods using selection techniques to select from displayed items
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0489Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using dedicated keyboard keys or combinations thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2200/00Indexing scheme relating to G06F1/04 - G06F1/32
    • G06F2200/16Indexing scheme relating to G06F1/16 - G06F1/18
    • G06F2200/163Indexing scheme relating to constructional details of the computer
    • G06F2200/1637Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer

Definitions

  • Present invention is a system and method for control of functions and data entry data in electronic devices and computer software implementing this system and method. More specifically it is a system that enables communication with mobile electronic devices, more specifically a keypad which enables quick data entry through the position of keys and through detecting the motion of the device.
  • the driver of the device reads the movement of the device and interprets the movement.
  • the movement is detected using accelerometers or equivalent means located in the device itself.
  • the combination of keys arrangement and accelerometers and appropriate preprogrammed graphic representation enables users to enter data simply and quickly.
  • the invention solves the technical problem of typing with 10 fmgers on a small hand-held device, without using an additional surface where the device can be placed.
  • Present system and method for data entry discloses combination of keys on the up and downsides of the device with recording of movement of the device which is held with both hands and computer program for implementing said system and method. Therefore the lack of keyboard space for tablet PCs or smartphones (mobile communication devices) is solved and data entry made faster and easier.
  • Second objective of the invention is showing graphically the position of the 8 fingers placed on the bottom side of the device and thumbs on the upper side and attributing dynamically and in real time a function or a sign to each finger-key. In this way there is no need for the user to memorize the position of the keys, which is necessary for the standard keyboard types.
  • Third objective of the invention is that user can define his/her sets of symbols, signs and functions attributed to the keys, using the program module described below.
  • the device contains not only keys on the top and bottom sides of the device, but also an accelerometer or its equivalent means and specialized driver program which, based on the measurement of the inclination (tilt) of the device, shows graphically the position of 8 fingers on the downside, thumbs on the upper side, descriptions of the current key attributions and changes those according to the changes of the inclination of the device for at least 8 keys on the lower and 2 keys on the upper side of the device.
  • the implementation of the keys can be hardware or software based.
  • Tablet computer or tablet PC is a completely new category of digital computer created in the beginning of 2010. Development of this type of computer was attempted earlier, but was rejected by the market due to poor technical realization. The key advancement came from American company Apple where the success of their Ipad shows that there is a need for such a device. A majority of other computer manufacturers are also preparing or have ready devices in this category. The main feature of these computers, separating them from notebooks, is a lack of physical keyboard. Furthermore, in the past several years there was a huge development of mobile communication devices occupying the area between computers and mobile phones. One of the main problems shared between tablet PCs and mobile communication devices is the problem of effective typing on a very small device, as can be easily seen on Ipad.
  • the device shows graphically in real time the position of 8 fingers on the downside, thumbs on the upper side, and descriptions of the current key attributions, changes those according to the changes of the inclination of the device for at least 8 keys on the lower and 2 keys on the upper side of the device.
  • the keys themselves can be hardware based or virtual.
  • the invention deals with the system and method for controlling functions of and entering data in electronic devices.
  • the electronic devices in question can be mobile electronic devices or external units connected with a mobile electronic device.
  • Electronic devices can be tablet PCs, smartphones, MID (mobile internet device), PDA and small mobile computers.
  • Electronic device is produced with the touch-sensitive screen, without physical keyboard with graphical user interface and at least 8 appropriately placed keys on the bottom side of the device and at least two keys on the upper side, which should ergonomically correspond to thumbs.
  • Hardware of the electronic device consists of a MID (mobile internet device).
  • MID mobile internet device
  • the MID's case has installed at least 10 keys distributed so that 8 of these are located on the bottom (back) side of the device (figure 3: 1a, 2a, 3 a, 4a, lb, 2b, 3b and 4b) so that their positions correspond to 8 fingers and at least 2 keys 102 and 103 are located on the upper (front) side of the device (figure 1, figure 2) so that they can be operated by the thumbs when MID is held with open palms.
  • the 8 keys on the bottom side are located so that they can be operated with fingers (one with each finger).
  • the keys can be physical (using micro switches), touch-activated (or virtual), on touch screen or on touch- sensitive bottom side of the device or a combination of the physical and touch keys.
  • sensors capable of detecting motion of the device and its position in space, where sensors can be accelerometers.
  • Special driver program - software which, based on the angle of inclination of the device, dynamically in real time draws 8 finger positions on the lower side of the device, thumb positions on the upper side and appropriate descriptions for the current functions of the keys. Based on the position/tilt of the device, the program dynamically allocates functions of at least 8 keys on the bottom side and at least two (virtual or hardware) keys on the upper side.
  • the program also shows a list of all functions (letters, numbers or signs) on the screen, grouped in sets corresponding to the tilts of the device so that active set is shown in a different way from other sets (using color, shape, size or font).
  • Computer program for implementing system and method for control of functions and data entry in electronic devices is placed on the media that can be read by a computer and consists of the following modules: 1. Graphical user interface module;
  • Module for user settings enabling the user to define own sets of symbols, signs and functions to be attributed to the keys, defining of the active set of signs, defining of the ranges for changing of the active set, defining styles and fonts and their exchange between users or between users and software manufacturers;
  • Exit module for passing sound and or vibration signals to the user to provide him/her with feedback.
  • Signal processing module obtains from the operating system data about active keys and state of the accelerometers i.e. position of the device. Based on these, it dynamically in real time creates functions and symbols and feeds them back to the operating system. The operating system in turn feeds these to other applications in the same manner used for physical and virtual keyboards.
  • the majority of MID's and tablet PC's have vibration capabilities similar to those found in the mobile phones.
  • System can be used for blind typing, which is the fastest way to type in date to the computer. Every change in this chain and every element of interaction are also reflected in the graphical output via graphical user interface module.
  • figure 1 shows the device and its screen with the graphical user interface
  • figure 2 shows the device and its screen with graphical user interface tilted with regard to one axis
  • - figure 6 shows implementation of keys on the bottom side of the device according to another embodiment of invention
  • - figure 7 shows the front/upper side of the external unit in case where the System and Method for Controlling Functions of and Entering Data in Electronic Devices is realized as the addition to an existing device
  • FIG. 8 shows the back/bottom side of the external unit in case where the System and Method for control of functions and data entry in Electronic Devices is realized as the addition to an existing device.
  • the System and Method for Controlling Functions of and Entering Data in Electronic Devices can be implemented as an integral part of an electronic device or as an external unit for an electronic device.
  • Figures 7-8 show an external unit of the same size as Apple Ipad and connected (for example using dock connector 200, as shown in Figure 7) to it.
  • the external unit could in a similar fashion be connected to another producer's unit.
  • the external unit, which implements the System and Method for Controlling Functions of and Entering Data in Electronic Devices has all of the necessary elements i.e. the 8 keys la, 2a, 3a, 4a, lb, 2b, 3b and 4b on the bottom side of the external unit, placed so that they correspond to the positions of 8 fingers.
  • the role of the screen 101 can be played by an existing screen.
  • the 8 keys can be made as shown in figure 3 or, alternatively, as shown in figure 6.
  • the software implementing the System and Method for Controlling Functions of and Entering Data in Electronic Devices shows on the graphical interface of the driver on the screen of the electronic device or the housing of the external unit at least two keys corresponding to the position of the thumbs.
  • the keys themselves can either be micro switches or touch sensitive part of the external unit's housing or touch sensitive screen.
  • Electronic device can be any mobile electronic device, such as tablet computer, smartphone, MID (mobile internet device), PDA or small mobile computer.
  • Present System and Method for Control of functions and data entry in Electronic Devices has at least 10 keys: 8 keys la, 2a, 3a, 4a, lb, 2b, 3b and 4b are located at the bottom side of the device so that their position corresponds to the 8 fingers and at least two keys 102 and 103 are located on the top side of the device in positions ergonomically appropriate for thumbs.
  • the keys themselves are either micro switches or parts of the surface of the device or external unit that are touch sensitive.
  • the system can be realized so that the device has on the bottom side 8 physical keys and on the top side two or more dynamically drawn virtual keys or so that the electronic devices has on the bottom side at least 8 virtual keys on the touch sensitive surface and on the top side two or more virtual keys on a touch screen.
  • the same variants of the keys and physical and virtual keys combinations can be used on the housing of the System realized as an external unit.
  • the System includes one or more motion sensors necessary to detect the position of the device itself and information about its movement with regard to surroundings. These sensors can be accelerometers or another type of sensor with the same function.
  • the System also has screen 101 witch is used as user interface, showing the text typed, part of the graphical interface with grouped sets of 105 to 109 signs 1 to m and graphical representation of the position of users hands with functions and signs for each finger for left hand 111 and for right hand 1 12, with the active set of signs marked specially.
  • the interface 104 is anchored to the coordinate set so that, regardless of the System's position regarding the x-axis the interface 104 is always parallel to it.
  • the drawings of the user's fingers and functions currently attributed to them, from the currently active set, are shown in real time, the changes of the active set are communicated to the user by touch (vibration system of the device), visually and with appropriate sound.
  • the elements of the screen 101 are shown in figures 1 and 2, where different sign sets are marked with 105, 106, 107, 108 and 109.
  • the sign sets are chosen and adjusted by the user, using user settings module, which enables the user to define his/her sets of symbols, signs and/or functions represented by the keyboard.
  • the graphical representation (vector drawing) of the user fingers is marked by 111 for left and 112 for right hand.
  • Polygon 104 represents the part of user interface showing the text typed so that the user can see what is being typed.
  • the module for user settings enables the user to define his/her sets of symbols, signs and/or functions performed, so that the user can choose the number of sets of signs and functions as well as their content.
  • the content can include different scripts (Arabic, Chinese, Latin, etc) or symbols (mathematical symbols, musical notes) or functions (piano keys).
  • the module also enables the user to choose the manner of feedback when the active set is changed (graphically, with sound, with voice, with vibration), the shape and style of the user interface, the size of font, the sensitivity of the accelerometer (also to the point of switching it off) and the number of axes taken into consideration for changing of the active set.
  • the driver software needs to be installed on the electronic device.
  • the keys 102, 103, la to 4a and lb to 4b are distributed as seen in figures 1 to 3.
  • the figure 6 shows the position of keys la to 4a and lb to 4b in second implementation.
  • the lower side of the either an electronic device or external unit has two symmetrically placed recesses 202.
  • the recesses are placed so that, when the user holds the device with his/her hands the tips of the fingers touch the keys la to 4a and lb to 4b.
  • the keys la to 4a and lb to 4b are placed in the outside lateral sides 203 of the recesses 202.
  • the outside lateral sides are either vertical or at a smaller angle to the bottom side of the device or external unit.
  • the keys can be either micro switches or parts of the touch sensitive surface.
  • the user holds the device with open palms or with finger tips, with all fingers available to operate the keys. The user can therefore quickly access 50 keys, which enables fast typing on a small device. Every finger has one key and the keys are placed so that they can easily be activated when the device is hand-held, as there are at least 8 from the bottom and 2 from the top side of the device.
  • the driver software can detect 5 positions. As each position corresponds to 10 keys, the user can type 50 signs or use 50 functions without removing his/her fingers from the keys, which makes typing faster and easier.
  • the adjustment of the program enables the change of measuring angles from only one to 3 different axes, depending of the speed of typing one wishes to achieve. To simplify graphical representation on 2D paper, all figures in this description show movement regarding one axis.
  • Figures 1 to 3 show tablet PC 100. It has 10 keys for 8 fingers and 2 thumbs. Eight keys on the bottom side are physical and two 102 and 103 on the top/front side of the device 100 can be either physical or virtual using touch screen technology. In case of virtual keys, the number is not limited to only 2 keys and the maximum primarily depends on the mobility of thumbs. The number can be set using the options of the user settings module. The keys are placed to enable both holding the device with fingertips and typing, where one embodiment of present invention is shown in figure 6.
  • Figures 1 and 2 show the screen 101 of the electronic device with graphical interface.
  • Graphical interface has three elements:
  • Part 104 of the user interface is used to show text that is typed.
  • Sets 105 to 109 of signs have 10 signs each i.e. the program assigns 10 signs to each set.
  • the symbolic representation of the fingers and thumbs is market with 1 11 and 112.
  • the part of the interface showing the text that was typed is marked with 104.
  • the Figure 1 shows the device at the moment of typing the letter S.
  • the accelerometer readings dependant of the devices physical position, the graphic user interface looks as shown in Figure 1.
  • the active set 107 has a different shape in comparison to the inactive sets.
  • the set 107 consists of signs AH, SJ, DK, FL and GC.
  • the interface shows the set of signs 111 on the left and 112 on the right side of the screen.
  • the thumbs can type "A" pressing the key 103 and "H” pressing the key 102.
  • Typing S, D, F or G and J, K, L or C from the active set 107 is achieved by pressing the keys la, 2a, 3a or 4a and lb, 2b 3b or 4b on the bottom side of the device 100.
  • User uses his/her index finger of the left hand to activate key la on the bottom side of the device.
  • the background of the letter S on the graphical representation of the user's index finger changes color (becomes darker), thus showing the user that the switch has been activated and that it currently behaves as a letter S on the keyboard.
  • the program writes "S" on the part of the user interface intended for control of typing (104) and the text typed in this step - "S” is forwarded to the operating system and further on to another application.
  • the graphical representation is marked and the letter "S” displayed on the screen 104 dynamically and in real time.
  • Figure 2 shows tablet PC or another device with the screen of the System with graphic interface when the device is tilted with regard to one axis. So, to activate the set QZ, WU, EI, O, TP it is sufficient to rotate the device to the right. Accelerometers in the device are very precise and able to detect even the smallest changes of the angle, so the device is able to adjust as soon as the change occurs.
  • the changed interface is shown in Figure 2.
  • the active set is distinctly colored or otherwise marked so that it is different from inactive sets.
  • the user interface 104 is corrected so that it is tilted by the same angle so that it remains parallel to x and y axes so that it seems anchored to the user, which makes reading easier, while other elements are tilted. So, having typed "S" the user turns the device to the right for certain angle K.
  • the beforehand the user has adjusted the trigger sensitivity to the range of degree values sufficient to change the sign set.
  • the program redraws the user interface.
  • set 106 corresponding to the keys QZ, WU, EI, RO, TP is activated.
  • the program changes visual representation of both set 107 (no longer active) and set 106 (newly active), so that the user can note the change and intuitively link it to the turning of the device.
  • Set 106 becomes active and now looks different then inactive sets.
  • program signals the operating system to start vibration which warns the user of the change, thus enabling blind typing.
  • the program shows new functions or symbols at the graphic representation of the user's digits.
  • program adjusts the tilt of the typed text representation under a new angle (no longer parallel to the edge of the device. This part of the graphical interface seems anchored and their edges parallel to the same x and y axes in space ( Figure 2).
  • the change of the active set is announced to the user in several ways and each type of announcement improves intuitive understanding of the device.
  • the user has in this step noticed the change of the active set and new interpretation of the keys.
  • the user activates keys in following order:
  • the part 104 of the graphic user interface in charge of displaying the typed text shows "STOP", this word is in turn sent to another application.
  • Figure 4 shows the algorithm of the driver software. All of the top and bottom side keys are connected to the electronic module shown as "keypad controller". The controller translates the state of each key, sent as binary digit 0 or 1 into signal compatible with serial communication, such as USB. To simplify this explanation, the chart shows USD communication, although in most applications, the module or even keys themselves would be integrated on the devices motherboard. In either case, such signal comes as input to the operating system of the device.
  • motion sensors are very precise and enable detection of position and movement in regard to 3 or even 6 axes. To simplify the explanation, we will talk about detecting movement regarding one axis, while most practical uses would probably include 3 or 6 axes. Movement sensors are already included in some of the target devices (for example Apple Ipad, Apple IPhone) and their signals can be used as entry variables for this system.
  • target devices for example Apple Ipad, Apple IPhone
  • Program processes information obtained from the keypad controller and motion sensors and passes the result (active sign set and active sign) to the:
  • the main module of the program based on the position of the device and the state of the keys determines which symbols are chosen, taking into consideration the user-defined symbol set, having checked first which keys were actually pressed (ACTIVE KEYS DATA).
  • the program acquires data about active sign set (ACTIVE SIGN SET DATA) and, processing this information, produces graphical instructions and forwards them to the screen of the device.
  • Rotation of the element 104 is directly proportional to the physical rotation of the device and the change of shape of the graphic interface elements showing the character sets (105, 106, 107, 108 and 109) is determined by the position of the device.
  • the System and Method for Controlling Functions of and Entering Data in Electronic Devices contains the following steps:
  • Number of ranges should correspond to the number of sign sets used, c) defining the correspondence between sign sets 1 to m and position ranges Kl to Kn, where m can either be equal to Kn or a multiple of Kn, d) registering the position of the device using one or more accelerometers and reading of the Kl to Kn ranges, e) based on the registered position of the device and read Kl to Kn values, activation of the individual 1 to m sign set and, at the same time, assigning the active functions and signs to the keys la to 4a, lb to 4b, as well as 103 and 102, and f) simultaneously with the previous step, graphic representation on the screen 101 of the assigned functions or signs from the currently active sign set.
  • the steps in which the preprogrammed functions and/or number of signs 1-m and their content and range of values Kl to Kn includes saving of the sign sets and dependence of the activation of each sign set on the range of values Kl and Kn as well as saving the assignments of signs in the set to particular keys.
  • the step in which position of the device is registered according to Kl to Kn ranges includes activation and access to the particular data set and also showing all signs on the screen 101 in real time.
  • Activating chosen key, to which certain sign is functionally and graphically assigned triggers the display of this sign on the interface element 104.
  • the information about activity of each dataset can be either graphical and or sound and/or vibration signal. Graphic signal would consist of the change in shape and/or color and/or font of the active sign set 1 to m and also the chosen individual sign.
  • the method adjusted to function access and data entry for tablet PCs and/or mobile communication devices with 8 physical keys on the bottom side and 2 on the top side ( Figure 2, 103 and 102 and Figure 3: la-4a, lb-4b).
  • the method is applicable also to the configurations with 8 physical keys on the bottom side with 2 virtual keys on the top side, on a touch screen or with all virtual keys in cases when the back of the device is touch sensitive.
  • Each specific device can still use this method implemented as program code.
  • Reading of the user-defined parameters containing the following customizable parameters a. Number and content of the sign sets ( Figure 1: 105-109, Figure 2: 105-109). Such sets could include Chinese, Arabic, Cyrillic, English or another set of letters, set of signs especially adjusted to a specific application (a synthesizer) or another set of signs or predefined functions. One such set would consist of 10 or more signs or functions, one for each key ( Figure 2: 103 and 102, Figures 3-6 la-4a, lb-4b). In case of virtual keys for thumbs, the number of those is limited by their flexibility. Still, for simplicity, further text and figures will describe the method in case the sets consist of up to 10 signs.
  • the graphic interface will show in an understandable way which sign or function belongs to which key ( Figure 1 : 111, 112) and which sign or function is currently active (the sign with dark background).
  • b. The range of values K on axes x, y and z defined for each set of signs. This means that each set of signs will correspond to the exactly determined range of positions in space, which will be read by accelerometer ( Figure 5), marked by range of x, y and z coordinates so that for each range of positions, different set is active.
  • Style template or skin of the user interface. This group of user-defined parameters adds primarily intuitive and esthetic value to the user and adds/removes certain elements of feedback (i.e. switches sound feedback on or off).
  • Graphic signal can be the change of shape of the active set, as seen in figures 1 and 2 or change of color or font of the active set.
  • the sound signal would be playing of the sound file and the vibration would be realized by activating vibration present in the large majority of mobile devices. This vibration signal is very important as it enables blind typing.
  • the System and Method for Controlling Functions of and Entering Data in Electronic Devices can be used as data entry device for computerized mobile or fixed devices such as mobile telephones, beepers, personal digital assistants (PDA) and others.
  • PDA personal digital assistants
  • the system described enables quick data entry trough the position of keys and movement of the device.
  • the invention enables a large number of sign sets, different modes of writing (for example Chinese, Japanese, Arabic) but also use for different signs in some applications (Emulating piano keys for music applications, emulation of function keys for quick shortcuts).
  • Program also includes a module enabling the user to define own sets of symbols, signs and functions that may be assigned to the keys.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Input From Keyboards Or The Like (AREA)

Abstract

System and Method for Controlling Functions of and Entering Data in Electronic Devices and computer software implementing this system and method can be used as data entry device for entering textual or other signs for mobile or fixed computerized devices, such as mobile phones, beepers, personal digital assistants (PDA) and similar. The System enables users to enter data quickly using conveniently placed keys and application that monitors motion of the device itself. The System for Controlling Functions of and Entering Data in Electronic Devices has at least 10 keys, so that at least 8 (1a, 2a, 3a, 4a, 1b, 2b, 3b and 4b) are located on the bottom/back side of the device, so that their position corresponds to 8 fingers and at least two (102) and (103) are located on the top/front side of the device in positions appropriate for thumbs. Furthermore, the System has a screen (101) with graphical user interface with the part showing the text typed (104), part showing the grouped sets of (105 to 109) signs 1 to m and graphic representation of the users fingers with functions and signs for each finger and thumb on left (111) and right (112) hand. The change of currently active sign set is shown graphically and/or by sound and/or by vibration.

Description

System and method for control of functions and data entry in electronic devices, and computer program for implementing said system and method
DESCRIPTION Technical field
Present invention is a system and method for control of functions and data entry data in electronic devices and computer software implementing this system and method. More specifically it is a system that enables communication with mobile electronic devices, more specifically a keypad which enables quick data entry through the position of keys and through detecting the motion of the device. The driver of the device reads the movement of the device and interprets the movement. The movement is detected using accelerometers or equivalent means located in the device itself. The combination of keys arrangement and accelerometers and appropriate preprogrammed graphic representation enables users to enter data simply and quickly.
Technical problem
The invention solves the technical problem of typing with 10 fmgers on a small hand-held device, without using an additional surface where the device can be placed. Present system and method for data entry discloses combination of keys on the up and downsides of the device with recording of movement of the device which is held with both hands and computer program for implementing said system and method. Therefore the lack of keyboard space for tablet PCs or smartphones (mobile communication devices) is solved and data entry made faster and easier.
Second objective of the invention is showing graphically the position of the 8 fingers placed on the bottom side of the device and thumbs on the upper side and attributing dynamically and in real time a function or a sign to each finger-key. In this way there is no need for the user to memorize the position of the keys, which is necessary for the standard keyboard types.
Third objective of the invention is that user can define his/her sets of symbols, signs and functions attributed to the keys, using the program module described below.
Fourth objective of the present invention is that the active character set needs to be quickly and simply changed and shown dynamically and in real time. The device contains not only keys on the top and bottom sides of the device, but also an accelerometer or its equivalent means and specialized driver program which, based on the measurement of the inclination (tilt) of the device, shows graphically the position of 8 fingers on the downside, thumbs on the upper side, descriptions of the current key attributions and changes those according to the changes of the inclination of the device for at least 8 keys on the lower and 2 keys on the upper side of the device. The implementation of the keys can be hardware or software based.
Background art
Tablet computer or tablet PC is a completely new category of digital computer created in the beginning of 2010. Development of this type of computer was attempted earlier, but was rejected by the market due to poor technical realization. The key advancement came from American company Apple where the success of their Ipad shows that there is a need for such a device. A majority of other computer manufacturers are also preparing or have ready devices in this category. The main feature of these computers, separating them from notebooks, is a lack of physical keyboard. Furthermore, in the past several years there was a huge development of mobile communication devices occupying the area between computers and mobile phones. One of the main problems shared between tablet PCs and mobile communication devices is the problem of effective typing on a very small device, as can be easily seen on Ipad.
The proposed solutions for typing fall into two main groups:
1. virtual keyboards, and
2. physical keyboards.
Both solutions have advantages and drawbacks:
1. Virtual keyboards lack precision and a„click" that can be felt. The solution to this problem attempted by some producers, such as American RIM, was attempted through the feedback in which pressing of the key is equivalent to pressing of the entire keyboard/screen.
It is much more difficult to learn blind typing using a virtual keyboard and the speed of typing is probably the most important issue.
2. Physical keyboards solve these problems but take up more space and there assembly is often very complicated.
Both solutions have an additional drawback in common, in the best-case scenario, as with IPhone in landscape mode, only thumbs are used for typing.
System and method according to the present invention solve the following technical problems: 1. the user gets„click" feedback for 8-10 digits, depending on the implementation, and not only for 1 as in RIM's solution,
2. all 10 digits are used for typing, so great speed can be achieved, which is currently the main advantage of desktop PCs,
3. there is no need for complicated assembly,
4. functions assigned to keys are attributed dynamically, so it is possible to implement unlimited number of localized/specialized keyboards on different markets, which is an advantage of the virtual keyboards, and
5. the device shows graphically in real time the position of 8 fingers on the downside, thumbs on the upper side, and descriptions of the current key attributions, changes those according to the changes of the inclination of the device for at least 8 keys on the lower and 2 keys on the upper side of the device. The keys themselves can be hardware based or virtual.
Detailed description of the invention
The invention deals with the system and method for controlling functions of and entering data in electronic devices. The electronic devices in question can be mobile electronic devices or external units connected with a mobile electronic device. Electronic devices can be tablet PCs, smartphones, MID (mobile internet device), PDA and small mobile computers.
The conditions for realization of the invention are described below.
Electronic device is produced with the touch-sensitive screen, without physical keyboard with graphical user interface and at least 8 appropriately placed keys on the bottom side of the device and at least two keys on the upper side, which should ergonomically correspond to thumbs.
Regarding the realization, the following modes are possible:
electronic device with 8 physical keys on the bottom side and two physical keys on the upper side; electronic device with 8 physical keys on the lower side and 2 or more virtual dynamically drawn keys on the upper side. Some positions/tilts of the device regarding measured axes can have 2, others 4 virtual keys. Maximal number of keys is limited by the flexibility of thumbs. The information about the number of keys possible is one of the device drivers' options; smartphone mobile telephone, with 8 physical keys; smartphone mobile telephone, with 10 physical keys; electronic device with touch sensitive downside and touch screen on the upper side. In this implementation all keys are virtual and their size, number and position are determined by the user in driver's options.
System and Method for Control of Functions and Data entry in Electronic Devices contains at least the following elements:
1. Hardware of the electronic device consists of a MID (mobile internet device). This is a new category of the mobile computer, which includes, for example, Apple Ipad. It does not have a standard keyboard, but a monitor, battery, accelerometer, memory, processor and other elements of the computer all packed in a single, light and compact case. The MID's case has installed at least 10 keys distributed so that 8 of these are located on the bottom (back) side of the device (figure 3: 1a, 2a, 3 a, 4a, lb, 2b, 3b and 4b) so that their positions correspond to 8 fingers and at least 2 keys 102 and 103 are located on the upper (front) side of the device (figure 1, figure 2) so that they can be operated by the thumbs when MID is held with open palms. The 8 keys on the bottom side are located so that they can be operated with fingers (one with each finger). The keys can be physical (using micro switches), touch-activated (or virtual), on touch screen or on touch- sensitive bottom side of the device or a combination of the physical and touch keys.
2. One or several sensors capable of detecting motion of the device and its position in space, where sensors can be accelerometers.
3. Special driver program - software which, based on the angle of inclination of the device, dynamically in real time draws 8 finger positions on the lower side of the device, thumb positions on the upper side and appropriate descriptions for the current functions of the keys. Based on the position/tilt of the device, the program dynamically allocates functions of at least 8 keys on the bottom side and at least two (virtual or hardware) keys on the upper side. The program also shows a list of all functions (letters, numbers or signs) on the screen, grouped in sets corresponding to the tilts of the device so that active set is shown in a different way from other sets (using color, shape, size or font).
Computer program for implementing system and method for control of functions and data entry in electronic devices is placed on the media that can be read by a computer and consists of the following modules: 1. Graphical user interface module;
2. Module for user settings enabling the user to define own sets of symbols, signs and functions to be attributed to the keys, defining of the active set of signs, defining of the ranges for changing of the active set, defining styles and fonts and their exchange between users or between users and software manufacturers;
3. Module for processing of the key and accelerometer-obtained data and their translation into characters and functions, as described;
4. Exit module for passing of the functions and symbols back to the operating system; and
5. Exit module for passing sound and or vibration signals to the user to provide him/her with feedback.
Signal processing module obtains from the operating system data about active keys and state of the accelerometers i.e. position of the device. Based on these, it dynamically in real time creates functions and symbols and feeds them back to the operating system. The operating system in turn feeds these to other applications in the same manner used for physical and virtual keyboards. The majority of MID's and tablet PC's have vibration capabilities similar to those found in the mobile phones. Thus System can be used for blind typing, which is the fastest way to type in date to the computer. Every change in this chain and every element of interaction are also reflected in the graphical output via graphical user interface module.
Detailed description of drawings
The invention will in the following text be described with reference to the following figures:
- figure 1 : shows the device and its screen with the graphical user interface,
- figure 2: shows the device and its screen with graphical user interface tilted with regard to one axis,
- figure 3 shows the keys on the bottom side of the electronic device,
- figure 4: Shows the system of the present invention,
- figure 5 shows the software algorithm of the present invention,
- figure 6 shows implementation of keys on the bottom side of the device according to another embodiment of invention, - figure 7 shows the front/upper side of the external unit in case where the System and Method for Controlling Functions of and Entering Data in Electronic Devices is realized as the addition to an existing device, and
- figure 8 shows the back/bottom side of the external unit in case where the System and Method for control of functions and data entry in Electronic Devices is realized as the addition to an existing device.
The System and Method for Controlling Functions of and Entering Data in Electronic Devices can be implemented as an integral part of an electronic device or as an external unit for an electronic device. Figures 7-8 show an external unit of the same size as Apple Ipad and connected (for example using dock connector 200, as shown in Figure 7) to it. The external unit could in a similar fashion be connected to another producer's unit. The external unit, which implements the System and Method for Controlling Functions of and Entering Data in Electronic Devices has all of the necessary elements i.e. the 8 keys la, 2a, 3a, 4a, lb, 2b, 3b and 4b on the bottom side of the external unit, placed so that they correspond to the positions of 8 fingers. In case when the System is installed in an existing electronic device, the role of the screen 101 can be played by an existing screen. The 8 keys can be made as shown in figure 3 or, alternatively, as shown in figure 6. The software implementing the System and Method for Controlling Functions of and Entering Data in Electronic Devices shows on the graphical interface of the driver on the screen of the electronic device or the housing of the external unit at least two keys corresponding to the position of the thumbs. The keys themselves can either be micro switches or touch sensitive part of the external unit's housing or touch sensitive screen.
Electronic device can be any mobile electronic device, such as tablet computer, smartphone, MID (mobile internet device), PDA or small mobile computer. Present System and Method for Control of functions and data entry in Electronic Devices has at least 10 keys: 8 keys la, 2a, 3a, 4a, lb, 2b, 3b and 4b are located at the bottom side of the device so that their position corresponds to the 8 fingers and at least two keys 102 and 103 are located on the top side of the device in positions ergonomically appropriate for thumbs. The keys themselves are either micro switches or parts of the surface of the device or external unit that are touch sensitive. The system can be realized so that the device has on the bottom side 8 physical keys and on the top side two or more dynamically drawn virtual keys or so that the electronic devices has on the bottom side at least 8 virtual keys on the touch sensitive surface and on the top side two or more virtual keys on a touch screen. The same variants of the keys and physical and virtual keys combinations can be used on the housing of the System realized as an external unit. Furthermore, the System includes one or more motion sensors necessary to detect the position of the device itself and information about its movement with regard to surroundings. These sensors can be accelerometers or another type of sensor with the same function. The System also has screen 101 witch is used as user interface, showing the text typed, part of the graphical interface with grouped sets of 105 to 109 signs 1 to m and graphical representation of the position of users hands with functions and signs for each finger for left hand 111 and for right hand 1 12, with the active set of signs marked specially. The interface 104 is anchored to the coordinate set so that, regardless of the System's position regarding the x-axis the interface 104 is always parallel to it. The drawings of the user's fingers and functions currently attributed to them, from the currently active set, are shown in real time, the changes of the active set are communicated to the user by touch (vibration system of the device), visually and with appropriate sound.
The elements of the screen 101 are shown in figures 1 and 2, where different sign sets are marked with 105, 106, 107, 108 and 109. The sign sets are chosen and adjusted by the user, using user settings module, which enables the user to define his/her sets of symbols, signs and/or functions represented by the keyboard. The graphical representation (vector drawing) of the user fingers is marked by 111 for left and 112 for right hand. Polygon 104 represents the part of user interface showing the text typed so that the user can see what is being typed.
The module for user settings enables the user to define his/her sets of symbols, signs and/or functions performed, so that the user can choose the number of sets of signs and functions as well as their content. The content can include different scripts (Arabic, Chinese, Latin, etc) or symbols (mathematical symbols, musical notes) or functions (piano keys). The module also enables the user to choose the manner of feedback when the active set is changed (graphically, with sound, with voice, with vibration), the shape and style of the user interface, the size of font, the sensitivity of the accelerometer (also to the point of switching it off) and the number of axes taken into consideration for changing of the active set.
The driver software needs to be installed on the electronic device. The keys 102, 103, la to 4a and lb to 4b are distributed as seen in figures 1 to 3. The figure 6 shows the position of keys la to 4a and lb to 4b in second implementation. In this variant, the lower side of the either an electronic device or external unit has two symmetrically placed recesses 202. The recesses are placed so that, when the user holds the device with his/her hands the tips of the fingers touch the keys la to 4a and lb to 4b. In this embodiment, the keys la to 4a and lb to 4b are placed in the outside lateral sides 203 of the recesses 202. The outside lateral sides are either vertical or at a smaller angle to the bottom side of the device or external unit. The keys can be either micro switches or parts of the touch sensitive surface. The user holds the device with open palms or with finger tips, with all fingers available to operate the keys. The user can therefore quickly access 50 keys, which enables fast typing on a small device. Every finger has one key and the keys are placed so that they can easily be activated when the device is hand-held, as there are at least 8 from the bottom and 2 from the top side of the device.
The driver software can detect 5 positions. As each position corresponds to 10 keys, the user can type 50 signs or use 50 functions without removing his/her fingers from the keys, which makes typing faster and easier. The adjustment of the program enables the change of measuring angles from only one to 3 different axes, depending of the speed of typing one wishes to achieve. To simplify graphical representation on 2D paper, all figures in this description show movement regarding one axis.
Figures 1 to 3 show tablet PC 100. It has 10 keys for 8 fingers and 2 thumbs. Eight keys on the bottom side are physical and two 102 and 103 on the top/front side of the device 100 can be either physical or virtual using touch screen technology. In case of virtual keys, the number is not limited to only 2 keys and the maximum primarily depends on the mobility of thumbs. The number can be set using the options of the user settings module. The keys are placed to enable both holding the device with fingertips and typing, where one embodiment of present invention is shown in figure 6.
Furthermore, Figures 1 and 2 show the screen 101 of the electronic device with graphical interface. Graphical interface has three elements:
1. Representation of all signs, with the active set especially colored or otherwise marked (105 to 109);
2. Symbolic representation 111 and 112 of all 10 fingers and thumbs, with currently assigned functions of the keys from the currently active set; and
3. Part 104 of the user interface is used to show text that is typed.
Sets 105 to 109 of signs have 10 signs each i.e. the program assigns 10 signs to each set. The symbolic representation of the fingers and thumbs is market with 1 11 and 112. The part of the interface showing the text that was typed is marked with 104.
The Figure 1 shows the device at the moment of typing the letter S. The accelerometer readings, dependant of the devices physical position, the graphic user interface looks as shown in Figure 1. According to the accelerometer readings, the active set 107 has a different shape in comparison to the inactive sets. The set 107 consists of signs AH, SJ, DK, FL and GC. When set 107 is activated, the interface shows the set of signs 111 on the left and 112 on the right side of the screen. The thumbs can type "A" pressing the key 103 and "H" pressing the key 102. Typing S, D, F or G and J, K, L or C from the active set 107 is achieved by pressing the keys la, 2a, 3a or 4a and lb, 2b 3b or 4b on the bottom side of the device 100. User uses his/her index finger of the left hand to activate key la on the bottom side of the device. As a result, the background of the letter S on the graphical representation of the user's index finger changes color (becomes darker), thus showing the user that the switch has been activated and that it currently behaves as a letter S on the keyboard. At the same time, the program writes "S" on the part of the user interface intended for control of typing (104) and the text typed in this step - "S" is forwarded to the operating system and further on to another application. So, when key la is pressed, the graphical representation is marked and the letter "S" displayed on the screen 104 dynamically and in real time. Figure 2 shows tablet PC or another device with the screen of the System with graphic interface when the device is tilted with regard to one axis. So, to activate the set QZ, WU, EI, O, TP it is sufficient to rotate the device to the right. Accelerometers in the device are very precise and able to detect even the smallest changes of the angle, so the device is able to adjust as soon as the change occurs. The changed interface is shown in Figure 2. The active set is distinctly colored or otherwise marked so that it is different from inactive sets. The user interface 104 is corrected so that it is tilted by the same angle so that it remains parallel to x and y axes so that it seems anchored to the user, which makes reading easier, while other elements are tilted. So, having typed "S" the user turns the device to the right for certain angle K. The beforehand, the user has adjusted the trigger sensitivity to the range of degree values sufficient to change the sign set. The program redraws the user interface. Several changes happen at this stage: set 106 corresponding to the keys QZ, WU, EI, RO, TP is activated. The program changes visual representation of both set 107 (no longer active) and set 106 (newly active), so that the user can note the change and intuitively link it to the turning of the device. Set 106 becomes active and now looks different then inactive sets. Also, if it is set this way, program signals the operating system to start vibration which warns the user of the change, thus enabling blind typing. With turning of the device to the right for K degrees the program shows new functions or symbols at the graphic representation of the user's digits. Also, program adjusts the tilt of the typed text representation under a new angle (no longer parallel to the edge of the device. This part of the graphical interface seems anchored and their edges parallel to the same x and y axes in space (Figure 2).
The change of the active set is announced to the user in several ways and each type of announcement improves intuitive understanding of the device. The user has in this step noticed the change of the active set and new interpretation of the keys. The user activates keys in following order:
1. Pinky on the left hand activates key 4b on the bottom side of the device and types„T". This symbol is sent to the operative system of the device and further on to the active application;
2. Ring finger of the right hand activates key 3b on the bottom side of the device and types„0". This symbol is than sent to the operative system of the device and further on to the active application; 3. Pinky of the right hand activates key 4b on the bottom side of the device and types„P". This symbol is than sent to the operative system of the device and further on to the active application.
As a result, the part 104 of the graphic user interface in charge of displaying the typed text shows "STOP", this word is in turn sent to another application.
Figure 4 shows the algorithm of the driver software. All of the top and bottom side keys are connected to the electronic module shown as "keypad controller". The controller translates the state of each key, sent as binary digit 0 or 1 into signal compatible with serial communication, such as USB. To simplify this explanation, the chart shows USD communication, although in most applications, the module or even keys themselves would be integrated on the devices motherboard. In either case, such signal comes as input to the operating system of the device.
Another necessary variable comes from motion sensors. These sensors are very precise and enable detection of position and movement in regard to 3 or even 6 axes. To simplify the explanation, we will talk about detecting movement regarding one axis, while most practical uses would probably include 3 or 6 axes. Movement sensors are already included in some of the target devices (for example Apple Ipad, Apple IPhone) and their signals can be used as entry variables for this system.
Program processes information obtained from the keypad controller and motion sensors and passes the result (active sign set and active sign) to the:
1. screen, in graphic form, as vector representation of user's fingers, animation of the change of active sign set, marking of active sign set by special shape, font or color, as a display of available signs and functions on the representations of user's fingers and thumbs, which change with turning/tilting of the device,
2. to the operating system, which in turn interprets these signs as signs from the keyboard, forwarding them to other applications.
This describes how the program gets information about the keys pressed and position of the device.
There is a great number of different keyboards, different writing systems (Chinese, Japanese, Arabic) and also different needs posed by applications (Musical application could use it to emulate piano keys, business applications to emulate function keys to enable shortcuts), program has a module enabling the user to define own symbols, signs and functions performed by the keypad.
The main module of the program, based on the position of the device and the state of the keys determines which symbols are chosen, taking into consideration the user-defined symbol set, having checked first which keys were actually pressed (ACTIVE KEYS DATA).
These are than sent in two modules: 1. as codes to the operating system, to be forwarded to application, and
2. as graphic instructions to the graphic controller so that the chosen sign can change color, font or shape, providing feedback to the user.
In parallel, the program acquires data about active sign set (ACTIVE SIGN SET DATA) and, processing this information, produces graphical instructions and forwards them to the screen of the device.
As the currently active data set (Figure 1 : 107, Figure 2: 106) depends on the tilt of the device, visual presentation of the connection improves intuitive understanding of the keypad, as graphical representation changes in real time in relation to the tilt of the device. This is achieved by counter-rotation of the graphic interface elements 104 (Figures 1 and 2), which, from the user perspective looks as though all elements of the GUI have rotated except for element 104 and elements have changed shape (Figure 1 : 107, Figure 2: 106).
Rotation of the element 104 is directly proportional to the physical rotation of the device and the change of shape of the graphic interface elements showing the character sets (105, 106, 107, 108 and 109) is determined by the position of the device.
The System and Method for Controlling Functions of and Entering Data in Electronic Devices contains the following steps:
a) defining preprogrammed functions and/or number of signs 1 - m and content of sign sets appropriate for each application, b) defining range of values l to Kn for the device positions in space, in regard to the x, y and z axes. Number of ranges should correspond to the number of sign sets used, c) defining the correspondence between sign sets 1 to m and position ranges Kl to Kn, where m can either be equal to Kn or a multiple of Kn, d) registering the position of the device using one or more accelerometers and reading of the Kl to Kn ranges, e) based on the registered position of the device and read Kl to Kn values, activation of the individual 1 to m sign set and, at the same time, assigning the active functions and signs to the keys la to 4a, lb to 4b, as well as 103 and 102, and f) simultaneously with the previous step, graphic representation on the screen 101 of the assigned functions or signs from the currently active sign set. The steps in which the preprogrammed functions and/or number of signs 1-m and their content and range of values Kl to Kn includes saving of the sign sets and dependence of the activation of each sign set on the range of values Kl and Kn as well as saving the assignments of signs in the set to particular keys. The step in which position of the device is registered according to Kl to Kn ranges includes activation and access to the particular data set and also showing all signs on the screen 101 in real time. Activating chosen key, to which certain sign is functionally and graphically assigned, triggers the display of this sign on the interface element 104. The information about activity of each dataset can be either graphical and or sound and/or vibration signal. Graphic signal would consist of the change in shape and/or color and/or font of the active sign set 1 to m and also the chosen individual sign.
In the following text, to simplify explanation, the method adjusted to function access and data entry for tablet PCs and/or mobile communication devices with 8 physical keys on the bottom side and 2 on the top side (Figure 2, 103 and 102 and Figure 3: la-4a, lb-4b). The method is applicable also to the configurations with 8 physical keys on the bottom side with 2 virtual keys on the top side, on a touch screen or with all virtual keys in cases when the back of the device is touch sensitive. Each specific device can still use this method implemented as program code.
The System and Method for Controlling Functions of and Entering Data in Electronic Devices will be implemented as described below.
1. Reading of the user-defined parameters containing the following customizable parameters: a. Number and content of the sign sets (Figure 1: 105-109, Figure 2: 105-109). Such sets could include Chinese, Arabic, Cyrillic, English or another set of letters, set of signs especially adjusted to a specific application (a synthesizer) or another set of signs or predefined functions. One such set would consist of 10 or more signs or functions, one for each key (Figure 2: 103 and 102, Figures 3-6 la-4a, lb-4b). In case of virtual keys for thumbs, the number of those is limited by their flexibility. Still, for simplicity, further text and figures will describe the method in case the sets consist of up to 10 signs. The graphic interface will show in an understandable way which sign or function belongs to which key (Figure 1 : 111, 112) and which sign or function is currently active (the sign with dark background). b. The range of values K on axes x, y and z defined for each set of signs. This means that each set of signs will correspond to the exactly determined range of positions in space, which will be read by accelerometer (Figure 5), marked by range of x, y and z coordinates so that for each range of positions, different set is active. c. Style, template or skin of the user interface. This group of user-defined parameters adds primarily intuitive and esthetic value to the user and adds/removes certain elements of feedback (i.e. switches sound feedback on or off).
2. Reading of the x, y and z coordinates using motion sensors - accelerometers (Figure 5).
3. Sending graphic, sound or vibration message to the user regarding the activation of the sign set, based on data in previous two steps. Graphic signal can be the change of shape of the active set, as seen in figures 1 and 2 or change of color or font of the active set. The sound signal would be playing of the sound file and the vibration would be realized by activating vibration present in the large majority of mobile devices. This vibration signal is very important as it enables blind typing.
4. Based on the active set and status of the keys (Figure 2: 103 and 102, Figure 3: la-4a, lb-4b) determining the function assigned to the key which is activated.
The change of background color on the graphic interface for the currently active signs or functions, forwarding that function or sign to the active application, as well as sending the user graphic, sound or vibration alert are all based on user preferences. Graphic signal about the chosen letter or function is grouped with the previously chosen ones, thus forming text displayed in part 104 of the graphic interface. This part of the interface is anchored regarding the coordinates 113 (Figure 2), so that, regardless of the angle of the device to x axis, the edge of the graphic interface element 104 is always parallel to x and y axes (Figure 2), while other elements of the graphic interface rotate with the device. For this to be possible, the accelerometer information is necessary: the obtaining of that information was explained before.
Possible industrial or other applications
The System and Method for Controlling Functions of and Entering Data in Electronic Devices can be used as data entry device for computerized mobile or fixed devices such as mobile telephones, beepers, personal digital assistants (PDA) and others. The system described enables quick data entry trough the position of keys and movement of the device. The invention enables a large number of sign sets, different modes of writing (for example Chinese, Japanese, Arabic) but also use for different signs in some applications (Emulating piano keys for music applications, emulation of function keys for quick shortcuts). Program also includes a module enabling the user to define own sets of symbols, signs and functions that may be assigned to the keys.

Claims

1. A system for control of functions and data entry in electronic devices which has switches used as keys, keypad controller, operating system of the electronic device, display controller and graphical user interface, vibration system controlled by a controller characterized by its implementation either as an integral part of the electronic device or as an external unit, where said system comprises the following elements:
a) at least 10 keys placed so that at least 8 (l a, 2a, 3a, 4a, lb, 2b, 3b and 4b) are on the bottom/back side of the device so that their position correspond to the position of 8 fingers excluding thumbs, and 2 keys (102 and 103) placed on the top/front side of the device in position economically appropriate for thumbs;
b) one or several motion sensors necessary to detect position of the device and supplying information about its position;
c) screen (101) comprising graphical user interface (104) showing the text that has been typed; part of the graphic interface showing the grouped sets (105 to 109) of signs 1 to m and graphic display of the position of the user's fingers with plotted functions or signs for each finger of left hand (111) and right hand (112), where currently active set is especially marked.
2. System according to claim 1, characterized by the fact that the electronic device is a mobile electronic device.
3. System according to claim 1, characterized by the fact that the key is either a micro switch or a part of a touch sensitive surface.
4. System according to claim 1, characterized by the fact that said device has on the bottom/back side 8 physical keys and on the top/front side two physical keys.
5. System according to claim 1, characterized by the fact that said device has on the bottom/back side 8 physical keys and on the top/front side two dynamically drawn virtual keys.
6. System according to claim 1, characterized by the fact that said device has on the bottom/back side at least 8 virtual keys on a touch sensitive surface and on the top/front side two dynamically drawn virtual keys.
7. System according to any of preceding claims, characterized by the fact that the bottom side has two symmetrically placed recesses (202) so that the keys (la to 4a) and (lb to 4b) are placed on the outside lateral walls (203) of the recesses (202).
8. System according to claim 7, characterized by the fact that outside lateral walls (203) of the recesses (202) are either perpendicular or at an angle to the bottom side of said device or external unit.
9. System according to claim 7, characterized by the fact that the keys (la to 4a) and (lb to 4b) are either micro switches or parts of the surface that is touch sensitive.
10. System according to claim 1, characterized by the fact that motion detectors are accelerometers.
11. System according to claim 1 , characterized by the fact that the part of the graphical user interface (104) is anchored in regard to the coordination system in the sense that, regardless of the angle of device with regard to the x axis, the edge of the interface (104) is always parallel to the x axis.
12. System according to claim 1, characterized by the fact that the current position of the user's fingers and signs from the active signs set is graphic and/or sound and/or vibration.
13. System according to claim 2, characterized by the fact that the electronic devices are tablet PC, smartphones, MID, PDA and small portable computers.
14. System according to claim 1, characterized by the fact that the external unit implementing said system and method for control of functions and data entry for electronic devices is connected to the said electronic device using connector (200) where the external unit has keys (la to 4a and lb to 4b).
15. System according to claim 14, characterized by the fact that the external unit is connected to the electronic device so that the signals about which key is activated is sent to the operational system of the said device and combines with signals about the position of the device from the movement sensor in the device itself and with the aid of software thus realizes the possibility of typing.
16. A method for control of functions and data entry in electronic devices, characterized by the fact that it involves the following steps:
a) defining of the preprogrammed functions and/or sets of signs 1 to m and content of the sign sets appropriate for an application,
b) defining the range of values Kl to Kn describing the position of the device in space, according to the x, y and z axes, where the number of values l to Kn defined depends on the defined number of sign sets,
c) defining the correspondence between sign sets 1 to m and position ranges Kl to Kn, where m can either be equal to Kn or a multiple of Kn,
d) registering the position of the device using one or more accelerometers and simultaneously reading of the Kl to Kn values, e) based on the registered position of the device and read Kl to Kn values, activation of the individual 1 to m sign set and, at the same time, assigning the active functions or signs to the keys (la to 4a, lb to 4b), as well as (103) and (102), and
f) simultaneously with the previous step, graphic representation on the screen (101) of the assigned functions or signs from the currently active sign set.
17. Method according to claim 16, characterized by the fact that step of defining preprogrammed functions and or number of sign sets 1-m and their content and step defining range of values Kl to Kn include saving of the sign sets and dependence of the activation of each sign set on the range of values Kl to Kn as well as saving a data of signs assignment from each sign set to the particular keys.
18. Method according to claim 16, characterized by the fact that the step in which position of said device is registered depending on the read Kl to Kn values includes activation and access to the particular data set and also displaying all signs on the screen (101) in real time.
19. Method according to claims 16 to 18, characterized by the fact that the activation of the chosen key to which certain sign is functionally and graphically assigned by the means of graphic controller, triggers the display of this sign on the interface (104).
20. Method according to claim 16, characterized by the fact that the information of activity of each 1 to m sign set or particular sign can be either graphical and/or sound and/or vibration signal.
21. T Method according to claim 20, characterized by the fact that the graphic signal consists of its change in shape and/or color and/or font of the active sign set 1 to m, or respectively of the chosen individual sign.
22. Computer software for implementation of the system and method for control of functions and data entry in electronic devices saved on a computer-readable media containing instructions which, when executed by electronic devices operate by the method specified in claims 16-21.
23. Computer software according to claim 22, characterized by the fact that it comprises:
a) graphical user interface module,
b) module for user settings enabling the user to define own sets of symbols, signs and functions to be attributed to the keys, defining of the active set of signs, defining of the ranges of values for changing of the active set, defining styles and fonts, and their Internet exchange between users or between users and software manufacturers,
c) module for signal processing of the key and accelerometer, and creation of functions and symbols based on the signals according to the described method,
d) exit module for sending functions and symbols back to the operating system, and e) exit module for sending sound and/or vibration signals to provide user with feedback.
PCT/HR2010/000019 2010-06-30 2010-06-30 System and method for control of functions and data entry for electronic devices, and computer program for implementing said system and method WO2012001432A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/HR2010/000019 WO2012001432A1 (en) 2010-06-30 2010-06-30 System and method for control of functions and data entry for electronic devices, and computer program for implementing said system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/HR2010/000019 WO2012001432A1 (en) 2010-06-30 2010-06-30 System and method for control of functions and data entry for electronic devices, and computer program for implementing said system and method

Publications (1)

Publication Number Publication Date
WO2012001432A1 true WO2012001432A1 (en) 2012-01-05

Family

ID=43650045

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/HR2010/000019 WO2012001432A1 (en) 2010-06-30 2010-06-30 System and method for control of functions and data entry for electronic devices, and computer program for implementing said system and method

Country Status (1)

Country Link
WO (1) WO2012001432A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103399694A (en) * 2013-07-25 2013-11-20 深圳市金立通信设备有限公司 Screen display controlling method and mobile terminal
WO2014078706A1 (en) * 2012-11-16 2014-05-22 Loopwirez, Inc. Ergonomic thumb interface for mobile phone, smart phone, or tablet
WO2014076258A1 (en) * 2012-11-15 2014-05-22 Schönleben Oliver Method and device for typing on mobile computing devices
US9268364B2 (en) 2012-08-31 2016-02-23 Trewgrip, Llc Ergonomic data entry device

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010048378A1 (en) * 2000-06-06 2001-12-06 Kenichi Horie Qwerty type ten-key board based character input device
US20020118175A1 (en) * 1999-09-29 2002-08-29 Gateway, Inc. Digital information appliance input device
WO2006040400A1 (en) * 2004-10-15 2006-04-20 Nokia Corporation An electronic hand-held device with a back cover keypad and a related method
US20080058033A1 (en) * 2006-09-06 2008-03-06 Sony Ericsson Mobile Communications Ab Portable electronic device
US20090269118A1 (en) * 2008-04-28 2009-10-29 Wilson Robert D Apparatus for ergonomic typing
EP2178277A1 (en) * 2008-10-15 2010-04-21 LG Electronics Inc. Portable terminal and method for controlling output thereof
US7724157B1 (en) * 2007-01-05 2010-05-25 Gray Keith I Keyboard

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020118175A1 (en) * 1999-09-29 2002-08-29 Gateway, Inc. Digital information appliance input device
US20010048378A1 (en) * 2000-06-06 2001-12-06 Kenichi Horie Qwerty type ten-key board based character input device
WO2006040400A1 (en) * 2004-10-15 2006-04-20 Nokia Corporation An electronic hand-held device with a back cover keypad and a related method
US20080058033A1 (en) * 2006-09-06 2008-03-06 Sony Ericsson Mobile Communications Ab Portable electronic device
US7724157B1 (en) * 2007-01-05 2010-05-25 Gray Keith I Keyboard
US20090269118A1 (en) * 2008-04-28 2009-10-29 Wilson Robert D Apparatus for ergonomic typing
EP2178277A1 (en) * 2008-10-15 2010-04-21 LG Electronics Inc. Portable terminal and method for controlling output thereof

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9268364B2 (en) 2012-08-31 2016-02-23 Trewgrip, Llc Ergonomic data entry device
WO2014076258A1 (en) * 2012-11-15 2014-05-22 Schönleben Oliver Method and device for typing on mobile computing devices
CN104903836A (en) * 2012-11-15 2015-09-09 奥利弗.舍恩勒本 Method and device for typing on mobile computing devices
US20150293695A1 (en) * 2012-11-15 2015-10-15 Oliver SCHÖLEBEN Method and Device for Typing on Mobile Computing Devices
US10013163B2 (en) 2012-11-15 2018-07-03 Oliver Schönleben Method and device for typing on mobile computing devices
WO2014078706A1 (en) * 2012-11-16 2014-05-22 Loopwirez, Inc. Ergonomic thumb interface for mobile phone, smart phone, or tablet
CN103399694A (en) * 2013-07-25 2013-11-20 深圳市金立通信设备有限公司 Screen display controlling method and mobile terminal

Similar Documents

Publication Publication Date Title
US9035883B2 (en) Systems and methods for modifying virtual keyboards on a user interface
US8390583B2 (en) Pressure sensitive user interface for mobile devices
JP6115867B2 (en) Method and computing device for enabling interaction with an electronic device via one or more multi-directional buttons
KR101636705B1 (en) Method and apparatus for inputting letter in portable terminal having a touch screen
US20060119582A1 (en) Unambiguous text input method for touch screens and reduced keyboard systems
US8856674B2 (en) Electronic device and method for character deletion
JP2013527539A5 (en)
US20150100911A1 (en) Gesture responsive keyboard and interface
JP5266320B2 (en) Portable device for controlling command execution using an actuator installed on the back side
US20150193011A1 (en) Determining Input Associated With One-to-Many Key Mappings
US20220253209A1 (en) Accommodative user interface for handheld electronic devices
US20120249425A1 (en) Character entry apparatus and associated methods
KR100414143B1 (en) Mobile terminal using touch pad
WO2012001432A1 (en) System and method for control of functions and data entry for electronic devices, and computer program for implementing said system and method
US20140104179A1 (en) Keyboard Modification to Increase Typing Speed by Gesturing Next Character
EP3580646B1 (en) Dynamic space bar
JP6057441B2 (en) Portable device and input method thereof
US20130069881A1 (en) Electronic device and method of character entry
EP2570892A1 (en) Electronic device and method of character entry
KR20100069089A (en) Apparatus and method for inputting letters in device with touch screen
WO2011085553A1 (en) Virtual keyboard
CA2791486C (en) Electric device and method for character deletion
KR101919515B1 (en) Method for inputting data in terminal having touchscreen and apparatus thereof
GB2495384A (en) Keyboard for an electronic device having a key with indicia to indicate the direction of the key function
WO2013048397A1 (en) Electronic device and method for character deletion

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10760402

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE