WO2020006676A1 - 利用触摸软键盘的盲式输入法、计算装置和存储介质 - Google Patents

利用触摸软键盘的盲式输入法、计算装置和存储介质 Download PDF

Info

Publication number
WO2020006676A1
WO2020006676A1 PCT/CN2018/094203 CN2018094203W WO2020006676A1 WO 2020006676 A1 WO2020006676 A1 WO 2020006676A1 CN 2018094203 W CN2018094203 W CN 2018094203W WO 2020006676 A1 WO2020006676 A1 WO 2020006676A1
Authority
WO
WIPO (PCT)
Prior art keywords
input
character
touch
keyboard
input method
Prior art date
Application number
PCT/CN2018/094203
Other languages
English (en)
French (fr)
Inventor
喻纯
王丰
石伟男
范舒翼
王童
史元春
Original Assignee
清华大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 清华大学 filed Critical 清华大学
Priority to PCT/CN2018/094203 priority Critical patent/WO2020006676A1/zh
Publication of WO2020006676A1 publication Critical patent/WO2020006676A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures

Definitions

  • the present invention generally relates to information input technology of a computing device, and particularly to a blind input technology using a soft keyboard on a touch screen device, a computing device, and a computer storage medium.
  • Text input on a touch screen device is one of the most basic functions of a smart device.
  • an input method using a touch-type soft keyboard in a computing device comprising: determining a coordinate position of a user's finger on a screen in response to a touch input signal from a touch sensor; and according to the touch position , Calculate the input probability of each character; determine the optimal character according to the input probability value of each character, and play the voice of the optimal character.
  • determining the optimal character includes: determining, according to the input probability value of each character, the character with the highest probability value as the candidate optimal character; and trying to update the keyboard layout according to the touch-down position and the determined candidate optimal character, So that the candidate optimal character is at the touch-press position and the updated keyboard layout meets a predetermined standard; if the updated keyboard layout is successful, the candidate optimal character is determined to be the optimal character; otherwise, the default standard keyboard layout is used to determine The optimal character is the key at the touched position in this layout. .
  • the input method further includes: in response to the user continuing to move to access other key characters without lifting the finger, playing the voice of the key character corresponding to the finger position.
  • the input method further includes: after playing the voice of the key character, in response to detecting a confirmation signal originating from the user, completing the input of the confirmed character.
  • the confirmation signal includes: a signal triggered by the user lifting a finger or a signal triggered by a double-click operation after lifting the finger.
  • determining the input probability of each character includes: calculating the input probability of each character based on the character that has been input and the position where the user touched and pressed, using a drop point model and a language model.
  • the method before calculating the input probability of each character according to the touch position, the method further includes: determining whether the distance between the current touch position and the previous confirmed input touch position is less than a certain threshold, and if so, directly using the previous Confirm the keyboard layout at the time of input, and do not update the keyboard layout; if not, restore the keyboard layout to the default standard keyboard layout, and then do probability calculations and determine the optimal characters, and update the keyboard layout as needed based on the results.
  • the keyboard layout is an invisible keyboard response area setting capable of achieving the same effect.
  • the landing point model is a normal distribution.
  • the updating the keyboard layout includes moving and / or compressing the keyboard.
  • the moving and / or compressing the keyboard includes: performing branch or overall movement and compression; compressing the keyboard in a linear or exponential manner with the touch-pressed position as the center; the movement must not exceed a predefined effective range; the compression must not Make any key size smaller than the size threshold.
  • the input method further includes determining whether a gesture input has occurred, the determining whether the gesture input has occurred includes: detecting a contact trajectory within a full screen range instead of being limited to an area occupied by a soft keyboard; using a displacement of the contact trajectory- The time relationship determines whether a gesture input has occurred and determines the type of gesture; based on the determined gesture type, a corresponding input operation is determined.
  • gesture types and corresponding input operations include: single-finger swipe left: delete the last typed character; two-finger swipe left: clear all typed characters; single-finger swipe up: switch the highlighted candidate to the previous candidate Words: Single-finger swipe: Switch the highlighted candidate to the next candidate; Single-finger swipe right: Confirm input of the current candidate
  • the input method further includes: providing audio feedback for touch operations, including: feedback on key presses; feedback on confirming input; and response to gesture operations.
  • a computing device including: a touch screen with a touch sensor capable of detecting a touch input; a controller and a memory, wherein the computer-executable instructions are stored on the memory, and the touch-type soft keyboard is activated
  • the computer-executable instructions are executed by a controller, they are operable to perform the aforementioned input method.
  • a computer-readable storage medium having computer-executable instructions stored thereon, and in a case where a touch-type soft keyboard of a computing device has been activated: When executed, it is operable to perform the aforementioned input method.
  • the feedback of the input method is completely audioized, and the prediction algorithm is adjusted.
  • the soft keyboard zoom and translation can be implemented according to the characteristics of blind input, which can effectively improve Input accuracy and speed of blind people on touch screen devices.
  • the techniques used in this example are based on conditional probability formulas, Bayesian formulas, text-to-speech synthesis, and so on.
  • FIG. 1 illustrates an overall flowchart of an input method 100 for inputting by using a touch-type soft keyboard in a computing device according to an embodiment of the present invention.
  • FIG. 4 shows an overall flow of a text input method 200 according to another embodiment of the present invention.
  • Figures 5a and 5b show examples of soft keyboard movement and zooming processes in actual use.
  • FIG. 6 shows a schematic example of illegal movement of a soft keyboard in actual use.
  • Fig. 7a shows the keyboard state before the user presses a key in an input example
  • Fig. 7b shows the layout of the keyboard after the user first presses a key when entering a letter
  • Fig. 7c shows an error that occurs when "o" is entered situation.
  • Figure 7d uses the left swipe gesture to delete, at this time the voice feedback: "o has been deleted”.
  • Touch-press refers to the action from the finger never touching the touch screen to touching the touch screen.
  • Movement Refers to the movement of the finger while changing the position while keeping touching the touch screen.
  • Lift refers to the movement of a finger from touching the touch screen to not touching the touch screen.
  • FIG. 1 illustrates an overall flowchart of an input method 100 for inputting by using a touch-type soft keyboard in a computing device according to an embodiment of the present invention.
  • the input method of this embodiment is particularly suitable for inputting text.
  • the type of input text There is no limitation on the type of input text.
  • the input method may be English character input or Chinese character input.
  • step S110 in response to a touch input signal from a touch sensor, a coordinate position of a user's finger on a screen is determined.
  • step S120 the input probability of each character is calculated based on the touched position.
  • the input probability of each character currently input by the user is calculated based on the character string that has been input and the position where the user touches and presses, and uses a drop point model and a language model.
  • the language model is a Chinese / English dictionary consisting of vocabulary and frequency pairs.
  • the table shown in Figure 2 gives a schematic example of a language model.
  • the landing point model is a combined list of expected input keys and actual input key distributions on a touch-screen device when the user has no visual feedback. This model can be obtained through testing and statistics of user input behavior.
  • the table shown in Figure 3 gives a schematic example of the fall point model.
  • the N + 1th input result is combined to predict the N + 1th user input intention.
  • P (KI) is the probability that the user intends to press the button I.
  • Is the drop point model factor that is, the probability that the user enters key A when he wants to enter key I.
  • Is the language model factor that is, in the dictionary, all words with a length greater than or equal to N + 1 and the first N characters are the same characters that the user has entered, and the N + 1th character is the probability of pressing the key I.
  • the keys After calculating the probability values of all keys, the keys are sorted. The key with the highest probability is the key that the user expects to type, which is the optimal character described below.
  • step S120 the optimal character is determined according to the input probability value of each character, and the voice of the optimal character is played.
  • determining the optimal character in addition to considering the input probability value of each character, the touch-down position is also considered.
  • determining the optimal character includes: determining a character with the highest probability value as a candidate optimal character according to the input probability value of each character; and attempting to update the keyboard layout according to the touch-down position and the determined candidate optimal character to Make the candidate optimal character at the touch-press position and the updated keyboard layout meet predetermined criteria; if the updated keyboard layout is successful, determine the candidate optimal character as the optimal character; otherwise, use the default standard keyboard layout to determine the most optimal character.
  • the top character is the key at the touched position in this layout.
  • the input of the confirmed character is completed in response to detecting a confirmation signal originating from the user.
  • the confirmation signal is a signal triggered by a user gesture, such as a signal triggered by a user lifting a finger or a signal triggered by a double-click operation after lifting a finger.
  • a user gesture such as a signal triggered by a user lifting a finger or a signal triggered by a double-click operation after lifting a finger.
  • the confirmation signal is not limited to a signal triggered by a gesture.
  • the gesture is not limited to a gesture such as raising a finger or clicking after the finger is raised.
  • the text input method improves the accuracy of Chinese and English input without visual feedback, and uses voice to interact with the user, which is very suitable for blind people to use.
  • the text input method can be applied, for example, to a computing device such as a smart phone or a tablet computer.
  • FIG. 4 shows an overall flow of a text input method 200 according to another embodiment of the present invention.
  • step S210 the coordinate position of the user's finger on the screen is determined in response to a touch input signal from the touch sensor.
  • step S220 it is determined whether the distance between the current touch position and the previous touch lift position is less than a certain threshold.
  • the close position indicates that the user will next enter the same letter or the character next to the last key input.
  • the keyboard layout should be updated. For example, if the two input positions are basically the same, the current touch is considered to have occurred on the keyboard layout when it was last lifted, and the keys to be input are the same as the last time. For example, in "hello”, the last time you entered “L”, and the current touch position is still "L”, the two inputs are considered to be related, and there is no need to update the keyboard layout. After the update, it may lead to the inability to enter L correctly; another This situation is basically a key difference between the two input positions.
  • the blind user wants to input the adjacent letter of the last entered letter.
  • the blind user wants to enter "bed” and the last time “e” is input.
  • the position is the position below the last touched position, it is considered that the blind person wants to enter "d” this time.
  • step S220 determines whether the movement range does not exceed a certain threshold. If it exceeds a certain threshold, the process proceeds to step S230.
  • step S230 an input character of each character is calculated based on the touched position.
  • step S230 reference may be made to the description made in conjunction with step S120 in FIG.
  • step S250 the candidate optimal character is determined according to the input probability value of each character.
  • step S260 an attempt is made to update the keyboard layout as appropriate so that the candidate optimal character is at the touch position, and it is determined whether the keyboard layout meets a predetermined standard.
  • Updated keyboard layouts here include, for example, mobile and / or compressed keyboards.
  • the interaction between the soft keyboard and the user is adjusted by moving and zooming the soft keyboard, keeping the user's memory of the relative position of the keyboard during the key adjustment process, and improving the efficiency of the user's input adjustment.
  • Moving and / or compressing the keyboard may include: performing branch or overall movement and compression; compressing the keyboard in a linear or exponential manner with the touch position as the center, and the linear or exponential manner is user selectable.
  • the system After the system makes a request to update or adjust the soft keyboard, it will determine whether the request is legal, in other words, whether it meets the predetermined criteria.
  • the criterion for this judgment can be: the movement must not exceed the predefined effective range; the compression must not make any key size smaller than the size threshold that the user can touch, such as whether any key size is smaller than a certain percentage of the original size. If such a key exists later, the request is illegal and no adjustment will be made. If the request is valid, the keyboard will actually be adjusted.
  • the adjustment method is that the entire keyboard is scaled and translated linearly, so that the center of the target key is adjusted to the target position.
  • the movement must not exceed a predefined effective range, which can be determined by using, for example, an input range of a current commonly used input method as a reference.
  • step S260 if it is determined in step S260 that the keyboard layout meets the predetermined standard, proceed to step S280, that is, use the updated keyboard layout, confirm that the candidate optimal character is the optimal character, and play the optimal character, otherwise proceed to step In S270, a default keyboard layout is adopted, and the key corresponding to the touch position under the default keyboard layout is determined to be the optimal character, and the optimal character voice is played.
  • Figures 5a and 5b show examples of soft keyboard movement and zooming processes in actual use. This is the scene of entering the fourth pinyin letter when typing the word "PENGYOU”.
  • the actual touch key is "F”
  • the algorithm predicts that the user intends to type "G”
  • the algorithm requests the keyboard to move the center of the G key to the current touch point.
  • the keyboard confirms that this movement is legal, and finally realizes the movement of the soft keyboard.
  • the resulting voice feedback is "G”.
  • the dark gray circle on the letter G in FIG. 5b indicates the finger touch position.
  • the dark gray circles in Figs. 7b, 7c also indicate the finger touch position.
  • FIG. 6 shows a schematic example of illegal movement of a soft keyboard in actual use. Specifically, when the last letter s of the company is input, since the key position is too left, compressing the key will cause the a key width to be smaller than the minimum value (for example, 50% of the original key width), so the movement is illegal. Although the probability maximum key bit is s, because the movement is illegal, the final input key is the key a corresponding to the pressed position.
  • the minimum value for example, 50% of the original key width
  • step S290 the event is continuously monitored. If a mobile event is received, voice feedback is given during the movement. For example, after the system gives a voice prompt for the optimal character, the user thinks that this is not the character he wants, at this time the finger will continue to move to find the remaining characters, and in the process, the voice feedback will continue to be given.
  • step S291 in response to receiving the user lifting event, a gesture judgment is performed, and a voice prompt is given.
  • operations such as deleting, confirming word selection, switching candidate words, and clearing are implemented through gesture operations.
  • Determining whether a gesture input has occurred includes detecting contact trajectories in a full screen range instead of being limited to the area occupied by the soft keyboard; using the displacement-time relationship of the contact trajectories to determine whether gesture input has occurred and determining the type of gesture, so that the Type of gesture while determining the corresponding input operation.
  • the gesture type may include: single-finger swipe left, two-finger swipe left, single-finger swipe up, single-finger swipe down, single-finger swipe right, and so on.
  • gestures and operations can be as follows:
  • Single-finger left stroke In the Chinese multi-step word selection, the previous word selection is canceled. In other cases, the last pinyin or letter of the currently selected word is not confirmed. If not, the last Chinese character or letter that completed the word selection is deleted.
  • One-finger swipe up Set the currently highlighted candidate as the previous candidate in the list.
  • One-finger swipe down Set the currently highlighted candidate as the next candidate in the list.
  • corresponding voice feedback is given, for example, when it is determined that the two-finger left-swipe gesture is performed, the voice prompt "clear all current inputs".
  • step S292 the operation corresponding to the determined gesture action is performed, and the information related to the lifting action is also recorded under the clearing determined as the input operation.
  • the character is input, and the related information of the current lifting is recorded for comparison of the next input.
  • step S210 When a subsequent touch event is received from the user, the process starts from step S210 again.
  • the user input is predicted by combining the language model and the drop point model, and the input without visual feedback is adapted by moving and zooming the soft keyboard, using voice to interact with the user, and using gesture operations to complete the text Word selection, confirmation, deletion, and clearing operations during input can improve the accuracy of Chinese and English input without visual feedback and the speed of Chinese and English input without visual feedback.
  • the keyboard layout is not limited to a visually visible keyboard layout, and may also be an invisible keyboard response area setting capable of achieving the same effect, which has the same effect for blind people.
  • Figure 7a shows the keyboard state before the user presses a key.
  • the language model and the drop point model will be used to calculate the key with the highest probability that the user expects to press, and the center of the key will be moved to the user touch point by zooming and panning the keyboard.
  • the voice feedback content is the current key, such as 'H'. (The prediction order is displayed on the keyboard through the corner mark, for debugging and display use, non-feedback content)
  • Figure 7b shows the layout of the keyboard after the user first presses a letter when entering a letter.
  • the key can be retrieved by moving the finger while keeping the finger pressed.
  • the keys passed by the user's finger will be feedback in real time through voice, and the ongoing voice feedback will be interrupted by the new voice.
  • the keyboard layout is maintained.
  • Fig. 7c shows a situation where an error occurs when "o" is input.
  • "i” is input by mistake and deleted with a left swipe gesture.
  • the reference numeral 1 drawn by the horizontal line indicates the content currently input according to the embodiment of the present invention, that is, the content confirmed by the user.
  • the input is confirmed by the user's lifting action
  • the reference numeral 2 indicates according to the embodiment of the present invention.
  • reference numeral 3 indicates the input content without running the embodiment of the present invention, that is, the content input by touching the touch point according to the default layout.
  • the number 4 indicates that the currently entered character is i
  • the number 5 indicates candidate words, including heli, hellish, helling, and hellip.
  • the numbers displayed on the characters of the soft keyboard shown below indicate the input probability calculated by the probability ranking. Five characters, that is, the character with the first probability of input is I, the character with the second probability of input is O, the character with the third probability is U, the character with the fourth probability is J, and the character with the fifth probability Is Y.
  • the user can use the up and down gestures to select the previous and next candidate words, and at the same time, the voice feedback changes the candidate words.
  • This function is less useful in English input, mainly used when there are homophones in Chinese input.
  • the user can use a two-finger left swipe gesture to clear all input.
  • the voice feedback "empties".
  • FIG. 7d schematically illustrates the deletion of “o” using the left swipe gesture.
  • the voice feedback “o has been deleted”, the circle in FIG. 7d And the left arrow gestures to the left.
  • the input method of this embodiment is particularly suitable for use by blind people, but of course it can also be used by people with normal vision.
  • gestures and corresponding operations are merely examples, and other gestures may be designed and corresponding to different operations as required.
  • the input method of the embodiment of the present invention can be applied to various computing devices, such as a smart phone, a tablet computer, and the like.
  • Embodiments of the present invention may also be implemented as a computing device, including: a touch screen with a touch sensor capable of detecting a touch input; a controller and a memory, the computer-executable instructions stored on the memory, and when the touch soft keyboard is activated:
  • the computer-executable instructions, when executed by a controller, are operable to perform the input method described above.
  • Embodiments of the present invention may also be implemented as a computer-readable storage medium having computer-executable instructions stored thereon.
  • the touch-type soft keyboard of a computing device has been activated: when the computer-executable instructions are executed by a computer, Operable to perform the input method described above.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Input From Keyboards Or The Like (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

提供了利用触摸式软键盘进行输入的输入方法，计算装置和计算机可读介质。输入方法包括响应于来自触摸传感器的触摸输入信号，确定用户手指在屏幕上的坐标位置；根据触摸位置，计算各字符的输入概率；根据各字符的输入概率值，确定最优字符，并播放该最优字符的语音。进一步地，确定最优字符包括：根据各字符的输入概率值，确定概率值最大的字符作为候选最优字符；根据触摸按下位置和确定的候选最优字符，尝试更新键盘布局，以使得所述候选最优字符在所述触摸按下位置且更新后的键盘布局满足预定标准；如果更新键盘布局成功，则确定候选最优字符为最优字符；否则采用默认标准键盘布局，确定最优字符为该布局下的触摸按下位置的按键。

Description

利用触摸软键盘的盲式输入法、计算装置和存储介质 技术领域
本发明总体地涉及计算设备的信息输入技术，特别是涉及触屏设备上利用软键盘的盲式输入技术、计算装置和计算机存储介质。
背景技术
在触屏设备上进行文字输入是智能设备最基本的功能之一。
存在使用语言模型实现对输入的预测的现有技术，如CN102915122A。该现有技术默认键入操作的正确性，通过视觉进行反馈，不适用盲人输入应用。
也存在使用直接屏幕阅读的方式使得盲人可以在触屏设备上进行输入的技术，例如专利CN103064529A，该现有技术选词操作同键入操作同为手指在屏幕上检索，增加了误操作的可能性，盲人输入的实用性较低；以及无法对盲人意图进行预测，无法进行纠错，降低输入准确性，增加了检索按键的耗时。
发明内容
鉴于上述情况，提出了本发明。
根据本发明的一个方面，提供了一种计算装置中利用触摸式软键盘进行输入的输入方法，包括：响应于来自触摸传感器的触摸输入信号，确定用户手指在屏幕上的坐标位置；根据触摸位置，计算各字符的输入概率；根据各字符的输入概率值，确定最优字符，并播放该最优字符的语音。
可选地，所述确定最优字符包括：根据各字符的输入概率值，确定概率值最大的字符作为候选最优字符；根据触摸按下位置和确定的候选最优字符，尝试更新键盘布局，以使得所述候选最优字符在所述触摸按下位置且更 新后的键盘布局满足预定标准；如果更新键盘布局成功，则确定候选最优字符为最优字符；否则采用默认标准键盘布局，确定最优字符为该布局下的触摸按下位置的按键。。
可选地，输入方法还包括：响应于用户不抬起手指继续移动访问其他按键字符,播放手指位置所对应按键字符的语音。
可选地，输入方法还包括：在播放按键字符的语音之后，响应于检测到源于用户的确认信号，而完成被确认的字符的输入。
可选地，确认信号包括：用户抬起手指触发的信号或者抬起手指后双击点击操作触发的信号。
可选地，确定各字符的输入概率包括：基于已经输入的字符和用户触摸按下位置，结合使用落点模型与语言模型，计算各字符的输入概率。
可选地，在所述根据触摸位置计算各字符的输入概率之前，还包括：确定本次触摸位置与前次确认输入的触摸位置之间的距离是否小于一定阈值，如果是，则直接使用前次确认输入时的键盘布局，不对键盘布局做更新；如果否，则将键盘布局恢复为默认标准键盘布局，然后再做概率计算和确定最优字符，并根据该结果视需要更新键盘布局。
可选地，所述键盘布局为不可见的能够达到相同效果的键盘响应区域设置。
可选地，所述落点模型为正态分布。
可选地，所述更新键盘布局包括移动和/或压缩键盘。
可选地，所述移动和/或压缩键盘包括：进行分行或整体移动、压缩；以触摸按下位置为中心按照线性或指数方式对键盘进行压缩；移动不得超过预定义的有效范围；压缩不得使任何按键尺寸小于尺寸阈值。
可选地，输入方法还包括确定是否发生了手势输入，所述确定是否发生了手势输入包括：在全屏范围内而非局限于软键盘所占据区域检测触点轨迹；利用触点轨迹的位移-时间关系判断是否发生手势输入并判断手势类型；基于判定的手势类型，而确定对应的输入操作。
可选地，手势类型以及对应的输入操作包括：单指左划：删除最后键入的字符；双指左划：清空所有键入的字符；单指上划：将高亮的候选词切换至上一个候选词；单指下划：将高亮的候选词切换至下一个候选词；单指右 划：确认输入当前候选词
可选地，输入方法还包括：对于触摸操作提供音频反馈，包括：对按键按下的反馈；对确认输入的反馈；对手势操作的反聩。
根据本发明的另一方面，还提供了一种计算装置，包括：触摸屏，具有触摸传感器，能够检测触摸输入；控制器和存储器，存储器上存储有计算机可执行指令，在触摸式软键盘已启动的情况下：当所述计算机可执行指令当被控制器执行时，可操作来执行前述输入方法。
根据本发明另一方面，提供了一种计算机可读存储介质，其上存储有计算机可执行指令，在计算装置的触摸式软键盘已启动的情况下：当所述计算机可执行指令当被计算机执行时，可操作来执行前述输入方法。
根据本发明实施例的输入法、计算装置和计算机可读介质，将输入法的反馈完全音频化，并对预测算法做出调整，针对盲人输入的特点实现了软键盘的缩放平移，可以有效提升盲人在触屏设备上的输入准确率和速度。本例中使用的技术基于条件概率公式、贝叶斯公式、文字转语音合成等。
附图说明
从下面结合附图对本发明实施例的详细描述中，本发明的这些和/或其它方面和优点将变得更加清楚并更容易理解，其中：
图1示出了根据本发明实施例的计算装置中利用触摸式软键盘进行输入的输入方法100的总体流程图。
图2所示的表格给出了语言模型的一个示意性样例。
图3所示的表格给出了落点模型的一个示意性样例。
图4示出了根据本发明另一实施例的文本输入方法200的总体流程。
图5a和图5b展示了实际使用中，软键盘移动、缩放的过程示例。
图6示出了实际使用中，软键盘移动非法的一个示意性示例。
图7a示出了一个输入示例中用户按下按键前键盘状态，图7b示出了用户在输入一个字母时，首次按下后键盘的布局，图7c示出了输入“o”时发生错误的情形。图7d使用左划手势删除，此时语音反馈：“o已删除”。
具体实施方式
为了使本领域技术人员更好地理解本发明，下面结合附图和具体实施方式对本发明作进一步详细说明。
在介绍之前，解释一下有关术语在本文中的含义。
触摸按下：指手指从未接触触摸屏到接触触摸屏的动作。
移动：指手指在保持触摸触摸屏的状态下变化位置的动作。
抬起：指手指从触摸触摸屏到不触摸触摸屏的动作。
下面将参考附图描述本发明的具体实施例。
图1示出了根据本发明实施例的计算装置中利用触摸式软键盘进行输入的输入方法100的总体流程图。
本实施例的输入方法特别适用于输入文字，这里对于输入文字的类型没有限制，例如可以是英文字符输入或者中文汉字输入。
如图1所示，在步骤S110中，响应于来自触摸传感器的触摸输入信号，确定用户手指在屏幕上的坐标位置。
在步骤S120中，根据触摸按下位置，计算各字符的输入概率。
在一个示例中，基于已经输入的字符串和用户触摸按下位置，结合使用落点模型与语言模型，计算用户当前输入各字符的输入概率。
在一个示例中，语言模型是由词汇、出现频率成对组成的中/英文词典。图2所示的表格给出了语言模型的一个示意性样例。
在一个示例中，落点模型是用户在无视觉反馈状态下，在触屏设备上，期望输入按键与实际输入按键分布的组合列表。这一模型可以通过对用户输入行为的测试、统计得到。图3所示的表格给出了落点模型的一个示意性样例。
在一个示例中，通过结合语言模型与落点模型计算，对用户在单个单词/词汇拼音输入过程中的输入行为进行纠错与预测，提高输入结果的准确性并减少输入操作、修正操作次数，提高输入速度。一个具体的计算方式示例如下：
在用户键入一个单词/词汇拼音的第N+1个字符时，将结合前N次的输入结果对第N+1次用户输入意图进行预测。首先计算键盘上每一个按键可能为用户输入意图的概率，在用户第N+1次输入键入键A后，单个按键概率的计算方式如下：
Figure PCTCN2018094203-appb-000001
其中，P(KI)为用户意图为按键I的概率。
Figure PCTCN2018094203-appb-000002
为落点模型因子，即用户在希望输入键I时键入键A的概率。
Figure PCTCN2018094203-appb-000003
为语言模型因子，即字典中，所有长度大于等于N+1且前N个字符为用户已输入字符相同的单词中，第N+1个字符为按键I的概率。
计算得到所有按键的这一概率值后，对其进行排序，概率最高的按键即为预测用户期望键入的按键，即下文所说的最优字符。
在步骤S120中，根据各字符的输入概率值，确定最优字符，并播放该最优字符的语音。
在一个示例中，确定最优字符过程中，除了考虑各字符的输入概率值，还考虑触摸按下位置。具体地，例如，确定最优字符包括：根据各字符的输入概率值，确定概率值最大的字符作为候选最优字符；根据触摸按下位置和确定的候选最优字符，尝试更新键盘布局，以使得所述候选最优字符在所述触摸按下位置且更新后的键盘布局满足预定标准；如果更新键盘布局成功，则确定候选最优字符为最优字符；否则采用默认标准键盘布局，确定最优字符为该布局下的触摸按下位置的按键。。
在一个示例中，在播放按键字符的语音之后，响应于检测到源于用户的确认信号，而完成被确认的字符的输入。
在一个示例中，确认信号是由用户手势触发的信号，例如用户抬起手指触发的信号或者抬起手指后双击点击操作触发的信号。不过此仅为示例，确认信号不限于是手势触发的信号，在手势触发的情况下，手势也不限于抬起手指或者抬起手指后点击操作这样的手势。
根据本发明上述实施例的文本输入方法，提高无视觉反馈条件下中英文输入的准确性，以及使用语音与用户交互，非常适于盲人使用。该文本输入方法可以例如应用于智能手机、平板电脑等计算装置上。
图4示出了根据本发明另一实施例的文本输入方法200的总体流程。
如图4所示，在步骤S210中，响应于来自触摸传感器的触摸输入信号，确定用户手指在屏幕上的坐标位置。
在步骤S220中，确定本次触摸位置与前次触摸抬起位置之间距离是否小于某阈值。
接收到用户接触事件后，通过判断本次触摸位置与前次触摸位置是否位置接近，位置接近表明用户接下来要输入的是与上次输入按键相同的字母或者位于其旁边的字符，此时不应对键盘布局进行更新，例如，在两次输入位置基本相同的情况下，则认为本次触摸是在上次抬起时的键盘布局上发生的，且要输入的键是与上次相同的，例如“hello”中，上次输入了“L”，本次触摸位置还是“L”，则认为两次输入相关，无需更新键盘布局，如果更新后，反而可能导致无法正确地输入L；另一种情况是在两次输入位置基本相差一个按键，此时认为盲人用户是想输入上次输入字母相邻的字母，例如，盲人用户想输入“bed”上次输入了“e”，本次触摸位置为上次触摸位置下面的位置，则认为盲人本次欲输入“d”，这是也不应更新键盘布局，同样更新后，反而可能导致无法正确输入。
因此如果步骤S220确定结果为是，即移动范围没有超过一定阈值，则前进到步骤S240，即将键盘布局恢复为默认标准键盘布局，否则步骤S220确定结果为否，即用户手指相对上次点击位置移动超过一定阈值，则前进到步骤S230。
在步骤S230中，根据触摸位置，计算各字符的输入字符。关于步骤S230的具体实现，可以参考结合图1中的步骤S120进行的描述。
在步骤S250中，根据各字符的输入概率值，确定候选最优字符。
在步骤S260中，视情况尝试更新键盘布局，使得候选最优字符在触摸位置，确定此键盘布局是否符合预定标准。
在所确定的候选最优字符并不是当前键盘布局上手指所触摸位置对应的字符(键)时，尝试更新键盘布局，使得手指位置对应的字符是候选最优字符。
这里的更新键盘布局包括例如移动和/或压缩键盘。本例通过对软键盘的移动、缩放来调整软键盘与用户的交互，保持用户在调整按键过程中对键盘相对位置的记忆，提高用户调整输入的效率。
移动和/或压缩键盘可以包括：进行分行或整体移动、压缩；以触摸位置为中心按照线性或指数方式对键盘进行压缩，该线性或指数方式是用户可 选的。
在***做出一次更新或调整软键盘的请求后，将判断本次请求是否合法，换句话说，是否符合预定标准。这一判断的判据可以为：移动不得超过预定义的有效范围；压缩不得使任何按键尺寸小于用户能够进行触摸选择的尺寸阈值，例如是否有按键的尺寸小于原尺寸的一定百分比，若在调整后存在这样的按键，则本次请求不合法，不会进行调整。如果请求合法，键盘将实际进行调整，调整方式为键盘整体按照线性缩放、平移，从而使目标按键的中心调整至目标位置。这里，关于移动不得超过预定义的有效范围，可以例如以目前常用输入法的输入范围作为参考来加以确定。
如图4所示，在步骤S260中确定键盘布局符合预定标准的情况，前进到步骤S280，即采用更新的键盘布局，确认候选最优字符为最优字符，播放最优字符，否则前进到步骤S270，采用默认的键盘布局，确认在默认的键盘布局下触摸位置对应的按键为最优字符，播放最优字符语音。
图5a和图5b展示了实际使用中，软键盘移动、缩放的过程示例。具体为键入词汇“朋友”(PENGYOU)时输入第四个拼音字母的场景。实际触摸按键为“F”，算法预测用户意图为键入“G”，算法向键盘申请将G键的中心移动至当前触摸点，键盘确认这一移动合法，最终实现软键盘的移动，同时用户实际得到的语音反馈为“G”。图5b中的字母G上的暗灰色圆形示意手指触摸位置。类似地，图7b、7c中的暗灰色圆形也指示手指触摸位置。
图6示出了实际使用中，软键盘移动非法的一个示意性示例。具体地，输入companies最后一个字母s时，由于按键位置太靠左，压缩按键将导致a键宽度小于最小值(例如，原键宽度的50％)，因此移动不合法。虽然概率最大值按键位为s，但由于移动不合法，最终输入按键为按下位置所对应的按键a。
回到图4，在步骤S290中，持续监测事件，如果接收到移动事件，移动过程中给出语音反馈。例如，在***给出最优字符的语音提示后，用户认为这不是自己想要的字符，此时手指会继续移动以寻找其余的字符，此过程中，会持续给出语音反馈。
在步骤S291中，响应于接收到用户抬起事件，进行手势判断，给出语 音提示。
在一个示例中，通过手势操作实现删除、确认选词、切换候选词、清空等操作。
确定是否发生了手势输入可以包括：在全屏范围内而非局限于软键盘所占据区域检测触点轨迹；利用触点轨迹的位移-时间关系判断是否发生手势输入并判断手势类型,从而能基于判定的手势类型，而确定对应的输入操作。
在一个示例中，通过以下步骤判断用户是否发生了手势操作：用户完成抬起动作时，如果整个动作的持续时间小于例如200ms，且在垂直或水平方向上的位移大于例如70像素，且大于另一方向上的位移，则判定用户进行了手势操作。需要说明的是，这里的手势操作不包括手指在触摸状态下移动的操作，即不包括手指在屏幕上搜索的动作。单指或者双指操作可以由接触点来进行判定。
在一个示例中，手势类型可以包括：单指左划、双指左划、单指上划、单指下划、单指右划等。
示例性地，手势与操作的对应关系可以如下：
单指左划：中文多步选词时撤销上一步选词，其他情况下删除当前没有确认选词的最后一个拼音或者字母，如果没有则删除最后一个完成选词的汉字或者字母。
单指右划：对当前候选词确认输入，中文多步选词完成后自动上屏。
单指上划：设置当前高亮的候选词为列表里的上一个候选词。
单指下划：设置当前高亮的候选词为列表里的下一个候选词。
双指左划：清空当前所有的输入。
在一个示例中，在判断出手势类型后，给出相应地语音反馈，例如在判断为双指左划手势时，语音提示“清空当前所有的输入”。
在步骤S292中，进行所判断的手势动作对应的操作，在确定为输入操作的清空下，还记录下抬起动作的相关信息。
例如，在确定用户为进行了对应于输入已音频提示的字符的抬起动作时，输入该字符，并且记录本次抬起的相关信息，用于下一次输入的比较。
后续当收到用户的触摸事件时，再次从步骤S210开始执行。
根据本发明上述实施例的文本输入方法，结合语言模型与落点模型对用户输入进行预测，通过移动、缩放软键盘的方式适应无视觉反馈的输入，使用语音与用户交互，使用手势操作完成文字输入中的选词、确认、删除、清空等操作，能够提高无视觉反馈条件下中英文输入的准确性，以及提高无视觉反馈条件下中英文输入的速度。
需要说明的是，键盘布局并不限于是视觉可见的键盘布局，也可以是不可见的能够达到相同效果的键盘响应区域设置，这对于盲人来说效果是相同的。
下面以输入hello为例，给出利用本发明实施例的文本输入方法输入一个单词的完整过程。
图7a示出了用户按下按键前键盘状态。
1、用户在键盘区域进行按下操作后，将利用语言模型和落点模型计算用户期望按下的概率最大的键，并通过缩放、平移键盘，将该键的中心移至用户触点。同时，语音反馈内容为当前按键，如‘H’。(键盘上通过角标显示预测顺序，为调试、展示使用，非反馈内容)
图7b示出了用户在输入一个字母时，首次按下后键盘的布局。
2、若这一按键并非用户期望输入的按键，则可在保持手指按下的状态下，通过移动手指检索按键。检索过程中将实时通过语音反馈用户手指经过的按键，正在进行的语音反馈会被新的语音所中断。检索过程中，键盘的布局保持。
3、当用户通过语音反馈确认自己要输入的按键后，通过抬起手指的方式确认输入。同时，语音反馈用户输入的按键，如‘H’。用户键入后，键盘的布局保持。
4、重复1-3的过程，进行‘Hello’剩余字母的输入。
5、用户在单词输入过程中可以使用手势删除已输入的字母。通过左划操作，可以删除最后输入的字母。同时语音反馈：“X已删除”(X为被删除的字母)。
图7c示出了输入“o”时发生错误的情形，示例中，错误输入了“i”，使用左划手势删除，此时语音反馈：“i已删除”。
这里结合图7c说明一下该示例中屏幕显示的内容，不过需要注意，此仅 为示例，而且此主要为验证实验结果进行的显示，而在实际应用于盲人输入服务的情况下，屏幕上可以什么都不显示，可以显示部分内容，也可以显示不同的内容。图7c中，横线引出的标号1指示根据本发明实施例的当前输入的内容也即用户确认输入的内容，本示例中为通过用户抬起动作确认的输入，标号2指示根据本发明实施例的算法预测的内容，标号3指示在不运行本发明实施例的情况下的输入的内容，也即按照默认布局，触摸落点所输入的内容。标号4指示当前输入的字符为i，标号5指示候选单词，包括helli,hellish,helling和hellip，另外，这里下面显示的软键盘的字符上显示的数字表示通过概率计算得出的输入概率排名前五的字符，即该示例中输入概率第一的字符为I，输入概率第二的字符为O，输入概率第三的字符为U，输入概率第四的字符为J，输入概率第五的字符为Y。
6、当用户完成一个单词的输入时，使用右划手势确认输入单词。同时，语音反馈“Hello已输入”。
另外，在没有输入字母的情况下，右划操作会键入一个空格，同时，语音反馈“空格”。
另外，用户使用上划、下划手势可以选择上一个、下一个候选词，同时，语音反馈改候选词。这一功能在英文输入中用处较小，主要在中文输入有同音词时使用。
另外，用户使用双指左划手势可以清空所有输入的内容。同时，语音反馈“清空”。
另外，用户使用手指左划手势，可以删除刚输入的字符，例如图7d示意性地示出了使用左划手势删除“o”，此时语音反馈：“o已删除”，图7d中的圆圈和向左的箭头示意了手指左划的手势。
本实施例的输入方法特别适合于盲人使用，不过当然也可以由视力正常的人。
需要说明的是，前面的各个手势和对应的操作仅为示例，可以视需要设计其它的手势以及与不同的操作对应。
本发明实施例的输入方法可以应用于各种计算装置，例如智能手机、平板电脑等等。
本发明实施例还可以实施为计算装置，包括：触摸屏，具有触摸传感 器，能够检测触摸输入；控制器和存储器，存储器上存储有计算机可执行指令，在触摸式软键盘已启动的情况下：当所述计算机可执行指令当被控制器执行时，可操作来执行上述输入方法。
本发明实施例还可以实现为计算机可读存储介质，其上存储有计算机可执行指令，在计算装置的触摸式软键盘已启动的情况下：当所述计算机可执行指令当被计算机执行时，可操作来执行上述输入方法。
以上已经描述了本发明的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。因此，本发明的保护范围应该以权利要求的保护范围为准。

Claims (16)

  1. 一种计算装置中利用触摸式软键盘进行输入的输入方法，包括：
    响应于来自触摸传感器的触摸输入信号，确定用户手指在屏幕上的坐标位置；
    根据触摸按下位置，计算各字符的输入概率；
    根据各字符的输入概率值，确定最优字符；以及
    播放该最优字符的语音。
  2. 根据权利要求1所述的输入方法，所述确定最优字符包括：
    根据各字符的输入概率值，确定概率值最大的字符作为候选最优字符；
    根据触摸按下位置和确定的候选最优字符，尝试更新键盘布局，以使得所述候选最优字符在所述触摸按下位置且更新后的键盘布局满足预定标准；
    如果更新键盘布局成功，则确定候选最优字符为最优字符；
    否则采用默认标准键盘布局，确定最优字符为该布局下的触摸按下位置的按键。
  3. 根据权利要求2所述的输入方法，还包括：
    响应于用户不抬起手指继续移动访问其他按键字符,播放手指位置所对应按键字符的语音。
  4. 根据权利要求1所述的输入方法，还包括：
    在播放按键字符的语音之后，响应于检测到源于用户的确认信号，而完成被确认的字符的输入。
  5. 根据权利要求4所述的输入方法，所述确认信号包括：
    用户抬起手指触发的信号或者抬起手指后双击点击操作触发的信号。
  6. 根据权利要求1所述的输入方法，所述确定各字符的输入概率包括：
    基于已经输入的字符串和用户触摸按下位置，结合使用落点模型与语言模型，计算用户当前输入各字符的概率。
  7. 根据权利要求1所述的输入方法，在所述根据触摸位置计算各字符的输入概率之前，还包括：
    确定本次触摸位置与前次确认输入的触摸位置之间的距离是否小于一定阈值，
    如果是，则直接使用前次确认输入时的键盘布局，不对键盘布局做更新；
    如果否，则将键盘布局恢复为默认标准键盘布局，然后再做概率计算和确定最优字符，并根据该结果视需要更新键盘布局。
  8. 根据权利要求2-7所述的输入方法，所述键盘布局为不可见的能够达到相同效果的键盘响应区域设置。
  9. 根据权利要求7诉述的输入方法，所述落点模型为正态分布。
  10. 根据权利要求2所述的输入方法，所述更新键盘布局包括移动和/或压缩键盘。
  11. 根据权利要求11所述的输入方法，所述移动和/或压缩键盘包括：
    进行分行或整体移动、压缩；
    以触摸按下位置为中心按照线性或指数方式对键盘进行压缩；
    移动不得超过预定义的有效范围；
    压缩不得使任何按键尺寸小于尺寸阈值。
  12. 根据权利要求1所述的输入方法，还包括确定是否发生了手势输入，
    所述确定是否发生了手势输入包括：
    在全屏范围内而非局限于软键盘所占据区域检测触点轨迹；
    利用触点轨迹的位移-时间关系判断是否发生手势输入并判断手势类型；
    基于判定的手势类型，而确定对应的输入操作。
  13. 根据权利要求13的输入方法，所述手势类型以及对应的输入操作包括：
    单指左划：删除最后键入的字符；
    双指左划：清空所有键入的字符；
    单指上划：将高亮的候选词切换至上一个候选词；
    单指下划：将高亮的候选词切换至下一个候选词；
    单指右划：确认输入当前高亮的候选词
  14. 根据权利要求1到14任一项所述的输入方法，还包括：
    对于触摸操作提供音频反馈，包括：
    对按键按下的反馈；
    对确认输入的反馈；
    对手势操作的反聩。
  15. 一种计算装置，包括：
    触摸屏，具有触摸传感器，能够检测触摸输入；
    控制器和存储器，存储器上存储有计算机可执行指令，在触摸式软键盘已启动的情况下：当所述计算机可执行指令当被控制器执行时，可操作来执行权利要求1到15所述的输入方法。
  16. 一种计算机可读存储介质，其上存储有计算机可执行指令，在计算装置的触摸式软键盘已启动的情况下：当所述计算机可执行指令当被计算机执行时，可操作来执行权利要求1到15所述的输入方法。
PCT/CN2018/094203 2018-07-03 2018-07-03 利用触摸软键盘的盲式输入法、计算装置和存储介质 WO2020006676A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/094203 WO2020006676A1 (zh) 2018-07-03 2018-07-03 利用触摸软键盘的盲式输入法、计算装置和存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/094203 WO2020006676A1 (zh) 2018-07-03 2018-07-03 利用触摸软键盘的盲式输入法、计算装置和存储介质

Publications (1)

Publication Number Publication Date
WO2020006676A1 true WO2020006676A1 (zh) 2020-01-09

Family

ID=69059835

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/094203 WO2020006676A1 (zh) 2018-07-03 2018-07-03 利用触摸软键盘的盲式输入法、计算装置和存储介质

Country Status (1)

Country Link
WO (1) WO2020006676A1 (zh)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101719022A (zh) * 2010-01-05 2010-06-02 汉王科技股份有限公司 用于全功能键盘的字符输入方法及其处理装置
CN101937313A (zh) * 2010-09-13 2011-01-05 中兴通讯股份有限公司 一种触摸键盘动态生成和输入的方法及装置
CN102902351A (zh) * 2011-07-25 2013-01-30 富泰华工业（深圳）有限公司 触摸式电子装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101719022A (zh) * 2010-01-05 2010-06-02 汉王科技股份有限公司 用于全功能键盘的字符输入方法及其处理装置
CN101937313A (zh) * 2010-09-13 2011-01-05 中兴通讯股份有限公司 一种触摸键盘动态生成和输入的方法及装置
CN102902351A (zh) * 2011-07-25 2013-01-30 富泰华工业（深圳）有限公司 触摸式电子装置

Similar Documents

Publication Publication Date Title
US11868609B2 (en) Dynamic soft keyboard
US10140284B2 (en) Partial gesture text entry
US9851809B2 (en) User interface control using a keyboard
KR101366723B1 (ko) 다중 접점 문자 입력 방법 및 시스템
US9158757B2 (en) System and method for preview and selection of words
KR101602840B1 (ko) 스마트 사용자 커스터마이징된 가상 키보드
US20150193141A1 (en) Continuous Handwriting UI
US20110296333A1 (en) User interaction gestures with virtual keyboard
US20140198047A1 (en) Reducing error rates for touch based keyboards
EP2575009A2 (en) User interface method for a portable terminal
US20190196712A1 (en) Systems and Methods for Facilitating Data Entry into Small Screen Electronic Devices
US20180046224A1 (en) Input Apparatus, Information Processing Method, And Information Processing Apparatus
WO2020006676A1 (zh) 利用触摸软键盘的盲式输入法、计算装置和存储介质
CN111367459B (zh) 利用压力触控板的文本输入方法和智能电子设备
CN109240590A (zh) 用于虚拟键盘的输入控制方法及装置
US9261973B2 (en) Method and system for previewing characters based on finger position on keyboard
EP2811371B1 (en) Method and system for previewing characters based on finger position on keyboard

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18925401

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18925401

Country of ref document: EP

Kind code of ref document: A1