JP2005352739A - Portable terminal device, input system and information input method - Google Patents

Portable terminal device, input system and information input method Download PDF

Info

Publication number
JP2005352739A
JP2005352739A JP2004172336A JP2004172336A JP2005352739A JP 2005352739 A JP2005352739 A JP 2005352739A JP 2004172336 A JP2004172336 A JP 2004172336A JP 2004172336 A JP2004172336 A JP 2004172336A JP 2005352739 A JP2005352739 A JP 2005352739A
Authority
JP
Japan
Prior art keywords
terminal device
user
input information
sensor
control
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
JP2004172336A
Other languages
Japanese (ja)
Other versions
JP4379214B2 (en
Inventor
Hiroshi Otake
浩史 大竹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to JP2004172336A priority Critical patent/JP4379214B2/en
Publication of JP2005352739A publication Critical patent/JP2005352739A/en
Application granted granted Critical
Publication of JP4379214B2 publication Critical patent/JP4379214B2/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Images

Landscapes

  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Position Input By Displaying (AREA)
  • User Interface Of Digital Computer (AREA)
  • Arrangements For Transmission Of Measured Signals (AREA)

Abstract

<P>PROBLEM TO BE SOLVED: To solve the following problem that when making a portable terminal device perform operation intended by a user, it is difficult to miniaturize the portable terminal device while securing sufficient operability because using a keyboard, a keypad or the like. <P>SOLUTION: A piezoelectric sensor 25 is installed to a position of a tendon of a wrist of the user to indirectly measure movement of a finger of the user, and an acceleration sensor 23 measures movement of an arm of the user. A CPU 12 operates according to a value measured by each of the sensors and stored by a memory 16 for control definition and operation input information that is information showing relation with a process to be executed or a character to be inputted. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

本発明は、使用者の指や手首の腱および腕の動きを測定することによって、使用者が意図する動作を行うことができる携帯端末装置、入力システム、情報入力方法に関する。   The present invention relates to a portable terminal device, an input system, and an information input method capable of performing an operation intended by a user by measuring movements of a user's finger, wrist tendon, and arm.

従来、携帯端末装置に情報を入力するには、主にキーボードやキーパッド、入力ペン等の入力装置を使用していた。しかし、このような入力装置を、腕時計型やブレスレット型等の小型の端末に、十分な操作性を確保しつつ入力機能を搭載させることは難しい。   Conventionally, input devices such as a keyboard, a keypad, and an input pen have been mainly used to input information to a mobile terminal device. However, it is difficult to mount such an input device on a small terminal such as a wristwatch type or a bracelet type while ensuring sufficient operability.

そのため、筋電位などの生体信号や、指が物体を叩いたときの振動や加速度等を測定することによって、情報を入力する方法が提案されている。しかし、筋電位を測定する場合、皮膚表面の電気抵抗は発汗や周囲の湿度の影響を受けるため、医療現場等では、電解ペーストを皮膚に塗布した上に測定用の電極を貼付している。このような作業は、携帯端末装置の使用者の負担が大きい。また、周囲の電気製品等からの電磁波によるノイズが混入することが多く、電磁波によるノイズは筋電位を測定する妨げとなる。特に、入力を行う携帯端末装置が、携帯電話機やＰＨＳ（Ｐｅｒｓｏｎａｌ Ｈａｎｄｙｐｈｏｎｅ Ｓｙｓｔｅｍ）端末である場合には、電磁波の影響を避けることができない。   Therefore, a method for inputting information by measuring a biological signal such as myoelectric potential, vibration or acceleration when a finger hits an object, and the like has been proposed. However, when measuring the myoelectric potential, the electrical resistance of the skin surface is affected by perspiration and ambient humidity. Therefore, in the medical field or the like, the electrode for measurement is applied after the electrolytic paste is applied to the skin. Such work places a heavy burden on the user of the mobile terminal device. In addition, noise due to electromagnetic waves from surrounding electrical products or the like is often mixed, and the noise due to electromagnetic waves hinders measurement of myoelectric potential. In particular, when the mobile terminal device that performs input is a mobile phone or a PHS (Personal Handyphone System) terminal, the influence of electromagnetic waves cannot be avoided.

動きセンサを有する入力センサ装置を使用者の指先に装着して任意の対象に接触させ、対象の表面に沿って移動させる場合に生じる力に応じて、文字や命令の入力を認識する入力システムが提案されている（例えば、特許文献１参照。）。   An input system for recognizing input of characters and commands according to a force generated when an input sensor device having a motion sensor is attached to a user's fingertip, brought into contact with an arbitrary object, and moved along the surface of the object It has been proposed (see, for example, Patent Document 1).

また、加速度センサやジャイロセンサ、慣性センサによるセンサを指に装着して指の動きを測定し、測定した指の動きを画面に表示してある仮想キーボード上に重畳して表示し、指の動きに応じて、仮想キーボードのキー入力を検出する情報入力システムが提案されている（例えば、特許文献２参照。）。   In addition, an accelerometer, gyro sensor, or inertial sensor is attached to the finger to measure the movement of the finger, and the measured finger movement is displayed superimposed on a virtual keyboard displayed on the screen. In response to this, an information input system for detecting a key input of a virtual keyboard has been proposed (see, for example, Patent Document 2).

特開２００１−１４２６３１号公報 （段落（００２９）〜（０１０１）、図１）JP 2001-142631 A (paragraphs (0029) to (0101), FIG. 1) 特開２００２−２７８６７３号公報 （段落（００１２）〜（００１９）、図３）JP 2002-278673 A (paragraphs (0012) to (0019), FIG. 3)

しかし、特許文献１に記載されているシステムは、動きセンサを接触させる対象物が必要であり、ジェスチャーなどの動きを検出することができない。   However, the system described in Patent Document 1 requires an object to be brought into contact with the motion sensor, and cannot detect a motion such as a gesture.

また、特許文献２に記載されているシステムは、キーボードを表示することができる程度の大きさの表示手段が必要であり、小型化が求められる携帯端末装置に適用することはできない。   Further, the system described in Patent Document 2 requires display means large enough to display a keyboard, and cannot be applied to a portable terminal device that is required to be downsized.

そこで、本発明は、使用者の指および腕の動きを測定することによって、使用者が意図する動作を行うことができる携帯端末装置、入力システム、情報入力方法を提供することを目的とする。   Therefore, an object of the present invention is to provide a portable terminal device, an input system, and an information input method that can perform an operation intended by the user by measuring the movement of the user's finger and arm.

本発明による携帯端末装置は、使用者の腕に装着されて、使用者の指および腕の動きを間接的に測定して測定量を出力する複数のセンサと、複数種類の測定量または測定量の組み合わせのそれぞれに対応する入力情報を記憶する制御定義用メモリと、センサが出力した測定量に応じた入力情報を制御定義用メモリから抽出し、抽出した入力情報に従って制御を実行する制御手段とを備えたことを特徴とする。   A portable terminal device according to the present invention is mounted on a user's arm, indirectly measures the movement of the user's finger and arm, and outputs a measurement amount, and a plurality of types of measurement amounts or measurement amounts. A control definition memory for storing input information corresponding to each of the combinations, a control means for extracting input information corresponding to the measurement amount output from the sensor from the control definition memory, and executing control according to the extracted input information; It is provided with.

複数のセンサのうち、一部のセンサは、使用者の手首にある指の骨に結ばれている腱の***を測定するセンサであってもよい。そのような構成によれば、使用者の手首や指の動きを間接的に測定することができる。   Among the plurality of sensors, some sensors may be sensors that measure the tendon bulges tied to the bones of the fingers on the wrist of the user. According to such a configuration, the movement of the user's wrist or finger can be indirectly measured.

複数のセンサのうち、一部のセンサは、使用者の手首に装着されて使用者の腕の動きの加速度ベクトルを測定するセンサであってもよい。そのような構成によれば、使用者の腕の動きを間接的に測定することができる。   Some of the plurality of sensors may be sensors that are mounted on the user's wrist and measure an acceleration vector of the movement of the user's arm. According to such a configuration, the movement of the user's arm can be indirectly measured.

制御手段は、センサが出力した測定量を用いて演算を行い、演算によって得られた値に応じた入力情報を制御定義用メモリから抽出してもよい。そのような構成によれば、センサが出力した測定量にもとづいて制御を実行することができる。   The control means may perform calculation using the measured amount output from the sensor, and may extract input information corresponding to a value obtained by the calculation from the control definition memory. According to such a configuration, control can be executed based on the measurement amount output by the sensor.

制御定義用メモリおよび制御手段を含む筐体と有線または無線で接続された子機がセンサを搭載してもよい。そのような構成によれば、使用者が腕に装着する装置は子機であるため、使用者が腕に装着する装置をより小型化することができる。   The sensor may be mounted on a slave unit that is wired or wirelessly connected to a housing including the control definition memory and the control means. According to such a configuration, since the device worn on the arm by the user is a slave unit, the device worn on the arm by the user can be further downsized.

一の筐体が、センサと制御定義用メモリと制御手段とを搭載してもよい。そのような構成によれば、一の筐体の携帯端末装置が使用者の指および腕の動きを測定し、使用者が意図する動作を行うことができる。   One housing may be equipped with a sensor, a control definition memory, and a control means. According to such a configuration, the mobile terminal device in one housing can measure the movement of the user's finger and arm, and can perform an operation intended by the user.

互いの距離を測定して測定量を出力する機能を有する複数の筐体を含んでもよく、制御定義用メモリは、複数種類の測定量または測定量の組み合わせと筐体間の距離との組み合わせのそれぞれに対応する入力情報を記憶してもよく、制御手段は、センサが出力した測定量と複数の筐体が出力した測定量とに応じた入力情報を制御定義用メモリから抽出し、抽出した入力情報に従って制御を実行してもよい。そのような構成によれば、携帯端末装置は、センサが測定した使用者の指および腕の動きの測定量に加えて、筐体間の距離に応じて制御を実行することができる。   The control definition memory may include a plurality of types of measurement amounts or a combination of measurement amounts and a distance between the cases. Input information corresponding to each may be stored, and the control means extracts the input information corresponding to the measurement amount output from the sensor and the measurement amounts output from the plurality of housings from the control definition memory, and extracts the input information. Control may be executed according to the input information. According to such a configuration, the mobile terminal device can execute control according to the distance between the casings in addition to the measured amount of the user's finger and arm movement measured by the sensor.

本発明による入力システムは、複数種類の測定量または測定量の組み合わせのそれぞれに対応する入力情報を記憶する制御定義用メモリと、使用者の指および腕の動きを間接的に測定する複数のセンサが出力した測定量に応じた入力情報を制御定義用メモリから抽出し、抽出した入力情報に従って制御を実行する制御手段とを含む携帯端末装置と、携帯端末装置と有線または無線で接続された子機とを備え、携帯端末装置は、子機との距離を測定する距離測定手段を含み、携帯端末装置または子機は、使用者の腕に装着されるセンサを含むことを特徴とする。   An input system according to the present invention includes a control definition memory that stores input information corresponding to each of a plurality of types of measurement amounts or combinations of measurement amounts, and a plurality of sensors that indirectly measure a user's finger and arm movements. A mobile terminal device including a control means for extracting input information corresponding to the measurement amount output from the control definition memory and executing control according to the extracted input information, and a child connected to the mobile terminal device in a wired or wireless manner The portable terminal device includes distance measuring means for measuring a distance from the child device, and the portable terminal device or the child device includes a sensor attached to a user's arm.

本発明による情報入力方法は、使用者の腕に装着されて、使用者の指および腕の動きをセンサで間接的に測定し、複数種類の測定量または測定量の組み合わせのそれぞれに対応する入力情報を記憶し、センサが出力した測定量に応じた入力情報を抽出し、抽出した入力情報に従って制御を実行することを特徴とする。   The information input method according to the present invention is mounted on a user's arm, indirectly measures the movement of the user's finger and arm with a sensor, and inputs corresponding to each of a plurality of types of measurement amounts or combinations of measurement amounts. Information is memorize | stored, the input information according to the measured amount which the sensor output is extracted, and control is performed according to the extracted input information.

本発明によれば、使用者の指および腕の動きに応じて、使用者が意図する制御を実行することができる。   According to the present invention, the control intended by the user can be executed in accordance with the movement of the user's finger and arm.

また、本発明によれば、大きいサイズのキーボードやキーパッド等を用いることなく、使用者が意図する制御を実行することができるため、携帯端末装置をより小型化することができる。   Further, according to the present invention, since the control intended by the user can be executed without using a large-sized keyboard or keypad, the portable terminal device can be further downsized.

さらに、本発明によれば、ジェスチャーなどの動きを検出して、何らかの対象物に触れることなく使用者が意図する制御を実行することができる。   Furthermore, according to the present invention, it is possible to detect a movement such as a gesture and execute a control intended by the user without touching any object.

本発明の実施の形態について図面を参照して説明する。図１は、本発明による携帯端末装置１１を手首に装着した例を示す説明図である。なお、手首に装着するのは、携帯端末装置１１であっても、携帯端末装置１１に有線または無線で接続され、センサを搭載した子機であってもよいが、この実施の形態では、手首に携帯端末装置１１を装着しているものとして説明する。   Embodiments of the present invention will be described with reference to the drawings. FIG. 1 is an explanatory view showing an example in which a portable terminal device 11 according to the present invention is mounted on a wrist. The wrist terminal may be attached to the mobile terminal device 11 or may be a handset that is connected to the mobile terminal device 11 in a wired or wireless manner and is equipped with a sensor. In the following description, it is assumed that the mobile terminal device 11 is mounted.

ところで、指と手首との筋肉は腕にあり、この筋肉と手の骨とがそれぞれ腱で結ばれている。そして、指や手首を曲げるとその角度に応じてそれぞれの腱の部分が***する。そこで、携帯端末装置１１の、それぞれの腱の部分に接触するような位置に、圧力を電圧に変換する圧電センサ（センサ）２５を配置する。圧電センサ２５は、腱の部分の***によって生じる接触圧を測定する。表示部２２は、白黒またはカラーの液晶パネルや、有機ＥＬ（電子蛍光）等によって実現され、情報を表示する。キーボード１９は、押下して情報を入力するキーを備える。   By the way, the muscles of fingers and wrists are in the arms, and these muscles and the bones of the hands are connected by tendons. When the finger or wrist is bent, each tendon portion rises according to the angle. Therefore, a piezoelectric sensor (sensor) 25 that converts pressure into voltage is disposed at a position of the mobile terminal device 11 that is in contact with each tendon portion. The piezoelectric sensor 25 measures the contact pressure caused by the bulging of the tendon portion. The display unit 22 is realized by a monochrome or color liquid crystal panel, organic EL (electrofluorescence), or the like, and displays information. The keyboard 19 includes keys that are pressed to input information.

図２は、本発明による携帯端末装置１１の一構成例を示すブロック図である。携帯端末装置１１は、携帯端末装置１１の各部を制御するＣＰＵ（Ｃｅｎｔｒａｌ Ｐｒｏｃｅｓｓｉｎｇ Ｕｎｉｔ）（制御手段）１２、情報を入力するキーボード１９、キーボード１９に入力された情報をＣＰＵ１２に出力するキー入力部１８、情報を表示する表示部２２、表示部２２を制御する表示制御部２１、３次元の加速度ベクトルを測定する加速度センサ２３、複数の圧電センサ２５、圧電センサ２５を制御し、圧電センサ２５が測定した値をＣＰＵ１２に出力する圧力入力部２４、ＣＰＵ１２が実行する各種の制御プログラムや、かな文字変換等に使用される辞書等の固定的なデータを記憶するＲＯＭ（Ｒｅａｄ Ｏｎｌｙ Ｍｅｍｏｒｙ）１４、ＣＰＵ１２がプログラムを実行する上で一時的に必要とされるデータを記憶する作業用メモリ１５、圧電センサ２５と加速度センサ２３とが測定した値と、それに応じて実行するべき処理または入力するべき文字との関係の情報を記憶する制御定義用メモリ１６、各センサが測定するタイミングを示す信号であるトリガをＣＰＵ１２に出力するタイマ部１７、ＣＰＵ１２と各部を接続するバス１３を含む。   FIG. 2 is a block diagram showing a configuration example of the mobile terminal device 11 according to the present invention. The mobile terminal device 11 includes a CPU (Central Processing Unit) 12 that controls each unit of the mobile terminal device 11, a keyboard 19 that inputs information, and a key input unit 18 that outputs information input to the keyboard 19 to the CPU 12. A display unit 22 for displaying information, a display control unit 21 for controlling the display unit 22, an acceleration sensor 23 for measuring a three-dimensional acceleration vector, a plurality of piezoelectric sensors 25, a piezoelectric sensor 25, and the piezoelectric sensor 25 for measuring A pressure input unit 24 for outputting the measured value to the CPU 12, a ROM (Read Only Memory) 14 for storing fixed data such as various control programs executed by the CPU 12, a dictionary used for kana character conversion, and the like. Stores data that is temporarily needed to execute a program The industrial memory 15, the control definition memory 16 that stores information on the relationship between the values measured by the piezoelectric sensor 25 and the acceleration sensor 23, and the processing to be executed or the characters to be input in response thereto, each sensor measures The timer part 17 which outputs the trigger which is a signal which shows a timing to CPU12, and the bus | bath 13 which connects CPU12 and each part are included.

作業用メモリ１５は、例えば、ＲＡＭ（Ｒａｎｄｏｍ Ａｃｃｅｓｓ Ｍｅｍｏｒｙ）によって実現される。加速度センサ２３は、使用者の腕の３次元空間上における加速度ベクトルを測定し、３次元加速度センサによって実現されるが、２次元以下の空間の加速度ベクトルを測定するものでもよいし、例えば、複数の１次元の加速度センサを、それぞれが垂直方向の加速度を測定するように設置したものでもよい。   The working memory 15 is realized by, for example, a RAM (Random Access Memory). The acceleration sensor 23 measures the acceleration vector in the three-dimensional space of the user's arm and is realized by the three-dimensional acceleration sensor. However, the acceleration sensor 23 may measure an acceleration vector in a two-dimensional space or less. These one-dimensional acceleration sensors may be installed so as to measure the acceleration in the vertical direction.

制御定義用メモリ１６は、例えば、不揮発性メモリによって実現される。なお、例えば、圧電センサ２５と加速度センサ２３とが測定する値、または、圧電センサ２５と加速度センサ２３とが測定する値の組み合わせにもとづいて、指、手首および腕の動きがパターン化され、携帯端末装置１１の工場出荷時等に、制御定義用メモリ１６は、パターン化された動きと、それに応じて実行すべき処理または入力すべき文字との関係を示す情報である動作入力情報を予め記憶している。また、使用者は、キーボード１９を操作して、パターン化された動きと、それに応じて実行すべき処理または入力すべき文字との関係の情報である動作入力情報を、制御定義用メモリ１６に記憶させてもよい。タイマ部１７は、所定の時間間隔（例えば、１０ｍｓ毎）で、トリガをＣＰＵ１２に出力し、ＣＰＵ１２は、トリガが入力されると、加速度センサ２３に加速度ベクトルを測定させ、圧力入力部２４を介して圧電センサ２５に接触圧を測定させる。なお、ＣＰＵ１２は、加速度センサ２３および圧電センサ２５が測定した測定量を用いて所定の演算を行い、演算によって得られた値に応じた動作入力情報を制御定義用メモリから抽出し、抽出した動作入力情報に従って処理を実行したり、文字を入力したりする制御を実行してもよい。   The control definition memory 16 is realized by, for example, a nonvolatile memory. For example, the movements of fingers, wrists, and arms are patterned based on values measured by the piezoelectric sensor 25 and the acceleration sensor 23 or combinations of values measured by the piezoelectric sensor 25 and the acceleration sensor 23. When the terminal device 11 is shipped from the factory, the control definition memory 16 stores in advance operation input information that is information indicating the relationship between the patterned movement and the process to be executed or the character to be input. doing. Further, the user operates the keyboard 19 to store operation input information, which is information on the relationship between the patterned movement and the process to be executed or the character to be input, in the control definition memory 16. It may be memorized. The timer unit 17 outputs a trigger to the CPU 12 at a predetermined time interval (for example, every 10 ms), and when the trigger is input, the CPU 12 causes the acceleration sensor 23 to measure an acceleration vector and sends it through the pressure input unit 24. Then, the piezoelectric sensor 25 is caused to measure the contact pressure. The CPU 12 performs a predetermined calculation using the measurement amounts measured by the acceleration sensor 23 and the piezoelectric sensor 25, extracts operation input information corresponding to the value obtained by the calculation from the control definition memory, and extracts the extracted operation. You may perform control which performs a process according to input information, or inputs a character.

携帯端末装置１１は、例えば、携帯電話機やＰＨＳ端末、ＰＤＡ（Ｐｅｒｓｏｎａｌ ＤａｔａＡｓｓｉｓｔａｎｃｅ、Ｐｅｒｓｏｎａｌ Ｄｉｇｉｔａｌ Ａｓｓｉｓｔａｎｔｓ）等によって実現される。   The mobile terminal device 11 is realized by, for example, a mobile phone, a PHS terminal, a PDA (Personal Data Assistance, Personal Digital Assistant) or the like.

次に、実施の形態の動作について図面を参照して説明する。図３は、本発明の実施の形態の動作を説明するフローチャートである。   Next, the operation of the embodiment will be described with reference to the drawings. FIG. 3 is a flowchart for explaining the operation of the embodiment of the present invention.

携帯端末装置１１の電源が投入されると、タイマ部１７は、所定の時間間隔でトリガをＣＰＵ１２に出力する。ＣＰＵ１２は、トリガが入力されると、加速度センサ２３に加速度ベクトルを測定させ、圧力入力部２４を介して圧電センサ２５に接触圧を測定させる。このとき、使用者が、携帯端末装置１１を装着した腕や指を動かすと、加速度センサ２３は、加速度ベクトルを測定し、圧電センサ２５は、腱の***による接触圧を測定する。圧電センサ２５は、測定した接触圧を圧力入力部２４に入力する。ＣＰＵ１２は、加速度センサ２３が測定した加速度ベクトルを、作業用メモリ１５に記憶させる（ステップＳ１０１）。さらに、ＣＰＵ１２は、圧電センサ２５が圧力入力部２４に入力した接触圧を、作業用メモリ１５に記憶させる（ステップＳ１０２）。   When the mobile terminal device 11 is powered on, the timer unit 17 outputs a trigger to the CPU 12 at a predetermined time interval. When the trigger is input, the CPU 12 causes the acceleration sensor 23 to measure the acceleration vector, and causes the piezoelectric sensor 25 to measure the contact pressure via the pressure input unit 24. At this time, when the user moves the arm or finger wearing the portable terminal device 11, the acceleration sensor 23 measures the acceleration vector, and the piezoelectric sensor 25 measures the contact pressure due to the tendon bulge. The piezoelectric sensor 25 inputs the measured contact pressure to the pressure input unit 24. The CPU 12 stores the acceleration vector measured by the acceleration sensor 23 in the work memory 15 (step S101). Further, the CPU 12 stores the contact pressure input to the pressure input unit 24 by the piezoelectric sensor 25 in the work memory 15 (step S102).

ＣＰＵ１２は、作業用メモリ１５が記憶している加速度ベクトルおよび接触圧にもとづいて、使用者の指、手首および腕の動きをパターン認識し（ステップＳ１０３）、パターン認識した動きと、制御定義用メモリ１６が記憶している動作入力情報とを比較し、動きのパターンを判別する（ステップＳ１０４）。   The CPU 12 recognizes the movement of the user's finger, wrist and arm based on the acceleration vector and the contact pressure stored in the work memory 15 (step S103). The motion input information stored in 16 is compared with each other to determine the motion pattern (step S104).

ＣＰＵ１２は、パターン認識した動きと、制御定義用メモリ１６が記憶している動作入力情報のパターン化した動きとが合致した場合、動作入力情報に従って、パターン化した動きに応じて実行するべき処理を行ったり、文字を入力したりする（ステップＳ１０５）。   When the movement recognized by the pattern matches the movement of the movement input information stored in the control definition memory 16, the CPU 12 performs a process to be executed according to the movement of the pattern according to the movement input information. Go or input characters (step S105).

ＣＰＵ１２は、パターン認識した動きと、制御定義用メモリ１６が記憶している動作入力情報のパターン化した動きとが合致しない場合、所定時間経過後（ステップＳ１０６）、ステップＳ１０１に戻る。   If the movement recognized by the pattern does not match the movement patterned by the motion input information stored in the control definition memory 16, the CPU 12 returns to step S101 after a predetermined time has elapsed (step S106).

以上、述べたように、この実施の形態によれば、指や手首の屈伸と、腕の移動または回転の組み合わせによって、使用者は、携帯端末装置１１に対して、意図する操作または文字の入力を行うことができる。そのため、サイズの大きいキーボードやキーパッドが不要になり、携帯端末装置１１をより小型化することができる。   As described above, according to this embodiment, the user can input an intended operation or character to the mobile terminal device 11 by a combination of bending and stretching of a finger or wrist and movement or rotation of an arm. It can be performed. Therefore, a large-sized keyboard and keypad are not required, and the portable terminal device 11 can be further downsized.

また、携帯端末装置１１を、手首付近に装着するので、腕時計型やブレスレット型等の小型の携帯端末装置として自然な形で装着することができる。   Further, since the mobile terminal device 11 is mounted near the wrist, it can be mounted in a natural form as a small mobile terminal device such as a wristwatch type or a bracelet type.

さらに、各センサによる測定対象が、腕の運動や指および手首の機械的な変化量であるので、携帯端末装置１１自体からの、または周囲の電気製品などからの電磁波による影響を受けることがない。   Furthermore, since the measurement object by each sensor is the movement of the arm and the mechanical change amount of the finger and wrist, the measurement object is not affected by electromagnetic waves from the mobile terminal device 11 itself or from surrounding electrical products. .

また、使用者は、対象物に触れることなく操作または文字の入力を行うことができるので、ジェスチャーなどの動きを検出することができる。   In addition, since the user can perform an operation or input characters without touching the object, it is possible to detect a movement such as a gesture.

なお、この実施の形態では、指や手首を曲げた角度（圧電センサ２５が測定した腱の接触圧）と腕の移動や回転の運動（加速度センサ２３が測定した加速度ベクトル）との組み合わせによって、携帯端末装置１１が行う処理や入力する文字を決定するが、これらの組み合わせのうち、一部の組み合わせを用いて携帯端末装置１１が行う処理や入力する文字を決定してもよいし、キーボード１９への入力と組み合わせて携帯端末装置１１が行う処理や入力する文字を決定してもよい。また、１つのセンサが測定した量にもとづいて、携帯端末装置１１が行う処理や入力する文字を決定してもよい。   In this embodiment, the combination of the angle at which the finger or wrist is bent (contact pressure of the tendon measured by the piezoelectric sensor 25) and the movement or rotation of the arm (acceleration vector measured by the acceleration sensor 23) The processing performed by the mobile terminal device 11 and the character to be input are determined. Of these combinations, the processing performed by the mobile terminal device 11 and the character to be input may be determined using some combinations. The process performed by the mobile terminal device 11 and the character to be input may be determined in combination with the input to the. Further, based on the amount measured by one sensor, the process performed by the mobile terminal device 11 and the character to be input may be determined.

また、この実施の形態では、指の腱の***を測定するセンサとして圧電センサ２５を用いたが、本発明はこれに限定されるものではなく、加速度センサや、歪みゲージ等の、皮膚の起伏の変位が測定できる他のセンサを用いてもよい。腕の移動や回転の運動を測定するセンサとして加速度センサ２３を用いたが、質量計と感圧センサを組み合わせて用いたり、ジャイロセンサが測定した姿勢変化量と姿勢変化速度と姿勢変化加速度とを用いたりして腕の移動や回転の運動を測定してもよい。   In this embodiment, the piezoelectric sensor 25 is used as a sensor for measuring the bulging of the tendon of the finger. However, the present invention is not limited to this, and the undulation of the skin such as an acceleration sensor or a strain gauge is used. Other sensors capable of measuring the displacement of may be used. Although the acceleration sensor 23 is used as a sensor for measuring the movement of the arm and the movement of the rotation, the mass sensor and the pressure sensor are used in combination, or the posture change amount, posture change speed, and posture change acceleration measured by the gyro sensor are used. It may be used to measure arm movement and rotation.

使用者の指や手首、腕の動きのパターンに応じて実行されるべき携帯端末装置１１の処理は、例えば、メニュー選択機能や、かな文字変換の機能、カーソル移動機能、画面スクロール機能、各種機能の呼び出し等である。   The processing of the mobile terminal device 11 to be executed according to the user's finger, wrist, and arm movement patterns includes, for example, a menu selection function, a kana character conversion function, a cursor movement function, a screen scroll function, and various functions. And so on.

使用者の腕の動きを、加速度センサ２３が測定した加速度ベクトルのうち、携帯端末装置１１の移動による成分を時間で１回積分して得られる速度ベクトルや、２回積分して得られる変位ベクトルから求めてもよい。   Of the acceleration vectors measured by the acceleration sensor 23, the velocity vector obtained by integrating the component due to the movement of the mobile terminal device once in time, or the displacement vector obtained by integrating twice, the acceleration vector measured by the acceleration sensor 23. You may ask for.

圧電センサ２５が測定した接触圧や、加速度センサ２３が測定した加速度ベクトルから使用者の指や手首、腕の動きのパターンを判別する基準を、使用者の操作によって、変更可能にしてもよい。すると、例えば、携帯端末装置１１を初めて利用する使用者は、大きな変化量で入力を確定することによって誤入力を抑制することができ、携帯端末装置１１の操作に慣れた使用者は、小さな変化量で入力を確定することによって、入力操作を迅速に行うことができる。なお、圧電センサ２５が測定した接触圧や、加速度センサ２３が測定した加速度ベクトルから使用者の指や手首、腕の動きのパターンを判別する基準を、使用者の操作によって、変更可能にしなくてもよい。その場合、動作入力情報をＲＯＭ１４に記憶させてもよい。   The reference for determining the movement pattern of the user's finger, wrist, and arm from the contact pressure measured by the piezoelectric sensor 25 and the acceleration vector measured by the acceleration sensor 23 may be changed by the user's operation. Then, for example, the user who uses the mobile terminal device 11 for the first time can suppress erroneous input by confirming the input with a large change amount, and the user who is used to the operation of the mobile terminal device 11 can make a small change. By confirming the input by the amount, the input operation can be performed quickly. It should be noted that the reference for discriminating the movement pattern of the user's finger, wrist and arm from the contact pressure measured by the piezoelectric sensor 25 and the acceleration vector measured by the acceleration sensor 23 must be changeable by the user's operation. Also good. In that case, the operation input information may be stored in the ROM 14.

携帯端末装置１１の消費電力を低減するために、ＣＰＵ１２は、圧電センサ２５および加速度センサ２３への入力時以外は、圧電センサ２５および加速度センサ２３の電源を切ったりして、パワーセーブ状態にしてもよい。   In order to reduce the power consumption of the portable terminal device 11, the CPU 12 turns off the power of the piezoelectric sensor 25 and the acceleration sensor 23 except when inputting to the piezoelectric sensor 25 and the acceleration sensor 23, and sets the power saving state. Also good.

なお、この実施の形態では、図１の例で示したように、圧電センサ２５を手首の円周に沿って直線上に配置したが、圧電センサ２５を２次元の平面上に配置したり、腱の***の移動を測定することができる平面状の圧電センサを用いてもよい。すると、腱の***の変化をより正確に測定することができる。すなわち、より複雑な指の動きを測定することが可能になる。   In this embodiment, as shown in the example of FIG. 1, the piezoelectric sensor 25 is arranged on a straight line along the circumference of the wrist, but the piezoelectric sensor 25 is arranged on a two-dimensional plane, A planar piezoelectric sensor that can measure the movement of tendon ridges may be used. Then, the change in the tendon bulge can be measured more accurately. That is, it is possible to measure more complicated finger movements.

また、制御定義用メモリ１６は、手話による手と腕との動きにもとづく圧電センサ２５と加速度センサ２３とが測定した値と、それに対応する文字や単語との関係の情報を記憶してもよい。聴覚にハンディキャップを持つ人は、手話でコミュニケーションを行うのが一般的であるが、制御定義用メモリ１６が、手話による手と腕との動きと、文字や単語との対応関係を示す情報を記憶していると、手話による手と腕との動きに応じて、携帯端末装置１１に文字や単語を入力することができる。すると、聴覚にハンディキャップを持つ人が、携帯端末装置１１に文字や単語を入力するために、新たな指や腕の動きを習得する必要がなくなる。   In addition, the control definition memory 16 may store information on the relationship between the values measured by the piezoelectric sensor 25 and the acceleration sensor 23 based on the movement of the hand and arm in sign language, and the corresponding characters and words. . Persons with hearing handicap generally communicate in sign language, but the control definition memory 16 stores information indicating the correspondence between the movement of the hand and arm in sign language and the character or word. If stored, characters and words can be input to the mobile terminal device 11 in accordance with the movement of the hand and arm in sign language. This eliminates the need for a person with hearing handicap to learn new finger and arm movements in order to input characters and words into the mobile terminal device 11.

ＣＰＵ１２は、各センサから、所定の長さの時間以上の入力や、所定の大きさ以上の入力、測定値の変化が所定の大きさ以上である入力があった場合に、携帯端末装置１１への入力であると判断してもよい。すると、チャタリングによる誤入力を防ぐことができる。   When there is an input from each sensor over a predetermined length of time, an input over a predetermined size, or an input whose measurement value changes over a predetermined size, the CPU 12 sends it to the mobile terminal device 11. It may be determined that the input is. Then, erroneous input due to chattering can be prevented.

また、携帯端末装置１１は、互いの距離を測定する機能を有し、有線または無線で接続された複数のユニットによって構成されてもよい。このとき、ＣＰＵ１２と、キーボード１９と、キー入力部１８と、表示部２２と、表示制御部２１と、ＲＯＭ１４と、作業用メモリ１５と、制御定義用メモリ１７と、タイマ部１７とを含むユニット（以下、親機という。）は、例えば、音波によって他のユニット（以下、子機という。）との距離を測定する距離測定手段を含む。そして、制御定義用メモリ１６は、手と腕との動きにもとづく圧電センサ２５と加速度センサ２３とが測定した値、および子機との間の距離と、それに対応する文字および処理との関係の情報を記憶してもよい。すると、指や手の動きに加えて、親機と子機との間の距離に応じて、携帯端末装置１１が行う処理や文字を入力することができる。なお、このとき、各センサは、親機が搭載していても、子機が搭載していてもよく、複数のセンサの一部を親機が搭載し、複数のセンサの他の一部を子機が搭載していてもよい。また、距離測定手段を子機が搭載していてもよい。   Moreover, the portable terminal device 11 has a function of measuring the distance between each other, and may be configured by a plurality of units connected by wire or wirelessly. At this time, a unit including the CPU 12, the keyboard 19, the key input unit 18, the display unit 22, the display control unit 21, the ROM 14, the work memory 15, the control definition memory 17, and the timer unit 17. (Hereinafter, referred to as a parent device) includes, for example, a distance measuring unit that measures a distance from another unit (hereinafter, referred to as a child device) using sound waves. The control definition memory 16 stores the relationship between the values measured by the piezoelectric sensor 25 and the acceleration sensor 23 based on the movement of the hand and the arm, the distance between the slave unit, and the corresponding character and processing. Information may be stored. Then, in addition to the movements of fingers and hands, processing and characters performed by the mobile terminal device 11 can be input in accordance with the distance between the parent device and the child device. At this time, each sensor may be mounted on the master unit or on the slave unit. A part of the plurality of sensors is mounted on the base unit, and another part of the plurality of sensors is mounted. The slave unit may be installed. Moreover, the slave unit may be equipped with a distance measuring means.

本発明は、携帯電話機やＰＨＳ端末、ＰＤＡ等の携帯端末装置に利用することができる。   The present invention can be used for mobile terminal devices such as mobile phones, PHS terminals, and PDAs.

本発明による携帯端末装置を手首に装着した例を示す説明図である。It is explanatory drawing which shows the example which mounted | wore the wrist with the portable terminal device by this invention. 本発明による携帯端末装置の一構成例を示すブロック図である。It is a block diagram which shows the example of 1 structure of the portable terminal device by this invention. 本発明の実施の形態の動作を説明するフローチャートである。It is a flowchart explaining operation | movement of embodiment of this invention.

符号の説明Explanation of symbols

１１ 携帯端末装置
１２ ＣＰＵ
１３ バス
１４ ＲＯＭ
１５ 作業用メモリ
１６ 制御定義用メモリ
１７ タイマ部
１８ キー入力部
１９ キーボード
２１ 表示制御部
２２ 表示部
２３ 加速度センサ
２４ 圧力入力部
２５ 圧電センサ 11 Mobile terminal device 12 CPU
13 bus 14 ROM
DESCRIPTION OF SYMBOLS 15 Memory for work 16 Memory for control definition 17 Timer part 18 Key input part 19 Keyboard 21 Display control part 22 Display part 23 Acceleration sensor 24 Pressure input part 25 Piezoelectric sensor

Claims (9)

使用者の腕に装着されて、使用者の指および腕の動きを間接的に測定して測定量を出力する複数のセンサと、
複数種類の測定量または測定量の組み合わせのそれぞれに対応する入力情報を記憶する制御定義用メモリと、
前記センサが出力した測定量に応じた入力情報を前記制御定義用メモリから抽出し、抽出した入力情報に従って制御を実行する制御手段と
を備えたことを特徴とする携帯端末装置。 A plurality of sensors that are attached to the user's arm and indirectly measure the movement of the user's finger and arm and output a measurement amount;
A control definition memory for storing input information corresponding to each of a plurality of types of measurement amounts or combinations of measurement amounts;
A mobile terminal device comprising: control means for extracting input information corresponding to the measurement amount output from the sensor from the control definition memory and executing control according to the extracted input information. 複数のセンサのうち、一部のセンサは、使用者の手首にある指の骨に結ばれている腱の***を測定するセンサである
請求項１記載の携帯端末装置。 The mobile terminal device according to claim 1, wherein some of the plurality of sensors are sensors that measure a tendon bulge tied to a finger bone on a user's wrist. 複数のセンサのうち、一部のセンサは、使用者の手首に装着されて使用者の腕の動きの加速度ベクトルを測定するセンサである
請求項１または請求項２記載の携帯端末装置。 The portable terminal device according to claim 1, wherein some of the plurality of sensors are sensors that are attached to a user's wrist and measure an acceleration vector of a user's arm movement. 制御手段は、センサが出力した測定量を用いて演算を行い、演算によって得られた値に応じた入力情報を前記制御定義用メモリから抽出する
請求項１から請求項３のうちいずれか１項記載の携帯端末装置。 The control means performs an operation using the measured amount output from the sensor, and extracts input information corresponding to a value obtained by the operation from the control definition memory. The portable terminal device described. 制御定義用メモリおよび制御手段を含む筐体と有線または無線で接続された子機がセンサを搭載する
請求項１から請求項４のうちいずれか１項記載の携帯端末装置。 The portable terminal device according to any one of claims 1 to 4, wherein a sensor is mounted on a slave unit wired or wirelessly connected to a housing including a control definition memory and a control unit. 一の筐体が、センサと制御定義用メモリと制御手段とを搭載する
請求項１から請求項４のうちいずれか１項記載の携帯端末装置。 The portable terminal device according to any one of claims 1 to 4, wherein a single housing includes a sensor, a control definition memory, and a control unit. 互いの距離を測定して測定量を出力する機能を有する複数の筐体を含み、
制御定義用メモリは、複数種類の測定量または測定量の組み合わせと前記筐体間の距離との組み合わせのそれぞれに対応する入力情報を記憶し、
制御手段は、センサが出力した測定量と前記複数の筐体が出力した測定量とに応じた入力情報を前記制御定義用メモリから抽出し、抽出した入力情報に従って制御を実行する
請求項１から請求項５のうちいずれか１項記載の携帯端末装置。 Including a plurality of housings having a function of measuring each other's distance and outputting a measurement amount;
The control definition memory stores input information corresponding to a plurality of types of measurement amounts or combinations of measurement amounts and combinations of distances between the housings,
The control means extracts input information corresponding to the measurement amount output from the sensor and the measurement amounts output from the plurality of housings from the control definition memory, and executes control according to the extracted input information. The mobile terminal device according to claim 5. 複数種類の測定量または測定量の組み合わせのそれぞれに対応する入力情報を記憶する制御定義用メモリと、使用者の指および腕の動きを間接的に測定する複数のセンサが出力した測定量に応じた入力情報を前記制御定義用メモリから抽出し、抽出した入力情報に従って制御を実行する制御手段とを含む携帯端末装置と、
前記携帯端末装置と有線または無線で接続された子機とを備え、
前記携帯端末装置は、前記子機との距離を測定する距離測定手段を含み、
前記携帯端末装置または前記子機は、使用者の腕に装着される前記センサを含む
ことを特徴とする入力システム。 Control definition memory that stores input information corresponding to each of multiple types of measurement amounts or combinations of measurement amounts, and measurement amounts output by multiple sensors that indirectly measure user's finger and arm movements A mobile terminal device including control means for extracting the input information from the control definition memory and executing control according to the extracted input information;
A portable device connected to the portable terminal device wired or wirelessly;
The portable terminal device includes distance measuring means for measuring a distance from the slave unit,
The portable terminal device or the slave unit includes the sensor that is worn on a user's arm. 使用者の腕に装着されて、使用者の指および腕の動きをセンサで間接的に測定し、
複数種類の測定量または測定量の組み合わせのそれぞれに対応する入力情報を記憶し、
前記センサが出力した測定量に応じた入力情報を抽出し、抽出した入力情報に従って制御を実行する
ことを特徴とする情報入力方法。 Attached to the user's arm, the movement of the user's finger and arm is indirectly measured with a sensor,
Stores input information corresponding to each of multiple types of measurement quantities or combinations of measurement quantities,
An information input method comprising: extracting input information corresponding to a measurement amount output by the sensor and executing control according to the extracted input information.
JP2004172336A 2004-06-10 2004-06-10 Mobile terminal device Expired - Fee Related JP4379214B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2004172336A JP4379214B2 (en) 2004-06-10 2004-06-10 Mobile terminal device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2004172336A JP4379214B2 (en) 2004-06-10 2004-06-10 Mobile terminal device

Publications (2)

Publication Number Publication Date
JP2005352739A true JP2005352739A (en) 2005-12-22
JP4379214B2 JP4379214B2 (en) 2009-12-09

Family

ID=35587176

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2004172336A Expired - Fee Related JP4379214B2 (en) 2004-06-10 2004-06-10 Mobile terminal device

Country Status (1)

Country Link
JP (1) JP4379214B2 (en)

Cited By (57)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100793079B1 (en) 2006-12-08 2008-01-10 한국전자통신연구원 Wrist-wear user input apparatus and methods
JP2008197801A (en) * 2007-02-09 2008-08-28 Ntt Docomo Inc Mounting type information input device
JP2008284116A (en) * 2007-05-17 2008-11-27 Anima Kk Motor function evaluation system
CN102483640A (en) * 2009-07-23 2012-05-30 高通股份有限公司 Method and apparatus for controlling mobile and consumer electronic devices
JP2012211932A (en) * 2011-03-30 2012-11-01 Toshiba Corp Voice recognition device and voice recognition method
CN103442129A (en) * 2013-08-09 2013-12-11 宇龙计算机通信科技(深圳)有限公司 Interactive method and system between intelligent watch and mobile terminal
US20140043794A1 (en) * 2012-08-09 2014-02-13 Leena Carriere Pressure activated illuminating wristband
CN103593053A (en) * 2013-11-22 2014-02-19 中国科学院深圳先进技术研究院 Intelligent glasses interaction system
JP2014102842A (en) * 2012-11-20 2014-06-05 Samsung Electronics Co Ltd User gesture input to wearable electronic device involving movement of device
US8748997B2 (en) 2008-12-16 2014-06-10 Samsung Electronics Co., Ltd. Contact-force sensor package and method of fabricating the same
WO2014134497A1 (en) * 2013-03-01 2014-09-04 Rufus Armor, Llc Wearable mobile device
CN104042193A (en) * 2014-05-22 2014-09-17 深圳奇沃智联科技有限公司 Watch ring with life coefficient sensing and processing functions
JP2015001978A (en) * 2013-06-17 2015-01-05 三星電子株式会社Ｓａｍｓｕｎｇ Ｅｌｅｃｔｒｏｎｉｃｓ Ｃｏ．，Ｌｔｄ． Method of recognizing motion using gripped object, and device and system therefor
JP2015008443A (en) * 2013-06-26 2015-01-15 セイコーエプソン株式会社 Input device, information processing apparatus and input method
JP2015084595A (en) * 2010-06-16 2015-04-30 クアルコム，インコーポレイテッド Rf ranging-assisted local motion sensing
JP2015088161A (en) * 2013-09-26 2015-05-07 富士通株式会社 Gesture input device, gesture input method, and gesture input program
KR101520904B1 (en) * 2014-03-10 2015-05-18 나시스 주식회사 Communication device using acoustic reflex and resonance
JP2015097556A (en) * 2013-11-18 2015-05-28 セイコーエプソン株式会社 Motion analysis method, motion analysis device, and motion analysis program
JP2015107281A (en) * 2013-12-06 2015-06-11 独立行政法人国立高等専門学校機構 Human body detection device
JP2015220622A (en) * 2014-05-19 2015-12-07 株式会社Ｎｔｔドコモ Function control apparatus and function control method
CN105138112A (en) * 2015-07-13 2015-12-09 腾讯科技（深圳）有限公司 Display control method and apparatus
WO2015199747A1 (en) * 2014-06-23 2015-12-30 Thalmic Labs Inc. Systems, articles, and methods for wearable human-electronics interface devices
JP2016038905A (en) * 2014-08-08 2016-03-22 パナソニックＩｐマネジメント株式会社 Input device and control method of apparatus
KR101629737B1 (en) * 2015-02-25 2016-06-21 연세대학교 산학협력단 Method, device, computer readable recording medium and computer program for inputting letters
CN105763715A (en) * 2014-12-17 2016-07-13 安徽华米信息科技有限公司 Method and device for searching position of terminal, and terminal
JP2016522049A (en) * 2013-05-31 2016-07-28 ナイキ イノベイト シーブイ Dynamic sampling
JP2016143139A (en) * 2015-01-30 2016-08-08 アルパイン株式会社 Information processing apparatus, operation control system, and operation control method
JP2017501469A (en) * 2013-10-24 2017-01-12 アップル インコーポレイテッド Wristband device input using wrist movement
JP2017509957A (en) * 2014-01-26 2017-04-06 ▲華▼▲為▼▲終▼端有限公司 Information processing method, apparatus, and device
WO2017065482A1 (en) * 2015-06-12 2017-04-20 스피어다인 주식회사 Input device and ui configuration and execution method thereof
JP2017518597A (en) * 2014-05-16 2017-07-06 中興通訊股▲ふん▼有限公司Ｚｔｅ Ｃｏｒｐｏｒａｔｉｏｎ Control method, apparatus, electronic device, and computer recording medium
CN107329574A (en) * 2017-06-30 2017-11-07 联想(北京)有限公司 Input method and system for electronic equipment
KR20180018629A (en) * 2018-02-07 2018-02-21 엘지이노텍 주식회사 Watch
KR101833131B1 (en) * 2016-06-08 2018-02-27 엘지이노텍 주식회사 Portable watch
JP2018116730A (en) * 2013-10-14 2018-07-26 ノキア テクノロジーズ オサケユイチア Deviational plane wrist input
US10194060B2 (en) 2012-11-20 2019-01-29 Samsung Electronics Company, Ltd. Wearable electronic device
US10216274B2 (en) 2014-06-23 2019-02-26 North Inc. Systems, articles, and methods for wearable human-electronics interface devices
JP2019032891A (en) * 2018-10-29 2019-02-28 アップル インコーポレイテッドＡｐｐｌｅ Ｉｎｃ． Motion gesture input detected by optical sensor
US10413218B2 (en) 2014-01-15 2019-09-17 Tdk Corporation Wristband-type arm movement determination device and wristband-type activity tracker
US10423214B2 (en) 2012-11-20 2019-09-24 Samsung Electronics Company, Ltd Delegating processing from wearable electronic device
US10551928B2 (en) 2012-11-20 2020-02-04 Samsung Electronics Company, Ltd. GUI transitions on wearable electronic device
US10684692B2 (en) 2014-06-19 2020-06-16 Facebook Technologies, Llc Systems, devices, and methods for gesture identification
US10691332B2 (en) 2014-02-28 2020-06-23 Samsung Electronics Company, Ltd. Text input on an interactive display
US11079846B2 (en) 2013-11-12 2021-08-03 Facebook Technologies, Llc Systems, articles, and methods for capacitive electromyography sensors
US11157436B2 (en) 2012-11-20 2021-10-26 Samsung Electronics Company, Ltd. Services associated with wearable electronic device
US11237719B2 (en) 2012-11-20 2022-02-01 Samsung Electronics Company, Ltd. Controlling remote electronic device with wearable electronic device
US11372536B2 (en) 2012-11-20 2022-06-28 Samsung Electronics Company, Ltd. Transition and interaction model for wearable electronic device
US11422635B2 (en) 2014-02-10 2022-08-23 Apple Inc. Optical sensing device
CN115016635A (en) * 2022-04-18 2022-09-06 深圳原宇科技有限公司 Target control method and system based on motion recognition
US11635736B2 (en) 2017-10-19 2023-04-25 Meta Platforms Technologies, Llc Systems and methods for identifying biological structures associated with neuromuscular source signals
US11644799B2 (en) 2013-10-04 2023-05-09 Meta Platforms Technologies, Llc Systems, articles and methods for wearable electronic devices employing contact sensors
US11666264B1 (en) 2013-11-27 2023-06-06 Meta Platforms Technologies, Llc Systems, articles, and methods for electromyography sensors
US11797087B2 (en) 2018-11-27 2023-10-24 Meta Platforms Technologies, Llc Methods and apparatus for autocalibration of a wearable electrode sensor system
US11868531B1 (en) 2021-04-08 2024-01-09 Meta Platforms Technologies, Llc Wearable device providing for thumb-to-finger-based input gestures detected based on neuromuscular signals, and systems and methods of use thereof
US11907423B2 (en) 2019-11-25 2024-02-20 Meta Platforms Technologies, Llc Systems and methods for contextualized interactions with an environment
US11921471B2 (en) 2013-08-16 2024-03-05 Meta Platforms Technologies, Llc Systems, articles, and methods for wearable devices having secondary power sources in links of a band for providing secondary power in addition to a primary power source
US11961494B1 (en) 2019-03-29 2024-04-16 Meta Platforms Technologies, Llc Electromagnetic interference reduction in extended reality environments

Cited By (79)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100793079B1 (en) 2006-12-08 2008-01-10 한국전자통신연구원 Wrist-wear user input apparatus and methods
WO2008069577A1 (en) * 2006-12-08 2008-06-12 Electronics And Telecommunications Research Institute Wrist-worn input apparatus and method
JP2010511960A (en) * 2006-12-08 2010-04-15 韓國電子通信研究院 Wrist-worn user command input device and method
JP2008197801A (en) * 2007-02-09 2008-08-28 Ntt Docomo Inc Mounting type information input device
JP2008284116A (en) * 2007-05-17 2008-11-27 Anima Kk Motor function evaluation system
US8748997B2 (en) 2008-12-16 2014-06-10 Samsung Electronics Co., Ltd. Contact-force sensor package and method of fabricating the same
US9000887B2 (en) 2009-07-23 2015-04-07 Qualcomm Incorporated Method and apparatus for communicating control information by a wearable device to control mobile and consumer electronic devices
CN102483640A (en) * 2009-07-23 2012-05-30 高通股份有限公司 Method and apparatus for controlling mobile and consumer electronic devices
JP2013500523A (en) * 2009-07-23 2013-01-07 クゥアルコム・インコーポレイテッド Method and apparatus for controlling mobile devices and consumer electronic devices
JP2013500522A (en) * 2009-07-23 2013-01-07 クゥアルコム・インコーポレイテッド Method and apparatus for dispensing user interface using wearable devices to control mobile devices and consumer electronic devices
JP2015046173A (en) * 2009-07-23 2015-03-12 クゥアルコム・インコーポレイテッドＱｕａｌｃｏｍｍ Ｉｎｃｏｒｐｏｒａｔｅｄ Method and apparatus for distributed user interfaces using wearable devices to control mobile and consumer electronic devices
US9030404B2 (en) 2009-07-23 2015-05-12 Qualcomm Incorporated Method and apparatus for distributed user interfaces using wearable devices to control mobile and consumer electronic devices
JP2013500521A (en) * 2009-07-23 2013-01-07 クゥアルコム・インコーポレイテッド Method and apparatus for communicating control information by a wearable device to control mobile devices and consumer electronic devices
US9024865B2 (en) 2009-07-23 2015-05-05 Qualcomm Incorporated Method and apparatus for controlling mobile and consumer electronic devices
JP2015084595A (en) * 2010-06-16 2015-04-30 クアルコム，インコーポレイテッド Rf ranging-assisted local motion sensing
JP2012211932A (en) * 2011-03-30 2012-11-01 Toshiba Corp Voice recognition device and voice recognition method
US10571114B2 (en) 2012-08-09 2020-02-25 Leena M. Carriere Pressure activated illuminating wristband
US9322544B2 (en) * 2012-08-09 2016-04-26 Leena Carriere Pressure activated illuminating wristband
US20160312992A1 (en) * 2012-08-09 2016-10-27 Leena M. Carriere Pressure activated illuminating wristband
US20140043794A1 (en) * 2012-08-09 2014-02-13 Leena Carriere Pressure activated illuminating wristband
US10551928B2 (en) 2012-11-20 2020-02-04 Samsung Electronics Company, Ltd. GUI transitions on wearable electronic device
US10423214B2 (en) 2012-11-20 2019-09-24 Samsung Electronics Company, Ltd Delegating processing from wearable electronic device
JP2014102842A (en) * 2012-11-20 2014-06-05 Samsung Electronics Co Ltd User gesture input to wearable electronic device involving movement of device
US10194060B2 (en) 2012-11-20 2019-01-29 Samsung Electronics Company, Ltd. Wearable electronic device
US11157436B2 (en) 2012-11-20 2021-10-26 Samsung Electronics Company, Ltd. Services associated with wearable electronic device
US10185416B2 (en) 2012-11-20 2019-01-22 Samsung Electronics Co., Ltd. User gesture input to wearable electronic device involving movement of device
US11237719B2 (en) 2012-11-20 2022-02-01 Samsung Electronics Company, Ltd. Controlling remote electronic device with wearable electronic device
US11372536B2 (en) 2012-11-20 2022-06-28 Samsung Electronics Company, Ltd. Transition and interaction model for wearable electronic device
WO2014134497A1 (en) * 2013-03-01 2014-09-04 Rufus Armor, Llc Wearable mobile device
JP2016522049A (en) * 2013-05-31 2016-07-28 ナイキ イノベイト シーブイ Dynamic sampling
US10398358B2 (en) 2013-05-31 2019-09-03 Nike, Inc. Dynamic sampling
US10019078B2 (en) 2013-06-17 2018-07-10 Samsung Electronics Co., Ltd. Device, method, and system to recognize motion using gripped object
US10649549B2 (en) 2013-06-17 2020-05-12 Samsung Electronics Co., Ltd. Device, method, and system to recognize motion using gripped object
JP2015001978A (en) * 2013-06-17 2015-01-05 三星電子株式会社Ｓａｍｓｕｎｇ Ｅｌｅｃｔｒｏｎｉｃｓ Ｃｏ．，Ｌｔｄ． Method of recognizing motion using gripped object, and device and system therefor
JP2015008443A (en) * 2013-06-26 2015-01-15 セイコーエプソン株式会社 Input device, information processing apparatus and input method
CN103442129A (en) * 2013-08-09 2013-12-11 宇龙计算机通信科技(深圳)有限公司 Interactive method and system between intelligent watch and mobile terminal
US11921471B2 (en) 2013-08-16 2024-03-05 Meta Platforms Technologies, Llc Systems, articles, and methods for wearable devices having secondary power sources in links of a band for providing secondary power in addition to a primary power source
JP2015088161A (en) * 2013-09-26 2015-05-07 富士通株式会社 Gesture input device, gesture input method, and gesture input program
US11644799B2 (en) 2013-10-04 2023-05-09 Meta Platforms Technologies, Llc Systems, articles and methods for wearable electronic devices employing contact sensors
JP2018116730A (en) * 2013-10-14 2018-07-26 ノキア テクノロジーズ オサケユイチア Deviational plane wrist input
JP2017501469A (en) * 2013-10-24 2017-01-12 アップル インコーポレイテッド Wristband device input using wrist movement
US11079846B2 (en) 2013-11-12 2021-08-03 Facebook Technologies, Llc Systems, articles, and methods for capacitive electromyography sensors
JP2015097556A (en) * 2013-11-18 2015-05-28 セイコーエプソン株式会社 Motion analysis method, motion analysis device, and motion analysis program
CN103593053A (en) * 2013-11-22 2014-02-19 中国科学院深圳先进技术研究院 Intelligent glasses interaction system
US11666264B1 (en) 2013-11-27 2023-06-06 Meta Platforms Technologies, Llc Systems, articles, and methods for electromyography sensors
WO2015083534A1 (en) * 2013-12-06 2015-06-11 独立行政法人国立高等専門学校機構 Human body detection device
JP2015107281A (en) * 2013-12-06 2015-06-11 独立行政法人国立高等専門学校機構 Human body detection device
US10413218B2 (en) 2014-01-15 2019-09-17 Tdk Corporation Wristband-type arm movement determination device and wristband-type activity tracker
JP2017509957A (en) * 2014-01-26 2017-04-06 ▲華▼▲為▼▲終▼端有限公司 Information processing method, apparatus, and device
US9965044B2 (en) 2014-01-26 2018-05-08 Huawei Device (Dongguan) Co., Ltd. Information processing method, apparatus, and device
US11422635B2 (en) 2014-02-10 2022-08-23 Apple Inc. Optical sensing device
US10691332B2 (en) 2014-02-28 2020-06-23 Samsung Electronics Company, Ltd. Text input on an interactive display
KR101520904B1 (en) * 2014-03-10 2015-05-18 나시스 주식회사 Communication device using acoustic reflex and resonance
EP3139246A4 (en) * 2014-05-16 2018-01-17 ZTE Corporation Control method and apparatus, electronic device, and computer storage medium
JP2017518597A (en) * 2014-05-16 2017-07-06 中興通訊股▲ふん▼有限公司Ｚｔｅ Ｃｏｒｐｏｒａｔｉｏｎ Control method, apparatus, electronic device, and computer recording medium
JP2015220622A (en) * 2014-05-19 2015-12-07 株式会社Ｎｔｔドコモ Function control apparatus and function control method
CN104042193A (en) * 2014-05-22 2014-09-17 深圳奇沃智联科技有限公司 Watch ring with life coefficient sensing and processing functions
US10684692B2 (en) 2014-06-19 2020-06-16 Facebook Technologies, Llc Systems, devices, and methods for gesture identification
WO2015199747A1 (en) * 2014-06-23 2015-12-30 Thalmic Labs Inc. Systems, articles, and methods for wearable human-electronics interface devices
US10216274B2 (en) 2014-06-23 2019-02-26 North Inc. Systems, articles, and methods for wearable human-electronics interface devices
JP2016038905A (en) * 2014-08-08 2016-03-22 パナソニックＩｐマネジメント株式会社 Input device and control method of apparatus
CN105763715A (en) * 2014-12-17 2016-07-13 安徽华米信息科技有限公司 Method and device for searching position of terminal, and terminal
JP2016143139A (en) * 2015-01-30 2016-08-08 アルパイン株式会社 Information processing apparatus, operation control system, and operation control method
KR101629737B1 (en) * 2015-02-25 2016-06-21 연세대학교 산학협력단 Method, device, computer readable recording medium and computer program for inputting letters
WO2017065482A1 (en) * 2015-06-12 2017-04-20 스피어다인 주식회사 Input device and ui configuration and execution method thereof
US10635457B2 (en) 2015-06-12 2020-04-28 Tyrenn Co., Ltd. Input device and UI configuration and execution method thereof
CN105138112A (en) * 2015-07-13 2015-12-09 腾讯科技（深圳）有限公司 Display control method and apparatus
KR101833131B1 (en) * 2016-06-08 2018-02-27 엘지이노텍 주식회사 Portable watch
CN107329574A (en) * 2017-06-30 2017-11-07 联想(北京)有限公司 Input method and system for electronic equipment
US11635736B2 (en) 2017-10-19 2023-04-25 Meta Platforms Technologies, Llc Systems and methods for identifying biological structures associated with neuromuscular source signals
KR20180018629A (en) * 2018-02-07 2018-02-21 엘지이노텍 주식회사 Watch
KR101976536B1 (en) * 2018-02-07 2019-05-09 엘지이노텍 주식회사 Watch
JP2019032891A (en) * 2018-10-29 2019-02-28 アップル インコーポレイテッドＡｐｐｌｅ Ｉｎｃ． Motion gesture input detected by optical sensor
US11797087B2 (en) 2018-11-27 2023-10-24 Meta Platforms Technologies, Llc Methods and apparatus for autocalibration of a wearable electrode sensor system
US11941176B1 (en) 2018-11-27 2024-03-26 Meta Platforms Technologies, Llc Methods and apparatus for autocalibration of a wearable electrode sensor system
US11961494B1 (en) 2019-03-29 2024-04-16 Meta Platforms Technologies, Llc Electromagnetic interference reduction in extended reality environments
US11907423B2 (en) 2019-11-25 2024-02-20 Meta Platforms Technologies, Llc Systems and methods for contextualized interactions with an environment
US11868531B1 (en) 2021-04-08 2024-01-09 Meta Platforms Technologies, Llc Wearable device providing for thumb-to-finger-based input gestures detected based on neuromuscular signals, and systems and methods of use thereof
CN115016635A (en) * 2022-04-18 2022-09-06 深圳原宇科技有限公司 Target control method and system based on motion recognition

Also Published As

Publication number Publication date
JP4379214B2 (en) 2009-12-09

Similar Documents

Publication Publication Date Title
JP4379214B2 (en) Mobile terminal device
JP6415592B2 (en) Wearable device
EP2573650A1 (en) Portable information processing terminal
EP1244003A2 (en) Information input system using bio feedback and method thereof
KR101549353B1 (en) smart watch with recognition function of bio sound source
US20170202484A1 (en) Wearable device and method
WO2011055326A1 (en) Universal input/output human user interface
US10614679B2 (en) Haptic presentation apparatus, recognition apparatus, control apparatus, and haptic presentation method
CN109303390B (en) Electronic string bead ring and counting method thereof
CN105814511B (en) Offset plane wrist input device, method and computer readable medium
JP6334601B2 (en) Portable information terminal, wearing arm judgment method, wearing direction judgment method, and program
KR100499391B1 (en) Virtual input device sensed finger motion and method thereof
KR101497829B1 (en) Watch type device utilizing motion input
JP2016177568A (en) Display device and information display metho, information display program
JP6270557B2 (en) Information input / output device and information input / output method
US20140022165A1 (en) Touchless text and graphic interface
KR20170091959A (en) Smart glove which is connected with smart device by bluetooth wireless communication
RU2651444C2 (en) Device and method for receiving and transmitting information by braille letters
KR20010110615A (en) Information input device operated by detecting movement of skin, mobile information processing device, computer and mobile communication device using the same
KR100838181B1 (en) Hand interface device using miniaturized absolute position sensors, system for hand interface using it
JP6170973B2 (en) Portable information terminal, information processing method, and program
Katti et al. Talking gloves: sign language to speech conversion for deaf and mute person
KR20180062593A (en) Apparatus for translating chirology in real time
TW201711010A (en) An electromyography sensor and inertia sensor-based posture recognition device for real-time sign language translation system
JP2024053348A (en) Input device and method

Legal Events

Date Code Title Description
RD03 Notification of appointment of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7423

Effective date: 20051121

RD04 Notification of resignation of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7424

Effective date: 20051121

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20070522

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20090501

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20090512

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20090626

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20090825

A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20090907

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20121002

Year of fee payment: 3

R150 Certificate of patent or registration of utility model

Free format text: JAPANESE INTERMEDIATE CODE: R150

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20131002

Year of fee payment: 4

S111 Request for change of ownership or part of ownership

Free format text: JAPANESE INTERMEDIATE CODE: R313113

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

LAPS Cancellation because of no payment of annual fees