JPH0695840A - Sound collecting device - Google Patents

Sound collecting device

Info

Publication number
JPH0695840A
JPH0695840A JP4240298A JP24029892A JPH0695840A JP H0695840 A JPH0695840 A JP H0695840A JP 4240298 A JP4240298 A JP 4240298A JP 24029892 A JP24029892 A JP 24029892A JP H0695840 A JPH0695840 A JP H0695840A
Authority
JP
Japan
Prior art keywords
sound
sound collection
video display
sound collecting
directivity
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
JP4240298A
Other languages
Japanese (ja)
Inventor
Tomohiko Arikawa
知彦 有川
Junji Kojima
順治 小島
Manabu Okamoto
学 岡本
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nippon Telegraph and Telephone Corp
Original Assignee
Nippon Telegraph and Telephone Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nippon Telegraph and Telephone Corp filed Critical Nippon Telegraph and Telephone Corp
Priority to JP4240298A priority Critical patent/JPH0695840A/en
Publication of JPH0695840A publication Critical patent/JPH0695840A/en
Pending legal-status Critical Current

Links

Landscapes

  • Circuit For Audible Band Transducer (AREA)
  • Details Of Audible-Bandwidth Transducers (AREA)
  • Obtaining Desirable Characteristics In Audible-Bandwidth Transducers (AREA)

Abstract

PURPOSE:To execute a sound collection with high sound collection directivity for considering a position relation of a video displaypart and a user by providing plural sound collecting parts, and allowing them to have the sound collection directivity in the center axis direction on a display screen of the video display part. CONSTITUTION:By containing electric signals from plural sound collecting parts 6 (6a, 6b) into a signal processing part 7, and executing a suitable signal processing, a sound signal having desired sound collection directivity is outputted to an output terminal 8. In this case, the collecting parts are allowed to have sound collection directivity in the center axis direction on a display screen of a video display part. Also, with respect to the display screen of the video display part, the sound collection directivity can be changed in an arbitrary direction by an instruction from the outside. Moreover, this device has a function for tracking a sound source being in front of the video display part, and the sound collection is executed by interlocking with sound collection directivity. In such a way, in the divice provided with the video display of a work station and a personal computer, etc., the sound collection can be executed with high sound collection directivity for considering a position relation of the video display part and a user.

Description

【発明の詳細な説明】Detailed Description of the Invention

【０００１】[0001]

【産業上の利用分野】本発明は、ワークステーション
（以下ＷＳと略す）やパーソナルコンピュータ（以下Ｐ
Ｃと略す）などの映像表示部を有する装置において用い
る集音装置に関するものである。BACKGROUND OF THE INVENTION The present invention relates to a workstation (hereinafter abbreviated as WS) and a personal computer (hereinafter P).
The present invention relates to a sound collecting device used in a device having an image display unit such as C).

【０００２】[0002]

【従来の技術】ＷＳやＰＣを用いた作業環境として、音
声の入出力を行なうアプリケーション例が増大している
（例えば、“Ｂ−ＩＳＤＮ用多地点間マルチメディア通
信会議システムＰＭＴＣ”、信学技法ＩＥ９０−４９，
ｐｐ．３１−３６など）。また、映像表示部の表示範囲
は、映像表示部の廉価化とヒューマンインタフェースの
高度化とにより、拡大する方向にあり、現状のＷＳ、Ｐ
Ｃでは１９インチ程度の大きさの映像表示部も広く用い
られている。2. Description of the Related Art As a work environment using a WS or a PC, there are an increasing number of application examples for inputting and outputting voice (for example, "B-ISDN multipoint multimedia communication conference system PMTC", a communication technique). IE90-49,
pp. 31-36). Further, the display range of the video display unit is in the direction of expanding due to the price reduction of the video display unit and the sophistication of the human interface.
In C, an image display unit having a size of about 19 inches is also widely used.

【０００３】大型の映像表示部を用いたＷＳやＰＣなど
における集音装置としては、指向性を持たせた集音装置
を机上や映像表示部の上または横に設置したものや、音
響結合を防ぐためにヘッドホンとあわせて設置された集
音装置を用いたヘッドセット型のものなどが用いられて
いる。図９は従来の装置における集音装置の設置例を示
す。図中の符号１は映像表示部、２はキーボード、３は
集音部、４は集音部、５は集音部を表わしている。As a sound collecting device in a WS or a PC using a large-sized image display unit, a sound collecting device having directivity is installed on a desk or on or side of the image display unit, or acoustic coupling is used. In order to prevent this, a headset type one using a sound collector installed together with headphones is used. FIG. 9 shows an installation example of a sound collecting device in a conventional device. In the figure, reference numeral 1 is a video display portion, 2 is a keyboard, 3 is a sound collecting portion, 4 is a sound collecting portion, and 5 is a sound collecting portion.

【０００４】[0004]

【発明が解決しようとする課題】ＷＳやＰＣ等における
集音装置は、最適集音ポイントである場所が映像表示部
であるため、集音装置を最適ポイントに設置できないと
いう第一の問題がある。The sound collecting device in WS, PC, etc. has the first problem that the sound collecting device cannot be installed at the optimum point because the place which is the optimum sound collecting point is the image display section. .

【０００５】また、集音装置の設置場所によっては集音
装置に近接して設置されるキーボードやマウスなどのデ
ータ入力・指示機器から発生するキータッチなどの雑音
が所望の集音音源位置（口の位置）に比べ近くなるため
非常に大きな妨害となるという第二の問題がある。Depending on the location where the sound collector is installed, noise such as key touch generated from a data input / instruction device such as a keyboard or a mouse installed near the sound collector is a desired sound source position (mouth). There is a second problem that it becomes a very big obstacle because it is closer than the position of.

【０００６】本発明は、映像表示部を有する装置におい
て、より高度な集音装置を提供することを目的としてい
る。An object of the present invention is to provide a more sophisticated sound collecting device in a device having a video display section.

【０００７】[0007]

【課題を解決するための手段】ＷＳやＰＣを用いた装置
での一般的な所望の音源位置はほぼ映像表示部の表示面
中心軸上にあり、その距離は、キーボードの位置が固定
されればヒューマンインタフェース設計上容易に得るこ
とができるが、実測の結果この距離は５０ｃｍ〜７０ｃ
ｍ程度でほぼ固定されている。A general desired sound source position in a device using WS or PC is almost on the central axis of the display surface of the image display section, and the distance is such that the position of the keyboard is fixed. This can be easily obtained in human interface design, but as a result of actual measurement, this distance is 50 cm to 70 c.
It is almost fixed at about m.

【０００８】この知見を用いて本発明では、ＷＳやＰＣ
で用いる集音装置において複数の集音部とこれらの信号
を処理する信号処理部とを用いることで、最適集音ポイ
ントである映像表示部表示面上に仮想的に集音部を設置
することを可能にし、集音特性を改善する。In the present invention, based on this knowledge, WS and PC
By installing a plurality of sound collecting units and a signal processing unit that processes these signals in the sound collecting device used in, the sound collecting unit is virtually installed on the display surface of the image display unit, which is the optimum sound collecting point. And improve the sound collection characteristics.

【０００９】[0009]

【作用】集音部の設置方法により、また信号処理技術に
より外部からの制御で任意の方向に集音指向性を持たせ
ることが可能となるため、音源追尾制御などと合わせて
より高度な集音が可能となる。[Function] Since the sound collection directivity can be given in an arbitrary direction by the control of the outside by the installation method of the sound collection unit and the signal processing technology, the sound collection tracking control can be performed in a more sophisticated manner. Sound is possible.

【００１０】また、ＷＳやＰＣに装備されるキーボード
やマウスといったデータ入力・指示部から放出される雑
音を、補助集音部などを設置し信号処理することにより
本装置の主出力信号から消去・軽減することが可能とな
る。Further, noise emitted from a data input / instruction unit such as a keyboard or a mouse equipped in WS or PC is eliminated from the main output signal of this device by installing an auxiliary sound collecting unit and performing signal processing. It is possible to reduce.

【００１１】[0011]

【実施例】図１は本発明の概略機能ブロック図を示し、
図２は信号処理部での処理例を示す。また、図３，図
４，図５は集音部の設置例を示す。1 shows a schematic functional block diagram of the present invention,
FIG. 2 shows an example of processing in the signal processing unit. Further, FIGS. 3, 4 and 5 show examples of installation of the sound collecting unit.

【００１２】複数の集音部６からの電気信号を信号処理
部７で収容し、適当な信号処理を行なうことにより、所
望の集音指向性を有した音声信号が出力端子８に出力さ
れる。By accommodating the electric signals from the plurality of sound collecting units 6 in the signal processing unit 7 and performing appropriate signal processing, a sound signal having a desired sound collecting directivity is output to the output terminal 8. .

【００１３】例えば、図２は信号処理部７の処理内容を
表わしているが、これは、図２記載の位置９（ａ），９
（ｂ）に無指向性の集音部を２台設置した時に得られる
それぞれの等音圧曲線１０ａ，１０ｂおよびそれらの単
純加算で得られた合成等音圧曲線１１である。これによ
ると、単に加算処理を行なうのみであっても２台の集音
部中心軸上に集音指向性が得られることが分かる。実際
は、単に電気信号の加算を行なったのみであっても集音
部本体の集音指向性や、音波の位相ずれによるキャンセ
ル効果を考慮する必要があり、さらにこの集音特性は鋭
敏になる。For example, FIG. 2 shows the processing contents of the signal processing unit 7, which is shown at positions 9 (a) and 9 in FIG.
FIG. 3B shows the respective equal sound pressure curves 10a and 10b obtained when two omnidirectional sound collecting units are installed, and the combined equal sound pressure curve 11 obtained by simple addition of these curves. According to this, it is understood that the sound collection directivity can be obtained on the central axes of the two sound collection units even if the addition process is simply performed. In fact, even if the electric signals are simply added, it is necessary to consider the sound collection directivity of the sound collection unit main body and the canceling effect due to the phase shift of the sound waves, and the sound collection characteristics become sharp.

【００１４】図３，図４は、２個の集音部１２ａ，１２
ｂ（１３ａ，１３ｂ）を垂直（水平）方向に映像表示部
全面に設置したもので、それぞれ集音部間を２等分する
位置（図中ハッチ部）で集音指向性が高くなる。図５
は、同様に集音部１４ａ，１４ｂ，１４ｃを３個設置し
たもので、図中のハッチ部分に対して集音指向性を高め
ることができる。FIGS. 3 and 4 show two sound collecting parts 12a and 12a.
b (13a, 13b) is installed vertically (horizontally) over the entire surface of the image display unit, and the sound collection directivity is increased at a position (hatched portion in the figure) that bisects the sound collection units. Figure 5
In the same manner, three sound collecting units 14a, 14b, 14c are similarly installed, and the sound collecting directivity can be enhanced with respect to the hatched portion in the drawing.

【００１５】これらの信号処理は、信号処理部７（図
１）で行なわれるが、ディジタル信号処理プロセッサ
（ＤＳＰ）などを用いることにより単に加算のみならず
信号の位相制御、遅延制御などを行なうことが可能でよ
り高度な集音指向性制御が可能となる。These signal processings are carried out by the signal processing unit 7 (FIG. 1). By using a digital signal processing processor (DSP) or the like, not only the addition but also the phase control and delay control of the signals are carried out. It is possible to achieve more advanced sound collection directivity control.

【００１６】図６は補助集音部を用いて、キーボード等
から放出される雑音を消去・軽減する機能をもつ場合の
概略ブロック図である。信号処理部７から出力された音
声信号８（端子８の信号）は、雑音処理部１６で、雑音
センス用補助集音部１５からのリファレンス信号を用い
て、減算などの適当な信号処理により、当該音声信号８
中の雑音の消去・軽減を行なうようにされる。従って雑
音処理部１６からの出力信号１７中にはキーボードなど
からの所望する音声以外の雑音を消去・軽減した信号が
出力される。これについてもＤＳＰなどを用いることに
より、雑音のスペクトル分析を行なった後、該当信号帯
域の信号帯域制限を行なうなど、より高度な雑音信号処
理が可能である。FIG. 6 is a schematic block diagram in the case where the auxiliary sound collecting unit is used to have a function of eliminating or reducing noise emitted from a keyboard or the like. The audio signal 8 (the signal of the terminal 8) output from the signal processing unit 7 is subjected to appropriate signal processing such as subtraction by the noise processing unit 16 using the reference signal from the noise sensing auxiliary sound collecting unit 15. The audio signal 8
It is designed to erase and reduce the noise inside. Therefore, in the output signal 17 from the noise processing unit 16, a signal from which noise other than the desired voice from the keyboard or the like is eliminated / reduced is output. Also in this case, by using a DSP or the like, it is possible to perform more advanced noise signal processing, such as performing signal spectrum analysis of noise and then limiting the signal band of the corresponding signal band.

【００１７】さらに、複数の集音部間の信号レベル差、
位相差を用いることにより雑音制御用の補助集音部を用
いずに動作させることも可能である。図７は雑音制御を
行なう場合の実施例を示す。符号１，２は図１に対応し
ている。集音部１８とは別に、雑音センス用補助集音部
１９を雑音源であるキーボードに設置したものである。
これらにより集音指向性を持たせると同時にかつ雑音消
去・軽減を同時に行なうことが可能となる。Further, a signal level difference between the plurality of sound collecting units,
By using the phase difference, it is possible to operate without using the auxiliary sound collecting unit for noise control. FIG. 7 shows an embodiment in which noise control is performed. Reference numerals 1 and 2 correspond to FIG. In addition to the sound collecting unit 18, a noise sensing auxiliary sound collecting unit 19 is installed on a keyboard which is a noise source.
As a result, it becomes possible to simultaneously provide sound collection directivity and simultaneously eliminate and reduce noise.

【００１８】図８は、映像表示部の直前にある音源を追
尾し、その結果を用いて集音指向性を制御し集音する場
合の概略ブロック図である。映像入力部２０は映像表示
部の直前で作業している人物像の撮像を行ないその映像
信号を音源追跡処理部２１に送出する。音源追跡処理部
２１は、映像信号中の人物像の認識を行ない、音源の座
標を信号処理部７に送出する。この場合、映像入力部２
０が直接人物を撮像する場合には、鏡像効果を考慮する
必要がある。信号処理部７はこの音源座標値２２を元に
集音部６からの信号に適当な処理を行い音源座標に適合
した集音指向性を実現し所望の音声信号８を出力する。FIG. 8 is a schematic block diagram in the case of tracking a sound source immediately in front of the video display unit and using the result to control the sound collecting directivity to collect sound. The video input unit 20 captures the image of the person who is working immediately before the video display unit, and sends the video signal to the sound source tracking processing unit 21. The sound source tracking processing unit 21 recognizes the person image in the video signal and sends the coordinates of the sound source to the signal processing unit 7. In this case, the video input unit 2
When 0 directly images a person, it is necessary to consider the mirror image effect. Based on the sound source coordinate value 22, the signal processing unit 7 performs appropriate processing on the signal from the sound collecting unit 6 to realize sound collecting directivity that matches the sound source coordinates and outputs a desired audio signal 8.

【００１９】[0019]

【発明の効果】以上説明したように、本発明によれば、
ＷＳやＰＣを用いた装置において、映像表示部と使用者
の位置関係とを考慮した集音指向性の良い集音が可能と
なるという利点がある。As described above, according to the present invention,
In a device using WS or PC, there is an advantage that it is possible to collect sound with good sound collection directivity in consideration of the positional relationship between the image display unit and the user.

【図面の簡単な説明】[Brief description of drawings]

【図１】本発明の概略機能ブロック構成図を示す。FIG. 1 is a schematic functional block configuration diagram of the present invention.

【図２】信号処理の例を示す。FIG. 2 shows an example of signal processing.

【図３】集音部設置例を示す。FIG. 3 shows an example of installing a sound collecting unit.

【図４】集音部設置例を示す。FIG. 4 shows an example of installation of a sound collecting unit.

【図５】集音部設置例を示す。FIG. 5 shows an example of installing a sound collecting unit.

【図６】雑音制御を行なった場合の概略機能ブロック構
成図を示す。FIG. 6 shows a schematic functional block configuration diagram when noise control is performed.

【図７】雑音制御を行なった場合の集音部設置例を示
す。FIG. 7 shows an example of installation of a sound collecting unit when noise control is performed.

【図８】音源追跡処理を行なった場合の概略機能ブロッ
ク構成図を示す。FIG. 8 is a schematic functional block configuration diagram when a sound source tracking process is performed.

【図９】従来の集音方法を説明する図である。FIG. 9 is a diagram illustrating a conventional sound collection method.

【符号の説明】[Explanation of symbols]

１ 映像表示部 ２ キーボード ３ 集音部（１） ４ 集音部（２） ５ 集音部（３） ６ 集音部 ７ 信号処理部 ８ 出力端子 ９ 集音部 １０ 等音圧曲線 １１ 合成等音圧曲線 １２ 集音部 １３ 集音部 １４ 集音部 １５ 雑音センス用補助集音部 １６ 雑音処理部 １７ 出力信号 １８ 集音部 １９ 雑音センス用補助集音部 ２０ 映像入力部 ２１ 音源追跡処理部 ２２ 音源座標値 1 Video Display Section 2 Keyboard 3 Sound Collection Section (1) 4 Sound Collection Section (2) 5 Sound Collection Section (3) 6 Sound Collection Section 7 Signal Processing Section 8 Output Terminal 9 Sound Collection Section 10 Equal Pressure Curve 11 Synthesis etc. Sound pressure curve 12 Sound collecting part 13 Sound collecting part 14 Sound collecting part 15 Noise sensing auxiliary sound collecting part 16 Noise processing part 17 Output signal 18 Sound collecting part 19 Noise sensing auxiliary sound collecting part 20 Video input part 21 Sound source tracking process Part 22 sound source coordinate values

Claims (4)

【特許請求の範囲】[Claims] 【請求項１】 映像表示部を有する装置において、 複数の集音部を有し、映像表示部表示画面の中心軸方向
に集音指向性を有することを特徴とする集音装置。1. A sound collecting device having a video display unit, comprising a plurality of sound collecting units, and having a sound collecting directivity in a central axis direction of a display screen of the video display unit. 【請求項２】 映像表示部を有する装置において、 複数の集音部と、信号処理部とを有し、映像表示部の表
示画面に対し、外部からの指示により任意の方向に集音
指向性を変更可能にしたことを特徴とする集音装置。2. A device having a video display unit, comprising a plurality of sound collection units and a signal processing unit, and directing sound collection in an arbitrary direction on a display screen of the video display unit according to an instruction from the outside. A sound collecting device characterized by making it possible to change. 【請求項３】 映像表示部を有する装置において、 複数の集音部からの音声信号または補助集音部からの音
声信号により、データ入力・指示部より発生する可聴範
囲の雑音を除去・軽減する事を特徴とする集音装置。3. In a device having a video display unit, noise in an audible range generated by a data input / instruction unit is removed or reduced by a sound signal from a plurality of sound collecting units or a sound signal from an auxiliary sound collecting unit. A sound collection device featuring things. 【請求項４】 映像表示部を有する装置において、 映像表示部の前にある音源を追尾する機能を有し、集音
指向性制御と連動させて集音することを特徴とする集音
装置。4. A sound collecting device having a video display unit, which has a function of tracking a sound source in front of the video display unit and collects sound by interlocking with sound collection directivity control.
JP4240298A 1992-09-09 1992-09-09 Sound collecting device Pending JPH0695840A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP4240298A JPH0695840A (en) 1992-09-09 1992-09-09 Sound collecting device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP4240298A JPH0695840A (en) 1992-09-09 1992-09-09 Sound collecting device

Publications (1)

Publication Number Publication Date
JPH0695840A true JPH0695840A (en) 1994-04-08

Family

ID=17057391

Family Applications (1)

Application Number Title Priority Date Filing Date
JP4240298A Pending JPH0695840A (en) 1992-09-09 1992-09-09 Sound collecting device

Country Status (1)

Country Link
JP (1) JPH0695840A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1294471C (en) * 2003-09-30 2007-01-10 株式会社东芝 Electronic apparatus capable of always executing noise canceling and voice input method for the apparatus
JPWO2007013129A1 (en) * 2005-07-25 2009-02-05 富士通株式会社 Sound receiver
JPWO2015049921A1 (en) * 2013-10-04 2017-03-09 日本電気株式会社 Signal processing apparatus, media apparatus, signal processing method, and signal processing program
JP2018014590A (en) * 2016-07-20 2018-01-25 株式会社オーディオテクニカ Microphone

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1294471C (en) * 2003-09-30 2007-01-10 株式会社东芝 Electronic apparatus capable of always executing noise canceling and voice input method for the apparatus
US8189818B2 (en) 2003-09-30 2012-05-29 Kabushiki Kaisha Toshiba Electronic apparatus capable of always executing proper noise canceling regardless of display screen state, and voice input method for the apparatus
JPWO2007013129A1 (en) * 2005-07-25 2009-02-05 富士通株式会社 Sound receiver
JP4769804B2 (en) * 2005-07-25 2011-09-07 富士通株式会社 Sound receiver
US8396242B2 (en) 2005-07-25 2013-03-12 Fujitsu Limited Sound receiver
JPWO2015049921A1 (en) * 2013-10-04 2017-03-09 日本電気株式会社 Signal processing apparatus, media apparatus, signal processing method, and signal processing program
JP2018014590A (en) * 2016-07-20 2018-01-25 株式会社オーディオテクニカ Microphone

Similar Documents

Publication Publication Date Title
US11531518B2 (en) System and method for differentially locating and modifying audio sources
US6694028B1 (en) Microphone array system
US9922663B2 (en) Voice signal processing method and apparatus
US6668062B1 (en) FFT-based technique for adaptive directionality of dual microphones
CN110970057B (en) Sound processing method, device and equipment
EP3304548B1 (en) Electronic device and method of audio processing thereof
JP2016146547A (en) Sound collection system and sound collection method
US10497356B2 (en) Directionality control system and sound output control method
CN1436436A (en) Method and apparatus for voice signal extraction
US20210158828A1 (en) Audio processing device, image processing device, microphone array system, and audio processing method
WO2014076707A2 (en) Method and system for generation of sound fields
US11638092B2 (en) Advanced speech encoding dual microphone configuration (DMC)
US11290839B2 (en) Headphone
CN110322892B (en) Voice pickup system and method based on microphone array
JP6447976B2 (en) Directivity control system and audio output control method
JPH0695840A (en) Sound collecting device
JPH09238394A (en) Directivity microphone equipment
JP2001045592A (en) Noise canceling microphone array
JP6569853B2 (en) Directivity control system and audio output control method
CN114422933A (en) Electronic equipment, sound field processing method thereof and sound field processing equipment
CN115665606B (en) Sound reception method and sound reception device based on four microphones
JP2016219965A (en) Directivity control system and speech output control method
JP7248048B2 (en) SOUND COLLECTION DEVICE, SOUND COLLECTION PROGRAM AND SOUND COLLECTION METHOD, AND KEYBOARD
KR100310283B1 (en) A method for enhancing 3-d localization of speech
WO2023286413A1 (en) Area reproduction system and area reproduction method

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20050218

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20070307

A131 Notification of reasons for refusal

Effective date: 20080415

Free format text: JAPANESE INTERMEDIATE CODE: A131

A521 Written amendment

Effective date: 20080616

Free format text: JAPANESE INTERMEDIATE CODE: A523

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Effective date: 20080708

Free format text: JAPANESE INTERMEDIATE CODE: A01

A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20080709

FPAY Renewal fee payment (prs date is renewal date of database)

Free format text: PAYMENT UNTIL: 20110718

Year of fee payment: 3

R150 Certificate of patent (=grant) or registration of utility model

Free format text: JAPANESE INTERMEDIATE CODE: R150

FPAY Renewal fee payment (prs date is renewal date of database)

Year of fee payment: 3

Free format text: PAYMENT UNTIL: 20110718

FPAY Renewal fee payment (prs date is renewal date of database)

Year of fee payment: 4

Free format text: PAYMENT UNTIL: 20120718

FPAY Renewal fee payment (prs date is renewal date of database)

Free format text: PAYMENT UNTIL: 20130718

Year of fee payment: 5

FPAY Renewal fee payment (prs date is renewal date of database)

Free format text: PAYMENT UNTIL: 20140718

Year of fee payment: 6

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250