JPH08237637A - Color detector and warning device - Google Patents
Color detector and warning deviceInfo
- Publication number
- JPH08237637A JPH08237637A JP6189095A JP6189095A JPH08237637A JP H08237637 A JPH08237637 A JP H08237637A JP 6189095 A JP6189095 A JP 6189095A JP 6189095 A JP6189095 A JP 6189095A JP H08237637 A JPH08237637 A JP H08237637A
- Authority
- JP
- Japan
- Prior art keywords
- signal
- image
- color detection
- distance
- image pickup
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Closed-Circuit Television Systems (AREA)
- Processing Of Color Television Signals (AREA)
- Emergency Alarm Devices (AREA)
Abstract
Description
【0001】[0001]
【産業上の利用分野】本発明は映像信号の色検出装置に
関し、特に、検出対象物の色検出に加え、その対象物の
距離判定測定機能を有する色検出装置及びその色検出装
置を用いた警告装置に関する。BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a color detection device for a video signal, and more particularly to a color detection device having a function of determining the distance of an object to be detected in addition to the color detection of the object and the color detection device using the same. Warning device.
【0002】[0002]
【従来の技術】建設現場等のように大型機械等を用いる
職場において、作業者等の着用するヘルメットや着衣の
色を弁別して危険区域にいる作業者等に警告を行うと共
にオペレータに対して危険区域内における人の存在を告
知するためにカラーテレビカメラで当該区域を監視する
等の安全対策として、 (イ)建設機械から超音波を発射して反射物体からの反
射波を検出して、反射物体からの距離の測定を行い、
発射物体が作業者か否かを何らかの方法で弁別する方
式 (ロ)特開平6−32012号、特願平5−12541
7号に開示の色検出装置(映像信号の色成分抽出装置及
び警告装置)のように危険物の近くにいる建設作業者の
用いているヘルメットの色や着衣の色(建設機械の稼働
現場では作業者同士とそれ以外の者を確認(区別)する
必要上、注意色として黄色が選ばれる)に注目し、カラ
ーテレビカメラをセンサとし、その映像信号から特定色
(例えば黄色)を検出する色検出方式がある。2. Description of the Related Art In a workplace such as a construction site where a large machine or the like is used, the colors of helmets and clothes worn by workers are discriminated from each other to warn workers in a dangerous area and to alert operators. As a safety measure such as monitoring the area with a color TV camera to notify the presence of people in the area, (a) The construction machine emits ultrasonic waves to detect the reflected wave from the reflective object, Measure the distance from the object,
A method for discriminating whether or not the projectile is an operator by some method (b) Japanese Patent Application Laid-Open No. 6-32012 and Japanese Patent Application No. 5-12541
The color of helmets and clothing used by construction workers who are near dangerous materials such as the color detection device (color component extraction device and warning device of the video signal) disclosed in No. 7 (in construction machinery operating sites Focusing on the fact that yellow is selected as a caution color because it is necessary to identify (distinguish) workers from each other, and a color TV camera is used as a sensor, and a color that detects a specific color (for example, yellow) from the video signal. There is a detection method.
【0003】図9は上述した色検出装置の一例を示すも
のであり、特願平5−125417号(映像信号の色成
分抽出装置及び警告装置の第6の発明)の色成分抽出回
路の一実施例の構成を示すブロック図であり、以下の説
明は特定色を黄色とした場合についてのものである。図
9で、NTSC信号入力41はカラーテレビカメラの出
力信号であり、AGC42は映像信号のレベル(白ピー
ク値)を一定に保つ。YC分離回路43は輝度信号とカ
ラー搬送波信号の分離回路であり簡単なものとしては分
波器があげられる。分波器を用いる場合、輝度信号には
3MHzのLPF(ローパスフィルタ)出力を、色信号
には3.6MHzを中心周波数とするBPFを用いる。
そして、通常の復調器と同様に、LPF出力を同期分離
回路45を経てバーストゲートパルス発生器46に加
え、BPF出力をバーストゲート44に加えてバースト
ゲートパルス発生器46の出力パルスでゲートし、バー
ストゲート44の出力で副搬送波cosωsctを発振
する副搬送波発振器47を駆動する。FIG. 9 shows an example of the above-mentioned color detecting device, which is an example of a color component extracting circuit of Japanese Patent Application No. 5-125417 (sixth invention of a color component extracting device of a video signal and a warning device). FIG. 3 is a block diagram showing the configuration of the embodiment, and the following description is for the case where the specific color is yellow. In FIG. 9, the NTSC signal input 41 is the output signal of the color television camera, and the AGC 42 keeps the level (white peak value) of the video signal constant. The YC separation circuit 43 is a separation circuit for the luminance signal and the color carrier signal, and a simple demultiplexer can be used. When a duplexer is used, a 3 MHz LPF (low-pass filter) output is used for the luminance signal and a BPF having a center frequency of 3.6 MHz is used for the color signal.
Then, similarly to a normal demodulator, the LPF output is applied to the burst gate pulse generator 46 via the sync separation circuit 45, the BPF output is applied to the burst gate 44 and gated by the output pulse of the burst gate pulse generator 46, The output of the burst gate 44 drives the subcarrier oscillator 47 that oscillates the subcarrier cosωsct.
【0004】位相調整回路48は副搬送波発振器47か
らの副搬送波cosωsctの位相を調整し、cos
(ωsct+φyellow)とする。更に、90°移相回路
49でsin(ωsct+φyellow)を作る。YC分離
回路43のBPF出力を更に同期検波回路50,51に
加え、同期検波回路50では位相調整回路48の出力
で、同期検波回路51では90°移相回路49の出力で
同期検波してEyellowと^Eyellowを得る。そして、同
期検波回路51の出力を利得kの利得制御回路62に加
えて数式(9)のkを制御し、利得制御回路22の出力
を絶対値回路52に加えて正極性のみとし|^Eyellow
|を得て、引算回路53で同期検波回路50の出力のE
yellowから引算する。引算回路53の出力は振幅比較回
路54に加えて基準電圧55と比較して基準値以上を出
力する。1ビット化回路60は振幅比較回路54の出力
をビット化するものであり、副搬送波信号から黄と判定
する領域を1、判定しない領域を0、とするように2値
で表すものである。The phase adjustment circuit 48 adjusts the phase of the subcarrier cos ωsct from the subcarrier oscillator 47,
(Ωsct + φ yellow ). Further, sin (ωsct + φ yellow ) is created by the 90 ° phase shift circuit 49. The BPF output of the YC separation circuit 43 is further added to the synchronous detection circuits 50 and 51, and the synchronous detection circuit 50 outputs the output of the phase adjustment circuit 48, and the synchronous detection circuit 51 performs synchronous detection with the output of the 90 ° phase shift circuit 49. Get yellow and ^ E yellow . Then, the output of the synchronous detection circuit 51 is added to the gain control circuit 62 of the gain k to control k in the equation (9), and the output of the gain control circuit 22 is added to the absolute value circuit 52 so that only the positive polarity is obtained. yellow
, And the subtraction circuit 53 outputs E of the output of the synchronous detection circuit 50.
Subtract from yellow . The output of the subtraction circuit 53 is compared with the reference voltage 55 in addition to the amplitude comparison circuit 54 and is output as a reference value or more. The 1-bit conversion circuit 60 is for converting the output of the amplitude comparison circuit 54 into bits, and is represented by a binary value such that the area determined to be yellow from the subcarrier signal is 1, and the area not determined is 0.
【0005】一方、YC分離回路43の出力のEY(輝
度信号)は振幅比較回路56で基準電圧57と比較して
一定値以上を取り出す。これは黄色は人工色の中では他
の色に比べて比較的輝度信号が大きいからである。振幅
比較回路56の出力も1ビット化回路61でビット化
し、遅延回路63で所定時間遅延された1ビット化回路
60からの出力と共にAND回路18に加える。AND
回路18では色度的黄色領域である1ビット化回路60
の出力と、輝度的黄色領域である1ビット回路21の出
力とから黄色の範囲を決めて出力信号59として出力す
る。遅延回路23は輝度信号と色度信号の遅延時間を調
整するために用いる。On the other hand, the EY (luminance signal) output from the YC separation circuit 43 is compared with the reference voltage 57 by the amplitude comparison circuit 56 to extract a certain value or more. This is because yellow has a relatively large luminance signal in artificial colors as compared with other colors. The output of the amplitude comparison circuit 56 is also converted into bits by the 1-bit conversion circuit 61 and added to the AND circuit 18 together with the output from the 1-bit conversion circuit 60 delayed by the delay circuit 63 for a predetermined time. AND
In the circuit 18, the 1-bit conversion circuit 60 which is a chromatic yellow region
And the output of the 1-bit circuit 21, which is a luminance yellow area, determine the yellow range and output it as the output signal 59. The delay circuit 23 is used to adjust the delay time of the luminance signal and the chromaticity signal.
【0006】なお、図9の構成によれば色度的には、 (a)本例では黄色のみを例としているが、位相回路4
8により任意の色が選択できる。 (b)利得制御回路62の利得kを制限して色相の範囲
を調整できる。 (c)基準電圧55により彩度の下限を設定できる。ま
た、振幅比較回路56による輝度の選択は色により条件
が異なる。図9の回路によれば映像信号(NTSC信
号)から画面のなかで所望の色相(例えば、黄)を必要
な許容範囲で検出することができる。With respect to the chromaticity according to the configuration of FIG. 9, (a) in this example, only yellow is taken as an example, but the phase circuit 4
Any color can be selected with 8. (B) The hue range can be adjusted by limiting the gain k of the gain control circuit 62. (C) The lower limit of saturation can be set by the reference voltage 55. In addition, the conditions for the brightness selection by the amplitude comparison circuit 56 differ depending on the color. According to the circuit of FIG. 9, it is possible to detect a desired hue (for example, yellow) in the screen from the video signal (NTSC signal) within a necessary allowable range.
【0007】[0007]
【発明が解決しようとする課題】しかしながら、上記
(a)の技術では、通常、作業者の周囲には機械等色
々な形状の物体があるので作業者からの反射波の強度が
最高強度になるとは限らず、従って、精度の高い距離測
定が行えるか否かがそのときの環境条件によって異なる
というデリケートさがあり、また、作業者か他の物体
かの判別が実用条件下では困難である、という問題点が
あった。However, in the above technique (a), there are usually objects having various shapes such as machines around the worker, and therefore the intensity of the reflected wave from the worker becomes maximum. Therefore, there is a delicate condition that whether or not accurate distance measurement can be performed depends on the environmental conditions at that time, and it is difficult to distinguish between an operator and another object under practical conditions. There was a problem.
【0008】また、(b)の技術では、色検出方式単独
では特定色(例えば、黄色のヘルメットや黄色の着衣)
は検出できるが距離の測定が困難であり、例えば、黄色
のヘルメットのみに限定すれば検出した黄色の物体のサ
イズから距離の判定は可能であるが、面積の大きい着衣
が遠方にある場合とヘルメットが近くにある場合の区別
は困難である。従って、建設現場の警報装置と組み合わ
せて使用する場合、建設機械から十分に離れた安全な場
所にいる作業者についてもその黄色のヘルメットや黄色
の着衣の色(黄色)を検出して警報を発してしまう場合
があるという不都合があった。Further, in the technique of (b), the color detection method alone is used for a specific color (for example, a yellow helmet or a yellow clothing).
Can be detected, but it is difficult to measure the distance.For example, if it is limited to only a yellow helmet, it is possible to judge the distance from the size of the detected yellow object, but if the clothing with a large area is in the distance and the helmet. It is difficult to distinguish when there are nearby. Therefore, when used in combination with an alarm device at a construction site, even a worker in a safe place far away from construction machinery can detect the yellow helmet or yellow clothing color (yellow) and issue an alarm. There was an inconvenience that it might happen.
【0009】本発明は上記従来技術の問題点や不都合を
解消するためになされたものであり、撮像しているシー
ン中の特定の色の検出及び当該物体までのカメラからの
距離の検出を同時に行いえる色検出装置の提供を目的と
する。応用目的としては、例えば、建設現場で建設機械
に組込み、安全対策として作業者が着用しているヘルメ
ットの黄色を検出すると共に作業者までの距離を検出し
危険レベル(距離)に応じて警報を発する安全監視装置
等の提供が挙げられる。The present invention has been made in order to solve the problems and inconveniences of the above-mentioned prior art, and simultaneously detects the specific color in the scene being imaged and the distance from the camera to the object. An object is to provide a color detection device that can be used. As an application purpose, for example, it is built into a construction machine at a construction site, and as a safety measure, the yellow color of the helmet worn by the worker is detected, the distance to the worker is detected, and an alarm is issued according to the danger level (distance). Providing safety monitoring equipment to issue.
【0010】[0010]
【課題を解決するための手段】上記の目的を達成するた
めに第1の発明の色検出装置は、2系統の撮像信号を得
る撮像部と、撮像部からの撮像信号を入力し特定色の信
号を検出する色検出部と、撮像部からの撮像信号および
色検出部からの出力信号に基づく特定色検出像の2系統
の信号の水平方向を時間的に一致させて重複した部分を
求めると共に、垂直方向の大きさを求めることにより特
定距離での特定色検出像の面積を得る面積算出部と、任
意の距離での1系統分の特定色検出像信号の立上りと他
の系統の特定色検出像信号の立上りの差信号成分を面積
算出部からの出力に加え、その信号レベルを閾値と比較
して上記特定色検出像の遠近判定を行う距離判定部と、
を有していることを特徴とする。In order to achieve the above-mentioned object, a color detecting apparatus according to the first aspect of the present invention provides an image pickup section for obtaining image pickup signals of two systems, and an image pickup signal from the image pickup section to input a specific color. A color detection unit that detects a signal and a signal of two systems of a specific color detection image based on an image pickup signal from the image pickup unit and an output signal from the color detection unit are temporally matched in the horizontal direction to obtain an overlapping portion. , An area calculator that obtains the area of the specific color detection image at a specific distance by obtaining the size in the vertical direction, the rise of a specific color detection image signal for one system at an arbitrary distance, and the specific color of another system A rising difference signal component of the detected image signal is added to the output from the area calculation unit, and a distance determination unit that performs distance determination of the specific color detection image by comparing the signal level with a threshold value,
It is characterized by having.
【0011】第2の発明の警告装置は上記第1の発明の
色検出装置からの出力信号のレベルによって報知状態が
制御される報知手段を有することを特徴とする。A warning device according to a second aspect of the present invention is characterized in that it has a notifying means for controlling the notification state according to the level of the output signal from the color detecting device according to the first invention.
【0012】[0012]
【作用】上記構成により第1の発明の色検出装置は、色
検出部で検出される検出信号から特定色検出像の面積
(大きさ)を面積算出部で求め、距離判定部で面積と距
離の関係から特定色検出像と撮像位置との距離の遠近関
係を判定する。With the above structure, in the color detecting apparatus of the first invention, the area calculating unit obtains the area (size) of the specific color detection image from the detection signal detected by the color detecting unit, and the distance determining unit determines the area and the distance. Based on the relationship of, the perspective relationship of the distance between the specific color detection image and the imaging position is determined.
【0013】第2の発明の警告装置は上記第1の発明の
色検出装置からの出力信号のレベルによって距離に応じ
て報知を行うことができる。この場合、安全位置(例え
ば、建設機械の安全圏)にいる者に対しては仮に特定色
が検出されても警告音等を発しないようにできる。The warning device according to the second aspect of the invention can give a notification according to the distance by the level of the output signal from the color detection device according to the first aspect of the invention. In this case, a warning sound or the like can be prevented from being emitted to a person in a safe position (for example, the safety zone of the construction machine) even if the specific color is detected.
【0014】[0014]
【実施例】図1は本発明に基づく色検出装置の構成を示
す基本ブロック図であり、色検出装置100は、 2系統の撮像信号を得る撮像部A、 撮像部Aからの撮像信号を入力し撮像対象について
特定色の有無を検出する色検出部B、 撮像部Aからの撮像信号および色検出部Bからの出
力信号に基づいて特定色検出像について遠方(例えば、
安全距離D0を設定)での像を仮定しそのうち先行した
1系統分の像の信号(例えば、左眼像信号)を遅延させ
て他の1系統分の像信号(例えば、右眼像信号)の水平
方向を時間的に一致させ重複した部分を求め、また、垂
直方向の高さを求めることにより特定距離(遠方距離D
0)位置での特定色検出像の面積を得る面積算出部C、 任意の距離での特定色検出像信号の立上りの左眼像
信号と右眼像信号の差信号成分を面積算出部Cの出力に
加算してその信号レベルを閾値と比較して特定色検出像
とカメラとの遠近判定を行う距離判定部D、とを有して
いる。1 is a basic block diagram showing the structure of a color detecting apparatus according to the present invention. A color detecting apparatus 100 receives an image pickup signal from the image pickup section A which obtains image pickup signals of two systems. The color detection unit B that detects the presence or absence of a specific color in the imaging target, and the distant position of the specific color detection image based on the imaging signal from the imaging unit A and the output signal from the color detection unit B (for example,
Assuming an image at a safety distance D0), the image signal of the preceding one system (for example, the left-eye image signal) is delayed and the image signal of another one system (for example, the right-eye image signal) is delayed. Of the specified distance (far distance D)
0) An area calculation unit C that obtains the area of the specific color detection image at the position, and a difference signal component of the rising left eye image signal and the right eye image signal of the specific color detection image signal at an arbitrary distance from the area calculation unit C. It has a distance determination unit D that adds the output to the output and compares the signal level with a threshold value to determine the distance between the specific color detection image and the camera.
【0015】なお、撮像部Aは後述(図2参照)するよ
うに立体撮像用の2系統の光学系と1つの撮像素子(例
えば、CCD(Charge Coupled Device))を有するテ
レビカメラ、または2系統の撮像素子を撮像部として構
成することができる。また、距離判定部Dによる距離の
精度は高精度を要せず、例えば、遠い、近い、その中間
を判別し得る程度の精度でもよい。また、距離判定部D
の出力信号(判別信号)を警報装置に入力することによ
り、例えば、建設機械に組み込まれたTVカメラと作業
者との距離が遠い(安全)、近い(危険)、その中間
(要注意)かにより無音、注意音、警告音等を発するよ
う構成することもできる。The image pickup unit A is a television camera having two optical systems for stereoscopic image pickup and one image pickup device (for example, CCD (Charge Coupled Device)) or two lines as described later (see FIG. 2). The image pickup device can be configured as an image pickup unit. Further, the accuracy of the distance determined by the distance determination unit D does not need to be high, and may be, for example, accuracy that allows determination of a distance, a distance, or an intermediate distance. In addition, the distance determination unit D
By inputting the output signal (discrimination signal) of the above into the alarm device, for example, the distance between the TV camera incorporated in the construction machine and the worker is far (safe), close (danger), or in between (dangerous). Can be configured to emit a silent sound, a caution sound, a warning sound, or the like.
【0016】図2は上述した撮像部Aの一実施例の構成
を示す俯瞰図であり、立体撮像用の2系統の光学系を有
する撮像部の例である。図2で光路Lは反射鏡1,3を
経てレンズ4でCCD5に結像し、同様に光路Rは反射
鏡2,3を経てレンズ4でCCD5に結像する。CCD
5からは出力信号(撮像信号)6が出力される。FIG. 2 is a bird's-eye view showing the configuration of an embodiment of the above-mentioned image pickup section A, which is an example of an image pickup section having two optical systems for stereoscopic image pickup. In FIG. 2, the optical path L is imaged on the CCD 5 by the lens 4 via the reflecting mirrors 1 and 3, and similarly, the optical path R is imaged on the CCD 5 by the lens 4 via the reflecting mirrors 2 and 3. CCD
An output signal (imaging signal) 6 is output from 5.
【0017】図3はCCD5の出力信号6の説明図であ
り、検出すべき被写体を方形とした場合の色検出回路7
からの出力L0,R0をディスプレイ表示して左眼像、
右眼像として画面に示したものである(画面の左半面に
は信号L0による左眼像が、右半面には信号R0による
右眼像が示されている)。FIG. 3 is an explanatory diagram of the output signal 6 of the CCD 5, and the color detection circuit 7 when the object to be detected is a square.
Output L0, R0 from the display to the left eye image,
The right-eye image is shown on the screen (the left half of the screen shows the left-eye image by the signal L0, and the right half-face shows the right-eye image by the signal R0).
【0018】図4は被写体と撮像素子との関係を示す説
明図であり、上述した方形の被写体がCCD位置からD
0,D2,D3の距離にある場合に撮像して遠方距離D
0でR(右眼像)とL(左眼像)の像を表示系で重ねた
場合(後述)のD1の位置の像をL1,R1に、D2の
位置の像をL2,R2に示した図である。FIG. 4 is an explanatory view showing the relationship between the object and the image pickup device.
When the distance is 0, D2, D3, the image is taken and the far distance D
When R (right eye image) and L (left eye image) are superposed on the display system at 0 (as described later), the image at the position D1 is shown as L1 and R1, and the image at the position D2 is shown as L2 and R2. It is a figure.
【0019】図5は図1の色検出部B、面積算出部C、
および距離判定部Dの構成例を示すブロック図である。
なお、色検出部Bは図5では色検出回路7として示され
ており、色検出回路7としては図9に示した特願平5−
125417号に開示の回路の他、例えば、特開平6−
327012号、特願平6−303066号に開示の色
検出装置等を用いることができる。FIG. 5 shows a color detecting section B, an area calculating section C, and
3 is a block diagram showing a configuration example of a distance determination unit D. FIG.
The color detection section B is shown as a color detection circuit 7 in FIG. 5, and the color detection circuit 7 is shown in FIG.
In addition to the circuit disclosed in Japanese Patent No. 125417, for example, Japanese Patent Laid-Open No. 6-
The color detection device disclosed in Japanese Patent No. 327012 and Japanese Patent Application No. 6-303066 can be used.
【0020】また、面積算出部Cは図5では可変遅延線
8、差動増幅器9、H/2ゲートパルス発生器10、ゲ
ート回路11、及びリミッター12と垂直エッジ検出回
路13、マルチバイブレータ(MV)14,15、ゲー
トパルス発生器16、及びゲート回路17により構成さ
れ、距離判定部Dは微分回路18,19、正パルス取り
出し回路20,22、負パルス取り出し回路21,2
3、フリップフロップ24、ゲート回路25、ローパス
フィルタ(LPF)26、及びレベル比較回路27から
構成されている。Further, the area calculation unit C in FIG. 5 includes a variable delay line 8, a differential amplifier 9, an H / 2 gate pulse generator 10, a gate circuit 11, a limiter 12, a vertical edge detection circuit 13, and a multivibrator (MV). ) 14, 15, a gate pulse generator 16, and a gate circuit 17, the distance determination unit D includes a differentiating circuit 18, 19, a positive pulse extracting circuit 20, 22, a negative pulse extracting circuit 21, 2.
3, a flip-flop 24, a gate circuit 25, a low-pass filter (LPF) 26, and a level comparison circuit 27.
【0021】図5で、CCD5からの撮像出力6は色検
出回路7により特定色(例えば、黄色)が検出色信号
(パルス)として出力され可変遅延線8、差動増幅器
9、垂直エッジ検出回路13、及び微分回路19に入力
される。In FIG. 5, an image pickup output 6 from the CCD 5 has a specific color (for example, yellow) output as a detection color signal (pulse) by a color detection circuit 7, and a variable delay line 8, a differential amplifier 9 and a vertical edge detection circuit. 13 and the differentiating circuit 19.
【0022】可変遅延線8の遅延時間はH/2(Hは1
ライン長を意味する)±τであり、可変遅延線8の入力
信号は図6に示すようにL,Rともに経時的で各々約H
/2を占める。可変遅延線8の出力信号をL’,R’と
すると可変遅延線8の入力のRの位置と可変遅延線8の
出力L’の位置(水平方向を合わせるものとする。但
し、この場合、被写体の位置はカメラ(CCD)からの
遠方位置(図4のD0の位置)とする。可変遅延線8の
出力を差動増幅器9、H/2ゲートパルス発生器10及
び微分回路18に与える。The delay time of the variable delay line 8 is H / 2 (H is 1
(Meaning the line length) ± τ, and the input signal of the variable delay line 8 is about H for both L and R over time as shown in FIG.
Occupy / 2. When the output signals of the variable delay line 8 are L ′ and R ′, the position of the input R of the variable delay line 8 and the position of the output L ′ of the variable delay line 8 (horizontal directions are aligned. However, in this case, The position of the object is a position far from the camera (CCD) (position D0 in FIG. 4) The output of the variable delay line 8 is given to the differential amplifier 9, the H / 2 gate pulse generator 10 and the differentiating circuit 18.
【0023】差動増幅器9はCCD5の出力信号6及び
可変遅延線8からの出力を得てその出力をゲート回路1
1に通す。ゲート回路11はH/2ゲートパルス発生器
10からのH/2幅のパルスで可変遅延線8からの出力
をゲートし、図6の距離D0上で示されるR及びL’の
重なった部分のみを取り出す。従って、ゲート回路11
の出力を取り出してディスプレイ上に再現すれば遠方距
離D0でのRとL’の位置は可変遅延線の調整により容
易に一致させることができる。The differential amplifier 9 obtains the output signal 6 of the CCD 5 and the output from the variable delay line 8 and outputs the output to the gate circuit 1.
Pass through 1. The gate circuit 11 gates the output from the variable delay line 8 with the H / 2 width pulse from the H / 2 gate pulse generator 10, and only the overlapping portion of R and L ′ shown on the distance D0 in FIG. Take out. Therefore, the gate circuit 11
By taking out the output of and reproducing it on the display, the positions of R and L'at the far distance D0 can be easily matched by adjusting the variable delay line.
【0024】リミッター12はゲート回路11の出力を
一定値に限定するために用いているものであり、色検出
回路7の出力が一定に保持される場合や、或いは差動増
幅器9の動作が十分であれば不要である。The limiter 12 is used to limit the output of the gate circuit 11 to a constant value, and when the output of the color detection circuit 7 is held constant, or the operation of the differential amplifier 9 is sufficient. If so, it is unnecessary.
【0025】垂直エッジ検出回路13〜ゲートパルス発
生器16は垂直方向のゲート信号を得るための構成であ
り、図3の検出パターンの上端NHから△本(水平走査
線Hの増分を意味する)経たところからhの高さだけの
部分をゲートするパルスを発生させる。垂直エッジ検出
回路13は可変遅延線8の入力信号(すなわち、色検出
回路7の出力信号)から垂直エッジを検出し(上端NH
は被写体の位置により変化する)、マルチバイブレータ
14で△本分の幅パルスを発生させ、マルチバイブレー
タ15では(△+h)本分の幅パルスを発生させ、ゲー
トパルス発生器16を通してゲートパルスを発生させゲ
ート回路17に与える。ゲート回路17ではリミッター
12(或いはゲート回路11)の出力をゲートパルス発
生器16の出力でゲートすることでL0,R0(図3参
照)に対してh本の高さに相当する信号を出力する。な
お、ここで、CCD上の信号をD0に投影したものが図
4に示した位置D0上のサイズとなる。The vertical edge detection circuit 13 to the gate pulse generator 16 are arranged to obtain a gate signal in the vertical direction, and Δ lines (meaning an increment of the horizontal scanning line H) from the upper end NH of the detection pattern of FIG. A pulse is generated that gates only a portion of height h from the passing point. The vertical edge detection circuit 13 detects a vertical edge from the input signal of the variable delay line 8 (that is, the output signal of the color detection circuit 7) (upper edge NH
Varies depending on the position of the subject), the multi-vibrator 14 generates Δ width pulses, the multi-vibrator 15 generates (Δ + h) width pulses, and the gate pulse generator 16 generates gate pulses. And supplies it to the gate circuit 17. In the gate circuit 17, the output of the limiter 12 (or the gate circuit 11) is gated by the output of the gate pulse generator 16 to output a signal corresponding to the height of h lines to L0 and R0 (see FIG. 3). . The size of the signal on the CCD projected onto D0 is the size on the position D0 shown in FIG.
【0026】図7は図5の差動増幅器9の入力信号を画
像として再現した像と入出力信号の分布を示す図であ
る。図7(a)は差動増幅器9の入力信号を視覚的に再
現した像であり、図4においてD=D0で図5で可変遅
延線8の長さを調節して図6で左眼像L0’と右眼像0
とを一致させたものとし、D=D1の像をR1,L1
で、D=D2の像をR2,L2で示す。(b)はD=D
1での差動増幅器9の入力信号の分布図であり(c)は
入力信号の分布図である。また、(d)はD=D2での
差動増幅器9の入力信号の分布図であり(e)は入力信
号の分布図である。なお、(a)〜(e)を通じてR像
(R信号)は実線で示しL像(L信号)は破線で示して
ある。FIG. 7 is a diagram showing an image obtained by reproducing the input signal of the differential amplifier 9 shown in FIG. 5 as an image and the distribution of the input and output signals. FIG. 7A is an image in which the input signal of the differential amplifier 9 is visually reproduced. In FIG. 4, when D = D0, the length of the variable delay line 8 is adjusted in FIG. L0 'and right eye image 0
And the image of D = D1 is R1, L1.
Then, the image of D = D2 is shown by R2 and L2. (B) is D = D
FIG. 3C is a distribution diagram of the input signal of the differential amplifier 9 in FIG. Further, (d) is a distribution diagram of the input signal of the differential amplifier 9 when D = D2, and (e) is a distribution diagram of the input signal. In addition, through (a) to (e), the R image (R signal) is shown by a solid line, and the L image (L signal) is shown by a broken line.
【0027】図8は距離Dと差動増幅器9の出力εとの
関係を示す図であり、図7の(c)及び(e)を斜線で
示した面積(ε1,ε2)をεとして横軸に示し、縦軸
に距離Dを示したものである。これにより図7の破線で
囲まれた面積を測定することにより図4での被写体とカ
メラ(CCD)との距離を算出することができる。な
お、図3で被写体を方形として示したが、図5の垂直エ
ッジ検出回路13〜ゲートパルス発生器16が図3の画
像の一定の高さh本のみを出力しているので被写体との
距離の算出は被写体の形状に依存することなくできるこ
とが明らかである(但し、距離Dが著しく大きくなり、
検出色の像の高さが著しく小さくなれば検出は不可能と
なる)。FIG. 8 is a diagram showing the relationship between the distance D and the output ε of the differential amplifier 9, where (c) and (e) of FIG. It is shown on the axis and the distance D is shown on the vertical axis. Thus, the distance between the subject and the camera (CCD) in FIG. 4 can be calculated by measuring the area surrounded by the broken line in FIG. Note that the subject is shown as a square in FIG. 3, but since the vertical edge detection circuit 13 to the gate pulse generator 16 in FIG. 5 output only the constant height h lines of the image in FIG. It is clear that can be calculated without depending on the shape of the subject (however, the distance D becomes significantly large,
If the height of the detected color image becomes extremely small, detection becomes impossible).
【0028】図5で微分回路18以降の系(すなわち、
距離判定部D)は図7のε1,ε2などを抽出する。差
動増幅器9の2つの入力(色検出回路7の出力及び可変
遅延線8の出力)のうち可変遅延線8の出力を微分回路
18で、色検出回路7の出力を微分回路19で微分して
検出色信号の立上りを正パルス取り出し回路20,22
で取り出し、立下りを負パルス取り出し回路21,23
で取り出してフリップフロップ24の入力としフリップ
フロップ24を動作させる。この場合、仮に正パルス取
り出し回路20出力の正パルスの後に正パルス取り出し
回路22出力のパルスがあっても、負パルス取り出し回
路21出力の負パルスの後に負パルス取り出し回路23
出力の負パルスであっても、フリップフロップ24の動
作は変わらない(正パルス取り出し回路22、負パルス
取り出し回路23が先行する場合も同様である)。In FIG. 5, the system after the differentiation circuit 18 (that is,
The distance determination unit D) extracts ε1, ε2, etc. in FIG. Of the two inputs of the differential amplifier 9 (the output of the color detection circuit 7 and the output of the variable delay line 8), the output of the variable delay line 8 is differentiated by the differentiation circuit 18, and the output of the color detection circuit 7 is differentiated by the differentiation circuit 19. The rising edge of the detected color signal is detected by the positive pulse extracting circuits 20 and 22.
With negative pulse extraction circuits 21, 23
And the flip-flop 24 is operated as an input to the flip-flop 24. In this case, even if there is a pulse output from the positive pulse extraction circuit 22 after the positive pulse output from the positive pulse extraction circuit 20, a negative pulse extraction circuit 23 follows the negative pulse output from the negative pulse extraction circuit 21.
The operation of the flip-flop 24 does not change even if the output is a negative pulse (the same applies when the positive pulse extraction circuit 22 and the negative pulse extraction circuit 23 precede).
【0029】フリップフロップ24の出力でゲート回路
25をゲートしゲート回路17の出力から図7(c),
(e)のε1,ε2を取り出し、ローパスフィルタ26
でε1,ε2に比例した直流電圧としてからレベル比較
回路27で基準電圧Er1,Er2と比較して信号レベ
ルに応じた出力信号を出力する。The gate circuit 25 is gated by the output of the flip-flop 24, and the output of the gate circuit 17 is shown in FIG.
Taking out ε1 and ε2 of (e), the low-pass filter 26
Then, a DC voltage proportional to ε1 and ε2 is obtained, and then the level comparison circuit 27 compares it with the reference voltages Er1 and Er2 and outputs an output signal corresponding to the signal level.
【0030】図示しないが、本発明の色検出装置を警報
装置に応用するような場合には、Er1>Er2>0と
して、次の(1)〜(3)の場合に分けて用いることが
できる。Although not shown, in the case where the color detecting device of the present invention is applied to an alarm device, Er1>Er2> 0 can be set and used in the following cases (1) to (3). .
【数1】 ε>Er2 (1) 上記(1)の場合は被写体が近いので緊急音(例えば高
周波f2など)を警報として出す。次に、## EQU00001 ## .epsilon.> Er2 (1) In the case of (1) above, since the subject is close, an emergency sound (for example, high frequency f2) is issued as an alarm. next,
【数2】 Er2>ε>Er1 (2) 上記2)の場合は被写体は遠いが注意を喚起するための
注意音(例えば、低周波f1など)を警報として出す。
また、## EQU00002 ## Er2>.epsilon.> Er1 (2) In the case of 2) above, a warning sound (for example, low frequency f1) for calling attention is issued as an alarm although the subject is far.
Also,
【数3】 Er1≧ε (3) 上記(3)の場合には被写体はカメラから遠く離れてお
り安全なのでたとえ検出色があっても警報を出さない。[Equation 3] Er1 ≧ ε (3) In the case of the above (3), the subject is far away from the camera and is safe, so no alarm is given even if there is a detected color.
【0031】本実施例では撮像部Aを立体撮像用の2系
統の光学系と1つの撮像素子を有するものとして構成し
たが、これに限られず撮像部Aは、例えば、2台のカメ
ラで(図2のR,Lの各光軸に独立のカメラを配置し各
々1つずつの撮像素子を用いて)実現することもでき
る。なお、この場合は図5で可変遅延線8はH/2±τ
ではなくて±△/2程度の短いものでよく、しかも撮像
範囲が水平方向に広くとれるメリットがある。In the present embodiment, the image pickup section A is configured to have two optical systems for stereoscopic image pickup and one image pickup element, but the present invention is not limited to this, and the image pickup section A is, for example, two cameras ( This can also be realized by disposing independent cameras on each of the R and L optical axes in FIG. 2 and using one image pickup device for each. In this case, in FIG. 5, the variable delay line 8 is H / 2 ± τ.
Instead, it may be as short as ± Δ / 2, and there is an advantage that the imaging range can be wide in the horizontal direction.
【0032】なお、上述の2台のカメラを用いる方式に
対し、前述した立体撮像用の2系統の光学系と1つの撮
像素子を用いる方式では、カメラが1台で済むので経
済的であること、1台のカメラでよいので2台のカメ
ラを用いる場合のように色特性を合わせる必要がない等
のメリットがある。以上本発明の一実施例について説明
したが、本発明は上記実施例に限定されるものではな
く、種々の変形実施が可能であることはいうまでもな
い。In contrast to the above-described system using two cameras, the above-described system using two optical systems for stereoscopic imaging and one image sensor requires only one camera, which is economical. Since one camera is enough, there is an advantage that it is not necessary to match the color characteristics as in the case of using two cameras. Although one embodiment of the present invention has been described above, it is needless to say that the present invention is not limited to the above embodiment and various modifications can be made.
【0033】[0033]
【発明の効果】以上説明したように第1の発明の色検出
装置によれば、色検出部で検出される検出信号から特定
色検出像の面積(大きさ)を面積算出部で求め、距離判
定部で面積と距離の関係から特定色検出像と撮像位置と
の距離の遠近関係を判定するので、距離算出精度はラフ
でもよいので回路構成が簡単になる。As described above, according to the color detecting apparatus of the first invention, the area calculating unit obtains the area (size) of the specific color detection image from the detection signal detected by the color detecting unit, and the distance is calculated. Since the determination unit determines the perspective relationship of the distance between the specific color detection image and the imaging position from the relationship between the area and the distance, the distance calculation accuracy may be rough and the circuit configuration can be simplified.
【0034】第2の発明の警告装置は上記第1の発明の
色検出装置からの出力信号のレベルによって距離に応じ
て報知を行うことができる。この場合、安全位置(例え
ば、建設機械の安全圏)にいる者に対しては仮に特定色
が検出されても警告音等を発しないようにできる。ま
た、1台のカラーカメラに2光路の簡単な光学系を付加
することで所望の色の色検出のみでなく、被写体までの
距離をラフに測定することができるので建設現場等の注
意色の黄色の検出装置に適用すると作業者が近い場合と
やや離れている場合とで異なる警報を出し、安全圏にい
る場合には色検出しても警報を出さないようにできるの
で、より安全かつ効率的な作業が可能となる。The warning device according to the second aspect of the present invention can give a notification according to the distance by the level of the output signal from the color detection device according to the first aspect of the invention. In this case, a warning sound or the like can be prevented from being emitted to a person in a safe position (for example, the safety zone of the construction machine) even if the specific color is detected. In addition, by adding a simple optical system with two optical paths to one color camera, not only the desired color can be detected but the distance to the subject can be roughly measured. If applied to the yellow detection device, different alarms will be issued depending on whether the worker is near or slightly away, and if the worker is in the safety zone, the alarm will not be issued even if the color is detected. Work is possible.
【図1】本発明に基づく色検出装置の構成を示す基本ブ
ロック図である。FIG. 1 is a basic block diagram showing a configuration of a color detection device according to the present invention.
【図2】図1の撮像部の一実施例の構成を示す俯瞰図で
あり、立体撮像用の2系統の光学系を有する撮像部の例
である。FIG. 2 is a bird's-eye view showing a configuration of an embodiment of the image pickup unit in FIG. 1, and is an example of an image pickup unit having two optical systems for stereoscopic image pickup.
【図3】図2の撮像装置のCCD(撮像素子)からの出
力信号の説明図である。FIG. 3 is an explanatory diagram of an output signal from a CCD (image pickup element) of the image pickup apparatus in FIG.
【図4】被写体と撮像素子との関係を示す説明図であ
る。FIG. 4 is an explanatory diagram showing a relationship between a subject and an image sensor.
【図5】図1の色検出部B、面積算出部C、および距離
判定部Dの構成例を示すブロック図である。5 is a block diagram showing a configuration example of a color detection unit B, an area calculation unit C, and a distance determination unit D in FIG.
【図6】図5の可変遅延線の入出力信号の説明図であ
る。FIG. 6 is an explanatory diagram of input / output signals of the variable delay line of FIG.
【図7】図5の差動増幅器の入力再現像と入出力信号の
分布を示す図である。7 is a diagram showing input re-development and input / output signal distribution of the differential amplifier of FIG.
【図8】被写体との距離と差動増幅器の出力との関係を
示す図である。FIG. 8 is a diagram showing a relationship between a distance to a subject and an output of a differential amplifier.
【図9】色検出装置の一例を示す図である。FIG. 9 is a diagram illustrating an example of a color detection device.
1,2,3 反射鏡1(撮像部) 4 レンズ(撮像部) 5 CCD(撮像部) 6 出力信号(撮像信号) 7 色検出回路(色検出部) 8 可変遅延線(面積算出部) 9 差動増幅器(面積算出部) 10 H/2ゲートパルス発生器(面積算出部) 11 ゲート回路(面積算出部) 13 垂直エッジ検出回路(面積算出部) 14,15 マルチバイブレータ(面積算出部) 17 ゲート回路(面積算出部) 18,19 微分回路(距離判定部) 20,22 正パルス取り出し回路(距離判定部) 21,23 負パルス取り出し回路(距離判定部) 24 フリップフロップ(距離判定部) 25 ゲート回路(距離判定部) 26 ローパスフィルタ(距離判定部) 27 レベル比較回路(距離判定部) 100 色検出装置 A 撮像部 B 色検出部 C 面積算出部 D 距離判定部 1, 2 and 3 Reflector 1 (imaging unit) 4 Lens (imaging unit) 5 CCD (imaging unit) 6 Output signal (imaging signal) 7 Color detection circuit (color detection unit) 8 Variable delay line (Area calculation unit) 9 Differential amplifier (area calculation unit) 10 H / 2 gate pulse generator (area calculation unit) 11 Gate circuit (area calculation unit) 13 Vertical edge detection circuit (area calculation unit) 14, 15 Multivibrator (area calculation unit) 17 Gate circuit (area calculation unit) 18, 19 Differentiation circuit (distance determination unit) 20, 22 Positive pulse extraction circuit (distance determination unit) 21,23 Negative pulse extraction circuit (distance determination unit) 24 Flip-flop (distance determination unit) 25 Gate circuit (distance determination unit) 26 Low-pass filter (distance determination unit) 27 Level comparison circuit (distance determination unit) 100 Color detection device A Imaging unit B Color detection unit C Area calculation unit D Away the determination unit
Claims (2)
色検出部と、 撮像部からの撮像信号および色検出部からの出力信号に
基づく特定色検出像の2系統の信号の水平方向を時間的
に一致させて重複した部分を求めると共に、垂直方向の
大きさを求めることにより特定距離での特定色検出像の
面積を得る面積算出部と、 任意の距離での1系統分の特定色検出像信号の立上りと
他の系統の特定色検出像信号の立上りの差信号成分を面
積算出部からの出力に加え、その信号レベルを閾値と比
較して上記特定色検出像の遠近判定を行う距離判定部
と、 を有していることを特徴とする色検出装置。1. An image pickup unit for obtaining image pickup signals of two systems, a color detection unit for inputting an image pickup signal from the image pickup unit and detecting a signal of a specific color, and an image pickup signal from the image pickup unit and an output from the color detection unit. An area of the specific color detection image at a specific distance is obtained by obtaining the overlapped portion by temporally matching the horizontal directions of the signals of the specific color detection image based on the signal in the horizontal direction and obtaining the size in the vertical direction. The area calculation unit and the difference signal component between the rising of the specific color detection image signal for one system and the rising of the specific color detection image signal of another system at an arbitrary distance are added to the output from the area calculation unit, and the signal level thereof is added. And a distance determining unit that determines the perspective of the specific color detection image by comparing with a threshold value.
号のレベルによって報知状態が制御される報知手段を有
することを特徴とする警告装置。2. A warning device comprising: a notifying unit whose notification state is controlled by the level of an output signal from the color detection device according to claim 1.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP06189095A JP3578826B2 (en) | 1995-02-24 | 1995-02-24 | Color detection device and warning device |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP06189095A JP3578826B2 (en) | 1995-02-24 | 1995-02-24 | Color detection device and warning device |
Publications (2)
Publication Number | Publication Date |
---|---|
JPH08237637A true JPH08237637A (en) | 1996-09-13 |
JP3578826B2 JP3578826B2 (en) | 2004-10-20 |
Family
ID=13184202
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
JP06189095A Expired - Fee Related JP3578826B2 (en) | 1995-02-24 | 1995-02-24 | Color detection device and warning device |
Country Status (1)
Country | Link |
---|---|
JP (1) | JP3578826B2 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110279261A1 (en) * | 2010-05-12 | 2011-11-17 | Proxisafe Ltd. | Event warning system and method thereof |
CN111223261A (en) * | 2020-04-23 | 2020-06-02 | 佛山海格利德机器人智能设备有限公司 | Composite intelligent production security system and security method thereof |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106971500A (en) * | 2017-05-04 | 2017-07-21 | 国网山东省电力公司武城县供电公司 | The safe distance warning device and method of transformer |
-
1995
- 1995-02-24 JP JP06189095A patent/JP3578826B2/en not_active Expired - Fee Related
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110279261A1 (en) * | 2010-05-12 | 2011-11-17 | Proxisafe Ltd. | Event warning system and method thereof |
US8648709B2 (en) * | 2010-05-12 | 2014-02-11 | Proxisafe Ltd. | Event warning system and method thereof |
US20140159888A1 (en) * | 2010-05-12 | 2014-06-12 | Proxisafe Ltd. | Event warning system and method thereof |
US9202374B2 (en) * | 2010-05-12 | 2015-12-01 | Proxisafe Ltd. | Event warning system and method thereof |
CN111223261A (en) * | 2020-04-23 | 2020-06-02 | 佛山海格利德机器人智能设备有限公司 | Composite intelligent production security system and security method thereof |
CN111223261B (en) * | 2020-04-23 | 2020-10-27 | 佛山海格利德机器人智能设备有限公司 | Composite intelligent production security system and security method thereof |
Also Published As
Publication number | Publication date |
---|---|
JP3578826B2 (en) | 2004-10-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7167575B1 (en) | Video safety detector with projected pattern | |
US6765608B1 (en) | Image processing apparatus and system executing predetermined image processing on image data shot by camera, and image recording medium recorded with image processing program | |
JP4803376B2 (en) | Camera tampering detection method | |
US5422673A (en) | Video camera with automatic focus control | |
JP3011121B2 (en) | Security system | |
JP3330879B2 (en) | Object authentication system using color detection function | |
JP3578826B2 (en) | Color detection device and warning device | |
KR101656642B1 (en) | Group action analysis method by image | |
US3488436A (en) | Intrusion detector control for closed circuit television system | |
JP3640403B2 (en) | Color detection circuit for video signals | |
JPH10126766A (en) | Monitoring camera equipment | |
JPH11275566A (en) | Monitoring camera apparatus | |
KR102036593B1 (en) | Forward and back ward detecting system for emergency vehicle on bad visuality | |
KR101425658B1 (en) | Detection method of hacking and malfunction for surveillance camera system | |
JPH0389103A (en) | Apparatus for detecting approach of obstacle to overhead power transmission line | |
JP3249859B2 (en) | Circuit for extracting color component of video signal and warning device | |
JP4790555B2 (en) | Imaging device | |
JPH0158799B2 (en) | ||
JP2005031527A (en) | Picture projection device and picture projection method | |
KR20170052857A (en) | Driving assistant apparatus and driving assistant method | |
JP2002145064A (en) | Intruding object detecting device of railway line | |
JPS63102484A (en) | Abnormality monitoring device | |
JPH0916865A (en) | Monitor device using picture analysis | |
JP3510357B2 (en) | Color detector | |
JPH04193A (en) | Sighting device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
A977 | Report on retrieval |
Effective date: 20040116 Free format text: JAPANESE INTERMEDIATE CODE: A971007 |
|
A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20040210 |
|
A521 | Written amendment |
Effective date: 20040408 Free format text: JAPANESE INTERMEDIATE CODE: A523 |
|
TRDD | Decision of grant or rejection written | ||
A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20040706 |
|
A61 | First payment of annual fees (during grant procedure) |
Effective date: 20040714 Free format text: JAPANESE INTERMEDIATE CODE: A61 |
|
R150 | Certificate of patent (=grant) or registration of utility model |
Free format text: JAPANESE INTERMEDIATE CODE: R150 |
|
FPAY | Renewal fee payment (prs date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20080723 Year of fee payment: 4 |
|
FPAY | Renewal fee payment (prs date is renewal date of database) |
Year of fee payment: 5 Free format text: PAYMENT UNTIL: 20090723 |
|
FPAY | Renewal fee payment (prs date is renewal date of database) |
Year of fee payment: 6 Free format text: PAYMENT UNTIL: 20100723 |
|
FPAY | Renewal fee payment (prs date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20100723 Year of fee payment: 6 |
|
FPAY | Renewal fee payment (prs date is renewal date of database) |
Year of fee payment: 7 Free format text: PAYMENT UNTIL: 20110723 |
|
FPAY | Renewal fee payment (prs date is renewal date of database) |
Year of fee payment: 8 Free format text: PAYMENT UNTIL: 20120723 |
|
FPAY | Renewal fee payment (prs date is renewal date of database) |
Year of fee payment: 8 Free format text: PAYMENT UNTIL: 20120723 |
|
FPAY | Renewal fee payment (prs date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20130723 Year of fee payment: 9 |
|
LAPS | Cancellation because of no payment of annual fees |