JP2007151798A - Open/closed eye determining apparatus and method - Google Patents
Open/closed eye determining apparatus and method Download PDFInfo
- Publication number
- JP2007151798A JP2007151798A JP2005350496A JP2005350496A JP2007151798A JP 2007151798 A JP2007151798 A JP 2007151798A JP 2005350496 A JP2005350496 A JP 2005350496A JP 2005350496 A JP2005350496 A JP 2005350496A JP 2007151798 A JP2007151798 A JP 2007151798A
- Authority
- JP
- Japan
- Prior art keywords
- open
- eye
- eye determination
- opening
- closed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
Description
本発明は、運転者の目の開閉状態から、運転者の覚醒レベルを判定する開閉眼判定装置及び方法に関する。 The present invention relates to an open / closed eye determination device and method for determining a driver's arousal level from the open / closed state of the driver's eyes.
従来より、車両運転者の覚醒レベルを判定する手段として、運転者の顔を撮影するとともに、その運転者の目の開閉状態を検出し、その検出結果から覚醒レベルを判定する開閉眼判定装置が提案されている(例えば、特許文献1、特許文献2参照)。このような開閉眼判定装置を設置すれば、覚醒レベルが低下しているときは、運転者に警報等が発せられ、居眠り運転を回避することができる。 2. Description of the Related Art Conventionally, as a means for determining a vehicle driver's arousal level, an open / closed eye determination device that photographs a driver's face, detects the open / closed state of the driver's eyes, and determines the arousal level from the detection result. It has been proposed (see, for example, Patent Document 1 and Patent Document 2). If such an open / closed eye determination device is installed, when the arousal level is lowered, an alarm or the like is issued to the driver, and a drowsy driving can be avoided.
また、目の開度を検出し、所定時間毎における開度の度数分布を計測して、度数分布が略平坦となる範囲内に閾値を設定するとともに、度数分布の更新に応じて閾値を更新し、その閾値を用いて目の開閉を判定することにより、判定精度の向上を図った装置も提案されている。(特許文献3参照)
ところで、目の開閉を判断する装置においては、目の開閉を判断する指標として、目の縦幅を用いるものが多い。例えば、図17に示すような横320画素、縦240画素で撮影された顔画像に対しては、図18に示すように、目の周囲画像に対して2値化を行い、白→黒と変化する点(○印)と、黒→白と変化する点(×印)を求めることにより、目の縦幅を求めることができる。すなわち、○印のうち最も高い位置にある○印と、その○印の真下にある×印との間の画素数(○印と×印を含む画素数)を求めることにより、目の縦幅を求めることが可能である。 By the way, in many apparatuses that determine the opening and closing of the eyes, the vertical width of the eyes is often used as an index for determining the opening and closing of the eyes. For example, as shown in FIG. 18, for a face image shot with 320 pixels in the horizontal direction and 240 pixels in the vertical direction as shown in FIG. By obtaining a changing point (◯ mark) and a point changing from black to white (x mark), the vertical width of the eye can be obtained. That is, by calculating the number of pixels between the ○ mark at the highest position among the ○ marks and the X mark immediately below the ○ mark (the number of pixels including the ○ marks and the X marks), Can be obtained.
しかしながら、通常、上記特許文献1〜3においては、目の縦幅に相当する画素数は、図19に示すように、開眼持に10画素程度で、また閉眼持に5画素程度であり、画素数の変化は5画素程度に過ぎない。そのため、元々、目の小さい(細い)人を撮影した場合や、運転者が少し上を向いてカメラと顔との角度が変化した場合などには、開眼持と閉眼時の画素数の変化が非常に小さくなってしまい、開閉眼の判定ができないという問題がある。 However, normally, in Patent Documents 1 to 3, the number of pixels corresponding to the vertical width of the eye is about 10 pixels when the eye is open and about 5 pixels when the eye is closed, as shown in FIG. The change in number is only about 5 pixels. Therefore, when a person with a small (thin) eye is originally photographed or when the driver turns slightly upward and the angle between the camera and the face changes, there is a change in the number of pixels when holding the eye and closing the eye. There is a problem that it becomes very small and the determination of the open / closed eye cannot be made.
また、特許文献3のように度数分布を求めると、図20に示すように、開眼と閉眼分布の一部が重なったような形状となり、開閉眼を判定するための閾値を適切に決定することができないという問題がある。 Further, when the frequency distribution is obtained as in Patent Document 3, as shown in FIG. 20, a shape in which a part of the eye opening and the eye closing distribution overlap is obtained, and a threshold value for determining the opening / closing eye is appropriately determined. There is a problem that can not be.
本発明の課題は、車両運転者の開閉眼を確実かつ高精度に判定することのできる開閉眼判定装置、及び開閉眼判定方法を提供することにある。 An object of the present invention is to provide an open / closed eye determination device and an open / closed eye determination method that can reliably and accurately determine the open / closed eye of a vehicle driver.
上記課題を解決するために、本発明の開閉眼判定装置は、運転者の顔画像を取得する顔画像取得手段と、前記取得した顔画像のデータから目の位置を検出するとともに、該目の特徴点を抽出する特徴点抽出手段と、前記抽出された特徴点から開閉眼を判定する開閉眼判定手段とを備え、前記開閉眼判定手段は、前記特徴点として目の開き具合を示すデータを取り込んで、該データに対して目の開き具合を強調させる算術的な操作を行うとともに、その操作結果を用いて開閉眼の判定を行うことを特徴としている。 In order to solve the above problems, an open / closed eye determination device according to the present invention detects a face image acquisition means for acquiring a driver's face image, and detects an eye position from the acquired face image data. A feature point extracting unit configured to extract a feature point; and an open / closed eye determining unit configured to determine an open / closed eye based on the extracted feature point, wherein the open / closed eye determining unit includes data indicating an eye opening degree as the feature point. This is characterized in that an arithmetic operation for emphasizing the degree of eye opening is performed on the data, and the open / closed eye is determined using the operation result.
また、本発明の開閉眼判定方法は、運転者の顔画像を取得する顔画像取得ステップと、前記取得した顔画像のデータから目の位置を検出するとともに、該目の特徴点を抽出する特徴点抽出ステップと、前記抽出された特徴点から開閉眼を判定する開閉眼判定ステップとを含み、前記開閉眼判定ステップにおいて、前記特徴点として目の開き具合を示すデータを取り込んで、該データに対して目の開き具合を強調させる算術的な操作を行うとともに、その操作結果を用いて開閉眼の判定を行うことを特徴としている。 Further, the open / closed eye determination method of the present invention includes a face image acquisition step of acquiring a driver's face image, and a feature of detecting the eye position from the acquired face image data and extracting the feature point of the eye. A point extraction step and an opening / closing eye determination step for determining an opening / closing eye from the extracted feature point, and in the opening / closing eye determination step, data indicating the degree of opening of the eye is captured as the feature point, On the other hand, it is characterized by performing an arithmetic operation for emphasizing the degree of opening of the eyes and determining the open / closed eye using the operation result.
上記各構成によれば、取り込んだデータに対して目の開き具合を強調させる算術的な操作を行い、その操作結果を用いて開閉眼を判定するので、車両運転者の開閉眼を確実かつ高精度に判定することができる。 According to each of the above configurations, an arithmetic operation that emphasizes the degree of eye opening is performed on the captured data, and the opening / closing eye is determined using the operation result. The accuracy can be determined.
本発明によれば、車両運転者の開閉眼を確実かつ高精度に判定することが可能となる。 According to the present invention, it is possible to determine the opening / closing eyes of a vehicle driver reliably and with high accuracy.
以下、本発明の実施例を図面に従って説明する。 Embodiments of the present invention will be described below with reference to the drawings.
図1は、本発明に係る開閉眼判定装置の概略構成を示している。図に示すように、本開閉眼判定装置1は、運転者2の顔に近赤外線を照射する照明器3と、近赤外線を照射された運転者2の顔を撮像するビデオカメラと4、撮像結果であるビデオ信号がビデオカメラ4から時系列的に入力され、その入力情報に基づいて開閉眼判定結果を出力する判定装置本体5とを備えている。判定装置本体5から出力される開閉眼判定結果は、覚醒度推定装置6や居眠り検出装置7などに入力される。 FIG. 1 shows a schematic configuration of an open / closed eye determination device according to the present invention. As shown in the figure, the open / close eye determination device 1 includes an illuminator 3 that irradiates the face of the driver 2 with near infrared light, a video camera 4 that captures the face of the driver 2 irradiated with near infrared light, and an image. A video signal as a result is input from the video camera 4 in time series, and a determination device body 5 that outputs an open / closed eye determination result based on the input information is provided. The open / closed eye determination result output from the determination device body 5 is input to the arousal level estimation device 6 or the dozing detection device 7.
図2は、判定装置本体5の内部構成を示している。判定装置本体5は、ビデオカメラ4からのビデオ信号に基づいて顔画像を取得する顔画像取得手段8と、顔画像取得手段8から出力される顔画像を取り込んで、少なくとも目の上端と下端を含む目の特徴点を抽出する特徴点抽出手段9と、特徴点抽出手段9からの目の特徴点データを取り込んで、運転者2の目が開いているかどうかの判定(つまり開閉眼の判定)を行って、その判定結果(開閉眼判定結果)を出力する開閉眼判定手段10とを備えている。なお、顔画像取得手段8はメモリ(図示省略)を有し、ビデオカメラ4からのビデオ信号をディジタル画像としてキャプチャし、そのディジタル画像を前記メモリ内に一旦蓄積する。 FIG. 2 shows an internal configuration of the determination apparatus main body 5. The determination apparatus body 5 captures a face image acquisition unit 8 that acquires a face image based on a video signal from the video camera 4 and a face image output from the face image acquisition unit 8, and sets at least the upper and lower ends of the eyes. Feature point extraction means 9 for extracting the feature points of the eyes to be included, and determination of whether or not the eyes of the driver 2 are open by taking in the eye feature point data from the feature point extraction means 9 (that is, determination of the open / closed eyes) And an open / close eye determination means 10 for outputting the determination result (open / close eye determination result). The face image acquisition means 8 has a memory (not shown), captures a video signal from the video camera 4 as a digital image, and temporarily stores the digital image in the memory.
図3は、特徴点抽出手段9の内部構成を示している。特徴点抽出手段9は、顔画像取得手段8からの顔画像を取り込んで顔特徴検出結果を出力する顔特徴検出手段11と、顔画像取得手段8からの顔画像と顔特徴検出手段11からの顔特徴検出結果とを取り込んで目の特徴点データを出力する顔特徴追跡手段12とを備えている。 FIG. 3 shows the internal configuration of the feature point extraction means 9. The feature point extraction unit 9 captures the face image from the face image acquisition unit 8 and outputs a face feature detection result, and the face image from the face image acquisition unit 8 and the face feature detection unit 11 Face feature tracking means 12 for taking in the face feature detection results and outputting eye feature point data is provided.
顔特徴検出手段11は、図4(a)のフローチャートに示すように、検出モードによって、画面全体から片目となる候補を検出する片眼候補の検出処理(ステップS1及びステップS2)、または片眼候補の座標によって局所領域から両眼を検出し両眼を確定する両眼確定処理(ステップS1及びステップS3)のいずれかを実行する。 As shown in the flowchart of FIG. 4A, the face feature detection unit 11 detects a one-eye candidate (step S1 and step S2) for detecting a candidate for one eye from the entire screen, or one eye according to the detection mode. Either one of the binocular confirmation processing (step S1 and step S3) for detecting both eyes from the local region based on the candidate coordinates and confirming both eyes is executed.
ここで、片眼候補の検出処理は、図4(b)のフローチャートに示すように、片眼の候補を検出するための処理エリアを設定する処理(ステップS21)と、処理エリア内に連続データを検出する処理(ステップS22)と、片眼候補を絞込む処理(ステップS23)と、次フレームにおいて片眼候補の検出処理を行うか、両眼確定の処理を行うかを判断する処理(ステップS24)とを含んでいる。 Here, as shown in the flowchart of FIG. 4B, the single eye candidate detection process includes a process for setting a process area for detecting a single eye candidate (step S21), and continuous data in the process area. Detection processing (step S22), processing to narrow down one eye candidate (step S23), processing to determine whether to perform one eye candidate detection processing or binocular confirmation processing in the next frame (step S23) S24).
ステップS21における片眼の候補を検出するための処理エリアは、図5(a)に示すように、顔画像の左半分(もしくは右半分)に設定される。ステップS22における処理エリア内に連続データを検出する処理は、図7及び図8に示すように、処理エリア内の縦のラインごとに光量の濃度変化を調べて、白(淡)→黒(濃)と変化する点が横方向連続して連なる点を検出することにより行われる。ステップS23の片眼候補を絞り込む処理においては、検出された連続データが、上瞼のラインであれば、上に凸の形状をしているという仮定に基づいて形状認識を行い、上に凸の形状と合致したもののみを残すように処理する。 The processing area for detecting one eye candidate in step S21 is set to the left half (or right half) of the face image as shown in FIG. In the process of detecting continuous data in the processing area in step S22, as shown in FIGS. 7 and 8, the density change of the light amount is examined for each vertical line in the processing area, and white (light) → black (dark). ) Is detected by detecting a point where the points that change continuously in the horizontal direction are detected. In the process of narrowing down one eye candidate in step S23, if the detected continuous data is an upper eyelid line, shape recognition is performed based on the assumption that the shape is convex upward, and the upward convex Process so that only the one that matches the shape remains.
上記ステップS21〜S23の処理を複数フレームについて繰り返し行い、次フレームにおいて片眼候補の検出処理を行うか、両眼確定の処理を行うかを判断する処理において、図6に示すように、同一座標近辺に連続して上に凸の形状をして連続データとして検出される点を片眼候補点として、両眼確定の処理に遷移する。 As shown in FIG. 6, in the process of repeating the processes in steps S21 to S23 for a plurality of frames and determining whether to perform the one eye candidate detection process or the binocular confirmation process in the next frame, Transition to binocular determination processing is performed using a point detected as continuous data having a convex shape continuously in the vicinity and being detected as continuous data.
次に、両眼確定の処理は、図4(c)のフローチャートに示すように、片眼候補の座標に両眼を検出するための処理エリアを設定する処理(ステップS31)と、処理エリア内に連続データを検出する処理(ステップS32)と、両眼候補の検出とパターン認識により両眼を判定する処理(ステップS33)と、次フレームにおいて片眼候補の検出に戻るか引き続き両眼確定の処理を行うか眼位置追跡の処理を行うかを判断する処理(ステップS34)とを含んでいる。 Next, as shown in the flowchart of FIG. 4C, the binocular determination process includes a process of setting a process area for detecting both eyes at the coordinates of one eye candidate (step S31), Processing for detecting continuous data (step S32), processing for determining both eyes by detecting binocular candidates and pattern recognition (step S33), and returning to detection of one eye candidate in the next frame or continuing binocular confirmation And processing for determining whether to perform processing or eye position tracking processing (step S34).
パターン認識には、あらかじめ複数の眼の画像から辞書となる眼の画像を生成したものとの相関を求める相互相関法、あらかじめ複数の眼の画像と眼ではない顔部位画像(眼鏡のフレームや眉毛、鼻など)をニューラルネットワークによって学習し、眼か否かを判定するための画像を同ニューラルネットワークより判定する方法などが利用できる。 For pattern recognition, a cross-correlation method that obtains a correlation between a plurality of eye images generated beforehand from a dictionary eye image, a plurality of eye images and a non-eye facial part image (glasses frame or eyebrows) , Nose, etc.) are learned by a neural network, and an image for determining whether or not the eye is determined can be used.
顔特徴追跡手段12は、図9に示す処理を実行する。すなわち、顔特徴追跡手段12は、図10(a)に示すように、前回の該処理もしくは両眼確定処理により検出された眼の座標の近辺の領域から、基準眼を検出する(ステップS41及びS42)。基準眼が検出できた場合は、図10(b)に示すように、基準眼から前回の該処理もしくは両眼確定処理により検出された眼の座標から眼間距離を求めて反対眼の検出領域を設定し、その領域から反対眼を検出する(ステップS43及びS44)。 The face feature tracking unit 12 executes the process shown in FIG. That is, as shown in FIG. 10A, the face feature tracking unit 12 detects the reference eye from the region near the eye coordinates detected by the previous process or the binocular confirmation process (Step S41 and Step S41). S42). When the reference eye can be detected, as shown in FIG. 10B, the interocular distance is obtained from the coordinates of the eye detected by the previous processing or the binocular determination process from the reference eye, and the detection area of the opposite eye And the opposite eye is detected from the area (steps S43 and S44).
ステップS42において基準眼が検出できない場合には、反対眼を基準眼に変えて(ステップS46)、再び基準眼の検出を行う(ステップS47及びS48)。それでも基準眼が検出できない場合は、当フレームでの追記に失敗したとみなす(ステップS49)。 If the reference eye cannot be detected in step S42, the opposite eye is changed to the reference eye (step S46), and the reference eye is detected again (steps S47 and S48). If the reference eye still cannot be detected, it is considered that the additional writing in this frame has failed (step S49).
ステップS44において反対眼が検出できた場合には、検出した基準眼および反対眼の座標を確定する(ステップS45)。ステップS44において反対眼が検出できない場合には、前回の眼の座標から眼間距離を求めて、基準眼の座標から眼間距離相当分の移動位置を反対眼の座標として確定する(ステップS50)。 If the opposite eye can be detected in step S44, the coordinates of the detected reference eye and the opposite eye are determined (step S45). If the opposite eye cannot be detected in step S44, the interocular distance is obtained from the previous eye coordinates, and the movement position corresponding to the interocular distance is determined from the reference eye coordinates as the opposite eye coordinates (step S50). .
目の特徴点座標は、図18に示したように、目の付近の画像に対して、各列方向に画像の濃淡を調べて、淡(白)→濃(黒)と変化する点(図中○で表記)と、濃(黒)→淡(白)と変化する点(図中×で表記)を検出する。淡(白)→濃(黒)と変化する点の横方向への連続を調べることにより目の左端と右端を確定し、最も高い点を上端とし、濃(黒)→淡(白)と変化する点の最も低い点を下端とする。 As shown in FIG. 18, the eye feature point coordinates change from light (white) to dark (black) by examining the lightness and darkness of the image in the direction of each column with respect to the image near the eye (see FIG. A point that changes from dark (black) to light (white) (indicated by x in the figure) is detected. The left and right edges of the eye are determined by examining the horizontal continuity of the changing point from light (white) to dark (black), with the highest point at the top, and changing from dark (black) to light (white). The lowest point is the lower end.
開閉眼判定手段10は、図11に示すように、特徴点抽出手段9(図2参照)からの目の特徴点座標を取り込んで、開閉眼判定するための開閉眼判定指標値を出力する指標値計算手段13と、所定時間ごとに開閉眼判定指標値の分布を求め、この分布から開閉眼判定閾値を出力する閾値決定手段14と、指標値計算手段13からの開閉眼判定指標値及び閾値決定手段14からの開閉眼判定閾値を取り込んで、現開閉眼判定指標値が開・閉のいずれかを分別するとともに、その分別結果を開閉眼判定結果として出力する開閉眼分別手段15とを備えている。 As shown in FIG. 11, the open / close eye determination means 10 takes in the feature point coordinates of the eye from the feature point extraction means 9 (see FIG. 2) and outputs an open / close eye determination index value for open / close eye determination. A value calculation unit 13; a threshold value determination unit 14 that obtains an opening / closing eye determination index value distribution at predetermined time intervals; and an opening / closing eye determination index value and threshold value from the index value calculation unit 13 An open / close eye classification unit 15 that takes in the open / close eye determination threshold value from the determination unit 14 and classifies whether the current open / close eye determination index value is open or closed, and outputs the classification result as an open / close eye determination result; ing.
指標値計算手段13は、目の特徴点座標から、図12に示すように、右目の上端と下端の点の差を目の縦幅hrに、左目の上端と下端の点の差を目の縦幅hlにそれぞれ設定する。また、両目それぞれの縦幅を0.0〜1.0に正規化したものをdr,dlにそれぞれ設定する。正規化の方法としては、目の縦幅をhr,hl、横幅をwr,wlとしたときの縦横比をdr,dlとする方法や、目の縦幅hr,hlを一律の相場値で除算する方法などがある。 Eye index value calculating means 13, the eye feature point coordinates, as shown in FIG. 12, the vertical width h r a difference in terms of top and bottom eye of the right eye, the difference in point of the left eye of the upper and lower ends Is set to the vertical width hl of each. Further, it sets each obtained by normalizing each of the vertical width both eyes 0.0 to 1.0 d r, the d l. As a normalization method, the aspect ratio when the vertical width of the eye is h r , h l and the horizontal width is w r , w l is set to dr , d l , or the vertical width h r , h of the eye. There is a method of dividing l by a uniform market value.
そして、以下の(1)式のようにdr,dlの積を求めて対数を取り、符号を反転した値rを各フレームで求める。このrを開閉眼判定指標値と呼ぶ。
r=−log10(dr*dl) ・・・・・・(1)
Then, the product of d r and d l is obtained as shown in the following equation (1), the logarithm is taken, and the value r with the sign inverted is obtained for each frame. This r is called an open / close eye determination index value.
r = −log 10 (d r * d l ) (1)
片目のみしか追跡できていない場合は、開閉眼判定指標値を追跡できている眼の情報、右目ならばdr,左目ならばdlのみを用いて、以下の(2)式または(3)式で求める。
r=−log10(dr*dr) ・・・・・・(2)
または、
r=−log10(dl*dl) ・・・・・・(3)
When only one eye can be tracked, the following equation (2) or (3) is used by using only the eye information that can track the open / closed eye determination index value, d r for the right eye and d l for the left eye. Calculate by formula.
r = −log 10 (d r * d r ) (2)
Or
r = −log 10 (d 1 * d 1 ) (3)
閾値決定手段14は、所定数の各フレームで求めた開閉眼判定指標値rを、所定の区間(例えば0.1区間)を横軸に、度数を縦軸にした分布として表し、毎の開閉眼判定指標値分布を求め、閾値を決定する。開閉眼判定指標値分布は一般に分布は図13のような形状を示す。 The threshold value determination means 14 represents the opening / closing eye determination index value r obtained in a predetermined number of frames as a distribution with a predetermined interval (for example, 0.1 interval) on the horizontal axis and the frequency on the vertical axis. An eye determination index value distribution is obtained and a threshold value is determined. The open / closed eye determination index value distribution generally has a shape as shown in FIG.
図14に示すように、分布について、小さいほうから最初の極大値(第1の極大値)の存在するrをrmin、大きいほうから最初の極大値(第2の極大値)の存在するrをrmaxとする。rminとrmaxの中央値を開閉判定閾値tとして用いる。度数分布は直近の所定数のrによって決定されるため開閉判定閾値tは常に変化する。 As shown in FIG. 14, for the distribution, r min is the r where the first maximum value (first maximum value) exists from the smallest, and r is the r where the first maximum value (second maximum value) exists from the larger one. Is r max . The median value of r min and rmax is used as the open / close determination threshold value t. Since the frequency distribution is determined by the most recent predetermined number of r, the open / close determination threshold value t always changes.
また、図15に示すように、開眼の頻度に比べて閉眼の頻度が極めて少ない場合には、rmin =rmax となる場合がある。この場合は、rminから相場値をもって開閉判定閾値tを決定する。他の決定手法としては、分布の中心から一定割合の度数を占める範囲でt>rminとなる点にする方法がある。 Further, as shown in FIG. 15, r min = r max may be obtained when the frequency of closed eyes is very small compared to the frequency of opening eyes. In this case, the open / close determination threshold value t is determined from the market value from r min . As another determination method, there is a method of setting t> r min within a range that occupies a certain percentage of frequency from the center of the distribution.
開閉眼分別手段15は、図16に示すように、開閉眼判定閾値tよりも現在の開閉眼判定指標値rが大きい場合を閉眼、小さい場合を開眼することにより、開閉判定結果を出力する。 As shown in FIG. 16, the open / close eye classification unit 15 outputs an open / close determination result by opening the eyes when the current open / close eye determination index value r is larger than the open / close eye determination threshold t and opening the eyes when the current value is small.
本実施例によれば、開閉眼判定手段10が、算術的な操作を行うための開閉眼判定指標値を計算する指標値計算手段13と、所定時間ごとに開閉眼判定指標値の度数分布を求め、この度数分布から開閉眼判定閾値を求める閾値決定手段14と、求めた開閉眼判定閾値によって、現在の開閉眼判定指標値が開・閉のいずれかを分別する開閉眼分別手段15とを有するので、所定時間での目の開閉眼判定指標値の分布から目の開閉分別のための閾値が決定され、常に最適な閾値を決定することができる。 According to the present embodiment, the open / close eye determination means 10 calculates the index value calculation means 13 for calculating the open / close eye determination index value for performing an arithmetic operation, and the frequency distribution of the open / close eye determination index value every predetermined time. And a threshold value determining means 14 for determining an opening / closing eye determination threshold value from the frequency distribution, and an opening / closing eye classification means 15 for determining whether the current opening / closing eye determination index value is open or closed based on the determined opening / closing eye determination threshold value. Therefore, the threshold for eye opening / closing discrimination is determined from the distribution of the eye opening / closing eye determination index value at a predetermined time, and the optimum threshold can always be determined.
また、本実施例によれば、指標値計算手段13が、両目の縦幅を一定値で除算したもの同士を乗じて対数をとることにより得られた値を開閉眼判定指標値としているので、指標値として単に目の縦幅を用いる場合と比較して、目の開と閉における変化幅を大きくすることができる。 Further, according to the present embodiment, the index value calculation means 13 uses the value obtained by multiplying the vertical width of both eyes divided by a constant value and taking the logarithm as the opening / closing eye determination index value. Compared to the case where the vertical width of the eye is simply used as the index value, the change width at the opening and closing of the eye can be increased.
また、本実施例によれば、指標値計算手段13が、片方の目の縦幅を一定値で除算したものを二乗して対数をとることにより得られた値を開閉眼判定指標値としているので、片目のみしか検出できていない場面においても、開閉眼判定指標値を容易に計算することができる。 In addition, according to the present embodiment, the index value calculation means 13 uses the value obtained by squaring the value obtained by dividing the vertical width of one eye by a constant value as the open / close eye determination index value. Therefore, even in a scene where only one eye can be detected, the open / close eye determination index value can be easily calculated.
また、本実施例によれば、閾値決定手段14が、度数分布上で一方の端に最も近い箇所に現れる第1の極大値と、他方の端に最も近い箇所に現れる第2の極大値との間に開閉眼判定閾値を設定しているので、第1の極大値は開眼の最も多い位置を、第2の極大値は閉眼の最も多い位置をそれぞれ示すことになり、開眼と閉眼を分別する閾値を適切に設定することができる。 Further, according to the present embodiment, the threshold value determination means 14 includes the first maximum value appearing at the location closest to one end on the frequency distribution, and the second maximum value appearing at the location closest to the other end. Since the open / closed eye determination threshold is set between the first and second eyes, the first maximum value indicates the position with the most eyes open, and the second maximum value indicates the position with the most eyes closed. The threshold value to be set can be set appropriately.
また、本実施例によれば、閾値決定手段14が、度数分布上で一方の端に最も近い箇所に現れる極大値を求め、その極大値から一定指標値離れた位置に開閉眼判定閾値を設定しているので、開眼の状態のみでも閾値を決定することができる。すなわち、一般に極大値が一つしかない場合は、閉眼がほとんどない状態であるため、求められる極大値は開眼の最も多い位置を示すことになり、開眼の状態のみでも閾値を容易に決定することが可能となる。 Further, according to the present embodiment, the threshold value determination means 14 obtains a maximum value appearing at a location closest to one end on the frequency distribution, and sets the open / close eye determination threshold value at a position away from the maximum value by a certain index value. Therefore, the threshold value can be determined only with the open eye state. In other words, in general, when there is only one maximum value, there is almost no closed eye, so the required maximum value indicates the position with the most eyes open, and the threshold value can be easily determined even with only the eye open state. Is possible.
また、本実施例によれば、閾値決定手段14が、度数分布全体に対して一定の割合を占める位置に開閉眼判定閾値を設定しているので、開眼を閉眼と誤判定する機会を減らすことができる。すなわち、一般に極大値が一つしかない場合は、閉眼がほとんどない状態であるため、求められる極大値は開眼の最も多い位置を示すことになり、開眼の分布の一定割合を占める部分から離れた所に閾値が決定され、開眼を閉眼と誤判定する機会を減らすことが可能となる。 In addition, according to the present embodiment, the threshold value determination unit 14 sets the open / closed eye determination threshold value at a position that occupies a certain ratio with respect to the entire frequency distribution, thereby reducing the chance of erroneously determining that the eye is open. Can do. That is, in general, when there is only one maximum value, there is almost no closed eye, so the maximum value obtained will indicate the position where the eyes are most open, away from the portion that occupies a certain percentage of the distribution of eye opening The threshold value is determined in this place, and the chance of erroneously determining that the eye is open is reduced.
また、本実施例によれば、特徴点抽出手段9が、顔画像取得手段8から顔画像を取り込んで顔特徴を検出する顔特徴検出手段11と、顔特徴が検出された場合に、顔画像取得手段8からの顔画像と顔特徴検出手段11での顔特徴の検出結果を取り込んで顔特徴点を出力するとともに、以後は顔画像と過去の顔特徴点を取り込んで顔特徴点を更新出力する顔特徴追跡手段12とを有するので、常に顔画像全体から顔特徴を検出する必要がなく、過去の顔特徴点の情報を用いて、現在の顔特徴点を探すことが可能となり、処理の効率化及び高速化を図ることができる。 Further, according to the present embodiment, the feature point extracting unit 9 captures the face image from the face image acquiring unit 8 and detects the face feature, and when the face feature is detected, the face image is detected. The facial image from the acquisition unit 8 and the facial feature detection result by the facial feature detection unit 11 are captured and the facial feature point is output. Thereafter, the facial image and the past facial feature point are captured and the facial feature point is updated and output. Therefore, it is not always necessary to detect the face feature from the entire face image, and it is possible to search for the current face feature point using the information of the past face feature point. Efficiency and speed can be increased.
さらに、本実施例によれば、特徴点抽出手段9が、目の特徴点を出力する際に、両目のうち少なくとも片方の目の上端及び下端の座標を求めて目の縦方向の幅を算出し、その算出結果を、目の開き具合を示すデータとして出力するので、目の開閉判定のための開閉眼判定指標値を容易に計算できる。 Furthermore, according to the present embodiment, when the feature point extraction means 9 outputs the feature point of the eye, it calculates the vertical width of the eye by obtaining the coordinates of the upper and lower ends of at least one of the eyes. Since the calculation result is output as data indicating the degree of opening of the eyes, the opening / closing eye determination index value for opening / closing the eyes can be easily calculated.
以上、本発明の実施例を図面により詳述してきたが、上記実施例は本発明の例示にしか過ぎないものであり、本発明は上記実施例の構成にのみ限定されるものではない。本発明の要旨を逸脱しない範囲の設計の変更等があっても、本発明に含まれることは勿論である。 As mentioned above, although the Example of this invention has been explained in full detail with drawing, the said Example is only the illustration of this invention, and this invention is not limited only to the structure of the said Example. Needless to say, changes in design and the like within the scope of the present invention are included in the present invention.
例えば、開閉眼判定指標値rを求める場合、上記(1)式の代わりに次のような(4)又は(5)式を用いてもよい。
r=−log10(dl*dr)^k (”^” は巾乗の記号) ・・・・・・(4)
r=−k*log10(dl*dr) (kは任意係数) ・・・・・・(5)
For example, when obtaining the opening / closing eye determination index value r, the following equation (4) or (5) may be used instead of the above equation (1).
r = −log 10 (d 1 * d r ) ^ k (“^” is a sign of the power) ・ ・ ・ ・ ・ ・ (4)
r = −k * log 10 (d 1 * d r ) (k is an arbitrary coefficient) (5)
1 開閉眼判定装置
2 運転者
4 ビデオカメラ
5 判定装置本体
8 顔画像取得手段
9 特徴点抽出手段
10 開閉眼判定手段
11 顔特徴検出手段
12 顔特徴追跡手段
13 指標値計算手段
14 閾値決定手段
15 開閉眼分別手段
DESCRIPTION OF SYMBOLS 1 Opening / closing eye determination apparatus 2 Driver 4 Video camera 5 Determination apparatus main body 8 Face image acquisition means 9 Feature point extraction means 10 Opening / closing eye determination means 11 Face feature detection means 12 Face feature tracking means 13 Index value calculation means 14 Threshold value determination means 15 Open / close eye separation means
Claims (14)
前記開閉眼判定手段は、前記特徴点として目の開き具合を示すデータを取り込んで、該データに対して目の開き具合を強調させる算術的な操作を行うとともに、その操作結果を用いて開閉眼の判定を行うことを特徴とする開閉眼判定装置。 Face image acquisition means for acquiring a driver's face image, feature point extraction means for detecting eye positions from the acquired face image data, and extracting feature points of the eyes, and the extracted feature points And an open / close eye determination means for determining the open / close eye from
The open / close eye determination means takes in data indicating the degree of eye opening as the feature point, performs an arithmetic operation to emphasize the eye opening degree for the data, and uses the operation result to open / close the eye An open / close eye determination device characterized by
前記開閉眼判定ステップにおいて、前記特徴点として目の開き具合を示すデータを取り込んで、該データに対して目の開き具合を強調させる算術的な操作を行うとともに、その操作結果を用いて開閉眼の判定を行うことを特徴とする開閉眼判定方法。 A face image acquisition step of acquiring a driver's face image; a feature point extraction step of detecting a position of an eye from the acquired face image data; and extracting the feature point of the eye; and the extracted feature point And an open / close eye determination step for determining an open / close eye from
In the opening / closing eye determination step, data indicating the degree of opening of the eyes is taken as the feature point, and an arithmetic operation for emphasizing the degree of opening of the eyes is performed on the data, and the opening / closing eyes are used using the operation result. An open / closed eye determination method, comprising:
12. The open / closed eye determination method according to claim 11, wherein as an arithmetic operation in the open / close eye determination step, calculation is performed by squaring a value obtained by dividing the vertical width of one eye by a constant value to obtain a logarithm. .
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2005350496A JP4788319B2 (en) | 2005-12-05 | 2005-12-05 | Opening and closing eye determination device and method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2005350496A JP4788319B2 (en) | 2005-12-05 | 2005-12-05 | Opening and closing eye determination device and method |
Publications (2)
Publication Number | Publication Date |
---|---|
JP2007151798A true JP2007151798A (en) | 2007-06-21 |
JP4788319B2 JP4788319B2 (en) | 2011-10-05 |
Family
ID=38236901
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
JP2005350496A Expired - Fee Related JP4788319B2 (en) | 2005-12-05 | 2005-12-05 | Opening and closing eye determination device and method |
Country Status (1)
Country | Link |
---|---|
JP (1) | JP4788319B2 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2009279186A (en) * | 2008-05-22 | 2009-12-03 | Toyota Motor Corp | Face detecting device and method |
WO2010032725A1 (en) * | 2008-09-16 | 2010-03-25 | アイシン精機株式会社 | Drowsiness determining device and program |
JP2011152218A (en) * | 2010-01-26 | 2011-08-11 | Toyota Motor Corp | Eye opening degree determining device and eye opening degree determining method |
US8433095B2 (en) | 2009-02-09 | 2013-04-30 | Denso Corporation | Drowsiness detector |
US9082012B2 (en) | 2011-12-06 | 2015-07-14 | Denso Corporation | Eye state detection apparatus and method of detecting open and closed states of eye |
EP3965003A4 (en) * | 2019-05-21 | 2022-07-06 | Huawei Technologies Co., Ltd. | Image processing method and device |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2003000571A (en) * | 2001-06-18 | 2003-01-07 | Mitsubishi Electric Corp | Device and method for estimating arousal level |
JP2004341954A (en) * | 2003-05-16 | 2004-12-02 | Nissan Motor Co Ltd | Arousal level determination device |
-
2005
- 2005-12-05 JP JP2005350496A patent/JP4788319B2/en not_active Expired - Fee Related
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2003000571A (en) * | 2001-06-18 | 2003-01-07 | Mitsubishi Electric Corp | Device and method for estimating arousal level |
JP2004341954A (en) * | 2003-05-16 | 2004-12-02 | Nissan Motor Co Ltd | Arousal level determination device |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2009279186A (en) * | 2008-05-22 | 2009-12-03 | Toyota Motor Corp | Face detecting device and method |
WO2010032725A1 (en) * | 2008-09-16 | 2010-03-25 | アイシン精機株式会社 | Drowsiness determining device and program |
JP2010068848A (en) * | 2008-09-16 | 2010-04-02 | Toyota Central R&D Labs Inc | Drowsiness determination apparatus and program |
CN102149326A (en) * | 2008-09-16 | 2011-08-10 | 爱信精机株式会社 | Drowsiness determining device and program |
US8433095B2 (en) | 2009-02-09 | 2013-04-30 | Denso Corporation | Drowsiness detector |
JP2011152218A (en) * | 2010-01-26 | 2011-08-11 | Toyota Motor Corp | Eye opening degree determining device and eye opening degree determining method |
US9082012B2 (en) | 2011-12-06 | 2015-07-14 | Denso Corporation | Eye state detection apparatus and method of detecting open and closed states of eye |
EP3965003A4 (en) * | 2019-05-21 | 2022-07-06 | Huawei Technologies Co., Ltd. | Image processing method and device |
Also Published As
Publication number | Publication date |
---|---|
JP4788319B2 (en) | 2011-10-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP4307496B2 (en) | Facial part detection device and program | |
JP4845698B2 (en) | Eye detection device, eye detection method, and program | |
JP5359266B2 (en) | Face recognition device, face recognition method, and face recognition program | |
US8879847B2 (en) | Image processing device, method of controlling image processing device, and program for enabling computer to execute same method | |
JP6234762B2 (en) | Eye detection device, method, and program | |
JP4912206B2 (en) | Image processing method, image processing apparatus, image processing system, and computer program | |
JP2008113902A (en) | Apparatus and method for detecting eye opening degree | |
JP4788319B2 (en) | Opening and closing eye determination device and method | |
JP5271227B2 (en) | Crowd monitoring device, method and program | |
EP2704056A2 (en) | Image processing apparatus, image processing method | |
JP3926507B2 (en) | Eye position and face position detection device | |
EP2309454B1 (en) | Apparatus and method for detecting motion | |
CN107040713A (en) | Apparatus for determining motion vectors and its control method and picture pick-up device | |
CN105310690A (en) | Wrinkle detection apparatus and wrinkle detection method | |
JPWO2018078857A1 (en) | Gaze estimation apparatus, gaze estimation method, and program recording medium | |
JPWO2013008304A1 (en) | 瞼 Detection device | |
JP2012068948A (en) | Face attribute estimating apparatus and method therefor | |
KR20130091441A (en) | Object tracking device and method for controlling thereof | |
KR100977259B1 (en) | Method for a human searching and tracking using a multifulscale histogram of oriented gradient | |
KR101146417B1 (en) | Apparatus and method for tracking salient human face in robot surveillance | |
JP2004192551A (en) | Eye opening/closing determining apparatus | |
KR20130126386A (en) | Adaptive color detection method, face detection method and apparatus | |
JP2004192552A (en) | Eye opening/closing determining apparatus | |
KR101290517B1 (en) | Photographing apparatus for tracking object and method thereof | |
JP2007034436A (en) | Arousal estimation device and method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
A621 | Written request for application examination |
Free format text: JAPANESE INTERMEDIATE CODE: A621 Effective date: 20081126 |
|
A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20110329 |
|
A521 | Written amendment |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20110526 |
|
TRDD | Decision of grant or rejection written | ||
A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20110621 |
|
A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 |
|
A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20110704 |
|
FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20140729 Year of fee payment: 3 |
|
R150 | Certificate of patent or registration of utility model |
Free format text: JAPANESE INTERMEDIATE CODE: R150 |
|
LAPS | Cancellation because of no payment of annual fees |