JP2803222B2 - Eye gaze detection device - Google Patents
Eye gaze detection deviceInfo
- Publication number
- JP2803222B2 JP2803222B2 JP1247333A JP24733389A JP2803222B2 JP 2803222 B2 JP2803222 B2 JP 2803222B2 JP 1247333 A JP1247333 A JP 1247333A JP 24733389 A JP24733389 A JP 24733389A JP 2803222 B2 JP2803222 B2 JP 2803222B2
- Authority
- JP
- Japan
- Prior art keywords
- conversion element
- photoelectric conversion
- corneal reflection
- eyeball
- reflection image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B2213/00—Viewfinders; Focusing aids for cameras; Means for focusing for cameras; Autofocus systems for cameras
- G03B2213/02—Viewfinders
- G03B2213/025—Sightline detection
Landscapes
- Automatic Focus Adjustment (AREA)
- Eye Examination Apparatus (AREA)
- Exposure Control For Cameras (AREA)
- Focusing (AREA)
Description
【発明の詳細な説明】 （産業上の利用分野） 本発明は視線検出装置に関し、例えばカメラ等の光学
装置において、撮影系による被写体像が形成されている
観察面（ピント面）上の観察者（撮影者）が観察してい
る注視点方向の軸いわゆる視線（視軸）を、観察者の眼
球面上を照明したときに得られる眼球の反射像を利用し
て検出するようにした視線検出装置に関するものであ
る。Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a line-of-sight detection device, for example, in an optical device such as a camera, an observer on an observation surface (focusing surface) on which a subject image is formed by a photographing system. Eye gaze detection in which the axis in the direction of the point of gaze observed by the (photographer), that is, the so-called gaze (gaze axis) is detected using a reflection image of the eyeball obtained when the observer illuminates the eyeball. It concerns the device.
（従来の技術） 従来より観察者が観察面上のどの位置を観察している
かを検出する、いわゆる視線（視軸）を検出する装置が
種々提案されている。(Prior Art) Conventionally, various devices have been proposed for detecting what position on an observation surface the observer is observing, that is, detecting a so-called line of sight (a visual axis).
例えば特開昭61−172552号公報においては、光源から
の平行光束を観察者の眼球の前眼部へ投射し、角膜から
の反射光による角膜反射像と瞳孔の結像位置を利用して
視軸を求めている。第８図（Ａ），（Ｂ）は視線検出方
法の原理説明図で、同図（Ａ）は視線検出光学系の概略
図、同図（Ｂ）は光電素子列６からの出力信号の強度図
である。For example, in Japanese Patent Application Laid-Open No. Sho 61-172552, a parallel light beam from a light source is projected to the anterior segment of an observer's eyeball, and a visual image is formed by utilizing a corneal reflection image formed by light reflected from the cornea and an image forming position of a pupil. Seeking an axis. 8 (A) and 8 (B) are explanatory diagrams of the principle of the line-of-sight detection method. FIG. 8 (A) is a schematic diagram of the line-of-sight detection optical system, and FIG. 8 (B) is the intensity of the output signal from the photoelectric element array 6. FIG.
同図において５は観察者に対して不感の赤外光を放射
する発光ダイオード等の光源であり、投光レンズ３の焦
点面に配置されている。In the figure, reference numeral 5 denotes a light source such as a light emitting diode which emits infrared light insensitive to an observer, and is disposed on the focal plane of the light projecting lens 3.
光源５より発光した赤外光は投光レンズ３により平行
光となりハーフミラー２で反射し、眼球201の角膜21を
照明する。このとき角膜21の表面で反射した赤外光の一
部による角膜反射像ｄはハーフミラー２を透過し受光レ
ンズ４により集光され光電素子列６上の位置Zd′に再結
像する。The infrared light emitted from the light source 5 becomes parallel light by the light projecting lens 3 and is reflected by the half mirror 2 to illuminate the cornea 21 of the eyeball 201. At this time, the corneal reflection image d due to a part of the infrared light reflected on the surface of the cornea 21 is transmitted through the half mirror 2, condensed by the light receiving lens 4, and re-images at the position Zd ′ on the photoelectric element array 6.
また虹彩23の端部a,bからの光束はハーフミラー２、
受光レンズ４を介して光電素子列６上の位置Za′,Zb′
に該端部a,bの像を結像する。受光レンズ４の光軸（光
軸ア）に対する眼球の光軸イの回転角θが小さい場合、
虹彩23の端分a,bのＺ座標をZa,Zbとすると、虹彩23の中
心位置ｃの座標Zcは Zc≒（Za＋Zb）/2 と表わされる。Light beams from the ends a and b of the iris 23 are
Positions Za ′ and Zb ′ on the photoelectric element array 6 via the light receiving lens 4
Then, the images of the ends a and b are formed. When the rotation angle θ of the optical axis a of the eyeball with respect to the optical axis (optical axis a) of the light receiving lens 4 is small,
Assuming that the Z coordinates of the ends a and b of the iris 23 are Za and Zb, the coordinate Zc of the center position c of the iris 23 is expressed as Zca (Za + Zb) / 2.
又、角膜反射像の発生位置ｄのＺ座標をZd、角膜21の
曲率中心Ｏと虹彩23の中心Ｃまでの距離を▲▼とす
ると眼球光軸イの回転角θは、 ▲▼＊SINθ≒Zc−Zd ……（１） の関係式を略満足する。Further, assuming that the Z coordinate of the generation position d of the corneal reflection image is Zd, and the distance between the center of curvature O of the cornea 21 and the center C of the iris 23 is ▲ ▼, the rotation angle θ of the optical axis of the eyeball is ▲ ▼ * SINθ ≒ Zc−Zd... (1)
ここで角膜反射像の位置ｄのＺ座標Zdと角膜21の曲率
中心ＯのＺ座標Z0とは一致している。このため演算手段
９において、同図（Ｂ）のごとく光電素子列６面上に投
影された各特異点（角膜反射像ｄ及び虹彩の端部a,b）
の位置を検出することにより眼球光軸イの回転角θを求
めることができる。この時（１）式は、 とかきかえられる。但し、βは角膜反射像の発生位置ｄ
と受光レンズ４との距離L1と受光レンズ４と光電素子列
６との距離L0で決まる倍率で、通常ほぼ一定の値となっ
ている。22は眼球201の強膜、Ｏ′は眼球201の回転中心
である。Here are coincident with the Z-coordinate Z 0 of the center of curvature O of the Z-coordinate Zd and cornea 21 position d of the cornea reflected image. Therefore, in the arithmetic means 9, each singular point (corneal reflection image d and end portions a and b of the iris) projected on the photoelectric element array 6 as shown in FIG.
, The rotation angle θ of the eyeball optical axis A can be obtained. At this time, equation (1) is Can be changed. Here, β is the position d where the corneal reflection image is generated.
The magnification is determined by the distance L1 between the light-receiving lens 4 and the light-receiving lens 4 and the distance L0 between the light-receiving lens 4 and the photoelectric element array 6, and is usually substantially constant. 22 is the sclera of the eyeball 201, and O 'is the center of rotation of the eyeball 201.
これは例えばカメラの自動焦点検出装置において測距
点を画面中心のみならず画面内の複数箇所に設けた場
合、観察者がそのうち１つの測距点を選択して自動焦点
検出を行おうとする場合、その１つを選択入力する手間
を省き観察者が観察している点を測距点とみなし、該測
距点を自動的に選択して自動焦点検出を行うのに有効で
ある。カメラの測距点はファインダー画面内に２次元的
に設定されていることが望ましいため、視線検出装置も
２軸の視線情報を検出することが必要となってくる。This is the case, for example, in a case where a focus detection point is provided not only at the center of the screen but also at a plurality of places in the screen in an automatic focus detection device of a camera, and when an observer selects one of the focus detection points to perform automatic focus detection. This is effective in that the point observed by the observer is regarded as a distance measuring point by eliminating the trouble of selecting and inputting one of the distance measuring points, and the distance measuring point is automatically selected to perform automatic focus detection. Since it is desirable that the ranging points of the camera are set two-dimensionally in the viewfinder screen, it is necessary for the line-of-sight detection device to also detect two-axis line-of-sight information.
（発明が解決しようとする問題点） 第８図示す視線検出方法は、観察者の眼球がＺ−Ｘ平
面（例えば水平面）内で回転する場合の他にＸ−Ｙ平面
（例えば垂直面）内で回転する場合においても原理的に
は有効である。(Problems to be Solved by the Invention) The line-of-sight detection method shown in FIG. 8 uses the XY plane (for example, vertical plane) in addition to the case where the observer's eyeball rotates in the ZX plane (for example, horizontal plane). It is also effective in principle when rotating with.
第９図は２次元の光電素子列６面上に投影された眼球
の反射像を示す説明図である。眼球のＸ−Ｙ平面内の回
転角を検出するには虹彩23と瞳孔24とのＹ軸方向の境界
点Ya′,Yb′を検出しなければならない。FIG. 9 is an explanatory diagram showing a reflected image of the eyeball projected on the two-dimensional photoelectric element array 6. In order to detect the rotation angle of the eyeball in the XY plane, it is necessary to detect the boundary points Ya 'and Yb' between the iris 23 and the pupil 24 in the Y-axis direction.
一般に光電素子列６の読み出しは線順次で行なわれ、
光電素子列の列方向と直交する方向（Ｙ軸方向）の情報
を得るためには各光電素子列の像情報を一度記憶装置に
記憶し、その後該記憶装置より情報の読み出しを行って
前記境界点Ya′,Yb′を検出するという過程をふまなく
てはならないため、大容量の記憶装置が必要であるとい
う問題点があった。Generally, the reading of the photoelectric element array 6 is performed line-sequentially.
In order to obtain information in the direction (Y-axis direction) orthogonal to the column direction of the photoelectric element rows, the image information of each photoelectric element row is stored once in a storage device, and then the information is read out from the storage device to obtain the boundary information. Since the process of detecting the points Ya 'and Yb' must be taken into account, there is a problem that a large-capacity storage device is required.
そこで第10図（Ａ）に示すように瞳孔24を含む２つの
光電素子列２列Y1′,Y2′を順次選択し該光電素子列か
ら検出される虹彩23と瞳孔24との４つの境界点Z1a′,Z1
b′,Z2a′,Z2b′の内の３点を用いて瞳孔24の中心Ｃ′
の位置を算出しそれと不図示の角膜反射像の位置より観
察者の視線を算出する方法が考えられる。Therefore, as shown in FIG. 10 (A), two photoelectric element rows Y1 'and Y2' including the pupil 24 are sequentially selected, and four boundary points between the iris 23 and the pupil 24 detected from the photoelectric element rows are selected. Z1a ′, Z1
b ′, Z2a ′, Z2b ′ using the three points, the center C ′ of the pupil 24
A method of calculating the position of the observer and calculating the line of sight of the observer from the calculated position and the position of the corneal reflection image (not shown) can be considered.
しかしながら第８図（Ａ）に示すように被検体である
虹彩23はコンデンサーレンズの役割を果たす角膜21を介
して光電素子列６上に結像するため、光電素子列６上で
得られる虹彩23の位置は実際の虹彩の位置とは異なる見
かけの位置となる。そのため複数の光電素子列より２つ
の光電素子列Y1′,Y2′を任意に選択すると眼球の瞳孔
中心Ｃ′に対して上下どちらか（＋Ｙもしくは−Ｙ方
向）に偏った２列を選択してしまう場合がある。この場
合、算出される垂直方向（Ｘ−Ｙ平面内）の眼球の回転
角は実際の回転角に対して一部線形ではなくなり検出誤
差が生ずるという問題点があた。However, as shown in FIG. 8A, the iris 23, which is the subject, forms an image on the photoelectric element array 6 via the cornea 21 serving as a condenser lens. Is an apparent position different from the actual position of the iris. Therefore, when two photoelectric element arrays Y1 'and Y2' are arbitrarily selected from a plurality of photoelectric element arrays, two rows deviated upward or downward (+ Y or -Y direction) with respect to the pupil center C 'of the eyeball are selected. In some cases. In this case, there is a problem that the calculated rotation angle of the eyeball in the vertical direction (within the XY plane) is not linear with respect to the actual rotation angle and a detection error occurs.
また一般に観察者の眼球に入射する光量の変化に対す
る瞳孔径の変化及び個人差による瞳孔径の違いを考慮し
て、光電素子列Y1′と光電素子列Y2′との間隔は瞳孔径
の最小値の2/3程度に小さく初期設定される。そこで第1
0図（Ｂ）に示すように眼球の瞳孔中心Ｃ′に対して上
下（＋Ｙ及び−Ｙ方向）に分かれた２つの光電素子列Y
1′,Y2′を選択しても観察者の瞳孔径が大きくまた各光
電素子列Y1′,Y2′と瞳孔中心Ｃ′との間隔が非対称で
ある場合は、第11図に示すように算出される垂直方向
（Ｘ−Ｙ平面内）の眼球の回転角は実際の回転角に対し
て一部線形ではなくなり検出誤差が生ずるという問題点
があった。In general, in consideration of the change in pupil diameter with respect to the change in the amount of light incident on the observer's eyeball and the difference in pupil diameter due to individual differences, the interval between the photoelectric element rows Y1 'and Y2' is the minimum value of the pupil diameter. Initially set to about 2/3 of the initial value. So the first
0 As shown in FIG. 2B, two photoelectric element arrays Y divided vertically (+ Y and -Y directions) with respect to the pupil center C 'of the eyeball.
If the observer's pupil diameter is large even if 1 'and Y2' are selected and the distance between each photoelectric element array Y1 'and Y2' and the pupil center C 'is asymmetric, the calculation is performed as shown in FIG. However, the rotation angle of the eyeball in the vertical direction (within the XY plane) is not linear with respect to the actual rotation angle, and a detection error occurs.
本発明は受光手段を構成する複数の光電素子列のうち
から所定の２つの光電素子列を適切に選択し、該選択し
た２つの光電素子列より角膜反射像と虹彩反射像に基づ
く情報を得ることにより視線を高精度に算出することの
できる視線検出装置の提供を目的とする。According to the present invention, predetermined two photoelectric element arrays are appropriately selected from a plurality of photoelectric element arrays constituting a light receiving unit, and information based on a corneal reflection image and an iris reflection image is obtained from the selected two photoelectric element arrays. Accordingly, an object of the present invention is to provide a gaze detection device capable of calculating a gaze with high accuracy.
（問題点を解決するための手段） 本発明の視線検出装置は、 （１−１）眼球を照明手段により照明し、前記照明手段
によって照明されることで生じる角膜反射像と、虹彩と
瞳孔の境界を複数の光電変換素子列からなる受光手段に
て受光して、前記受光手段の所定面上における前記角膜
反射像と前記境界の位置から視線を演算手段により演算
する視線検出装置において、 前記演算手段は、前記複数の光電変換素子列のなかか
ら前記角膜反射像を受光する第１の光電変換素子列を選
出するとともに、前記複数の光電変換素子列のなかから
前記第１の光電変換素子列から所定の間隔にある第２の
光源変換素子列を選出し、前記第１の光電変換素子列か
ら検出される前記角膜反射像の位置と前記境界の位置お
よび前記第２の光電変換素子列から検出される前記境界
の位置から視線を演算することを特徴としている。(Means for Solving the Problems) The gaze detection device of the present invention comprises: (1-1) illuminating an eyeball with an illuminating unit, and a corneal reflection image generated by being illuminated by the illuminating unit; A visual axis detection device that receives a boundary by a light receiving unit including a plurality of photoelectric conversion element arrays and calculates a line of sight by a calculation unit from the corneal reflection image on a predetermined surface of the light receiving unit and the position of the boundary; The means selects a first photoelectric conversion element array that receives the corneal reflection image from the plurality of photoelectric conversion element arrays, and selects the first photoelectric conversion element array from the plurality of photoelectric conversion element arrays. And a second light source conversion element array at a predetermined interval is selected from the position of the corneal reflection image detected from the first photoelectric conversion element array, the boundary position, and the second photoelectric conversion element array. detection It is characterized by calculating the line of sight from the position of the boundaries.
特に、 （１−１−１）前記照明手段は一対の発光素子で前記眼
球を照明し、前記演算手段は前記受光手段の所定面上に
おける一対の角膜反射像の位置の間隔から、前記演算手
段が前記第１の光電変換素子列から前記第２の光電変換
素子列を選出する際の前記所定の間隔を変化させること
を特徴としている。In particular, (1-1-1) the illuminating means illuminates the eyeball with a pair of light emitting elements, and the calculating means calculates the calculating means based on an interval between a pair of corneal reflection images on a predetermined surface of the light receiving means. Is characterized in that the predetermined interval when selecting the second photoelectric conversion element row from the first photoelectric conversion element row is changed.
（実施例） 第１図（Ａ）は本発明を一眼レフカメラに適用したと
きの一実施例の光学系の要部概略図、同図（Ｂ）は同図
（Ａ）の一部分の説明図である。(Embodiment) FIG. 1 (A) is a schematic view of a main part of an optical system according to an embodiment when the present invention is applied to a single-lens reflex camera, and FIG. 1 (B) is an explanatory view of a part of FIG. It is.
図中、１は接眼レンズで、その内部には可視光透過・
赤外光反射のダイクロイックミラー1aが斜設されてお
り、光路分割器を兼ねている。In the figure, reference numeral 1 denotes an eyepiece, which transmits visible light.
A dichroic mirror 1a that reflects infrared light is obliquely provided and also functions as an optical path splitter.
４は受光レンズ、５（5a,5b,5c）は照明手段であり、
例えば発光ダイオードから成っている。６は光電素子列
（光電変換素子列）である。受光レンズ４と光電素子列
６は受光手段の一要素を構成している。光電素子列６は
通常は、図面垂直方向に１次元的に複数の光電素子が並
んだデバイスを使うが、必要に応じて２次元に光電素子
が並んだデバイスを使用する。各要素1,4,5,6より眼球
の視線検出系を構成している。4 is a light receiving lens, and 5 (5a, 5b, 5c) are illumination means,
For example, it consists of a light emitting diode. Reference numeral 6 denotes a photoelectric element row (photoelectric conversion element row). The light receiving lens 4 and the photoelectric element array 6 constitute one element of the light receiving means. The photoelectric element array 6 normally uses a device in which a plurality of photoelectric elements are arranged one-dimensionally in the direction perpendicular to the drawing, but uses a device in which the photoelectric elements are arranged two-dimensionally as necessary. The elements 1, 4, 5, and 6 constitute an eye gaze detection system.
101は撮影レンズ、102はクイックリターン（QR）ミラ
ー、103は表示素子、104はピント板、105はコンデンサ
ーレンズ、106はペンタダハプリズム、107はサブミラ
ー、108は多点焦点検出装置であり、撮影画面内の複数
の領域を選択して焦点検出を行っている。101 is a photographing lens, 102 is a quick return (QR) mirror, 103 is a display element, 104 is a focus plate, 105 is a condenser lens, 106 is a penta roof prism, 107 is a submirror, 108 is a multipoint focus detection device, and a photographing screen Are selected to perform focus detection.
多点焦点検出装置の説明は本発明理解のために必要な
いため概略に止める。The description of the multi-point focus detection device is not necessary for understanding the present invention, and thus will be briefly described.
即ち本実施例では第１図（Ｂ）に描く様に撮影レンズ
101の予定結像面近傍に配され、夫々測距域を決める複
数のスリットを有する視野マスク110と各スリット内の
像に対してフィールドレンズの作用を果たすレンズ部材
111を近接配置し、更にスリット数に応じた再結像レン
ズの組112と光電素子列の組113を順置する。スリット11
0、フィールドレンズ111、再結像レンズの組112、そし
て光電素子列の組113はそれぞれ周知の焦点検出系を構
成している。That is, in this embodiment, as shown in FIG.
A field mask 110 having a plurality of slits, each of which is disposed in the vicinity of a predetermined imaging plane 101 and has a plurality of slits for determining a distance measurement area, and a lens member which functions as a field lens for an image in each slit.
111 are arranged close to each other, and a set 112 of re-imaging lenses and a set 113 of photoelectric element arrays are arranged in order according to the number of slits. Slit 11
The set 0, the field lens 111, the set 112 of the re-imaging lens, and the set 113 of the photoelectric element array respectively constitute a well-known focus detection system.
本実施例では撮影レンズ101の透過した被写体光の一
部はQRミラー102によって反射してピント板104近傍に被
写体像を結像する。ピント板104の拡散面で拡散した被
写体光はコンデンサーレンズ105、ペンタダハプリズム1
06、接眼レンズ１を介してアイポイントＥに導かれる。In the present embodiment, a part of the subject light transmitted through the photographing lens 101 is reflected by the QR mirror 102 to form a subject image near the focus plate 104. The subject light diffused by the diffusing surface of the focus plate 104 is reflected by the condenser lens 105 and the penta roof prism 1
06, guided to the eye point E via the eyepiece 1.
ここで表示素子103は例えば偏光板を用いない２層タ
イプのゲスト−ホスト型液晶素子で、ファインダー視野
内の測距域（焦点検出位置）を表示するものである。Here, the display element 103 is, for example, a two-layer type guest-host type liquid crystal element that does not use a polarizing plate, and displays a distance measurement area (focus detection position) within a finder field.
又、撮影レンズ101を透過した被写体光の一部は、QR
ミラー102を透過し、サブミラー107で反射してカメラ本
体底部に配置された前述の多点焦点検出装置108に導か
れる。さらに多点焦点検出装置108の選択した被写体面
上の位置の焦点検出情報に基づいて、不図示の撮影レン
ズ駆動装置により撮影レンズ101の繰り出し（もしくは
繰り込み）が行なわれ、焦点調節が行なわれる。In addition, part of the subject light transmitted through the taking lens 101 is QR
The light passes through the mirror 102, is reflected by the sub-mirror 107, and is guided to the above-mentioned multipoint focus detection device 108 arranged at the bottom of the camera body. Further, based on the focus detection information of the position on the object plane selected by the multipoint focus detection device 108, the taking lens 101 is extended (or retracted) by a not-shown taking lens driving device, and the focus is adjusted.
本実施例に係る視線検出装置は、符番1,4,5,6で表わ
された部材より構成された視線検出系と、演算手段であ
る信号処理回路109に含まれる眼球光軸検出回路、眼球
判別回路、視軸補正回路、注視点検出回路等から構成さ
れている。The eye-gaze detecting device according to the present embodiment includes an eye-gaze detecting system including members denoted by reference numerals 1, 4, 5, and 6, and an eyeball optical axis detecting circuit included in the signal processing circuit 109 that is an arithmetic unit. , An eyeball discrimination circuit, a visual axis correction circuit, a gazing point detection circuit, and the like.
該視線検出系において、赤外発光ダイオード５から放
射される赤外光は、図中上方から接眼レンズ１に入射し
ダイクロイックミラー1aにより反射されアイポイントＥ
近傍に位置する観察者の眼球201を照明す。また眼球201
で反射した赤外光は、ダイクロイックミラー1aで反射さ
れ受光レンズ４によって収斂しながら光電素子列６上に
像を形成する。また、前記信号処理回路109はマイクロ
コンピュータのソフトで実行される。In the visual line detection system, infrared light emitted from the infrared light emitting diode 5 enters the eyepiece 1 from above in the figure, is reflected by the dichroic mirror 1a, and is reflected by the eye point E.
The observer's eyeball 201 located nearby is illuminated. Also eyeball 201
Is reflected by the dichroic mirror 1a and converged by the light receiving lens 4 to form an image on the photoelectric element array 6. Further, the signal processing circuit 109 is executed by software of a microcomputer.
注視点検出回路において検知された注視点情報は、ま
ず表示素子103と多点焦点検出装置108に伝送される。表
示素子103においては観察者が注視した場所をカメラの
ファインダー内に表示し、注視点（焦点検出点）の確認
を行う役割を果たす。The gazing point information detected by the gazing point detection circuit is first transmitted to the display element 103 and the multipoint focus detection device 108. The display element 103 plays a role of displaying the place watched by the observer in the viewfinder of the camera and confirming the gazing point (focus detection point).
また多点焦点検出装置108においては、観察者が注視
した点の焦点検出が行なわれ注視被写体に対して焦点調
節が行なわれる。In addition, in the multipoint focus detection device 108, focus detection is performed on a point watched by the observer, and focus adjustment is performed on the watched subject.
第２図は第１図の視線検出系の要部斜視図、第３図
（Ａ），（Ｂ）は視線検出系の光学原理図である。照明
用の赤外発光ダイオード5a,5b,5cはカメラと観察者の眼
球との距離を検出するために２個一組で使用され、カメ
ラの姿勢に応じて赤外発光ダイオード5a,5bで横位置、
赤外発光ダイオード5b,5cで縦位置の検出を行ってい
る。尚、同図においてカメラの姿勢検知手段は図示され
ていないが水銀スイッチ等を利用した姿勢検知手段が有
効である。FIG. 2 is a perspective view of a main part of the visual axis detection system of FIG. 1, and FIGS. 3A and 3B are optical principle diagrams of the visual axis detection system. The infrared light emitting diodes 5a, 5b, and 5c for illumination are used in pairs to detect the distance between the camera and the observer's eyeball. position,
The vertical position is detected by the infrared light emitting diodes 5b and 5c. Although the attitude detecting means of the camera is not shown in the figure, an attitude detecting means using a mercury switch or the like is effective.
赤外発光ダイオード5a,5bは受光レンズ４の光軸（Ｘ
軸）に対して光電素子列６の列方向（Ｚ軸方向）及び該
列方向と直交する方向にシフトした位置に配置されてい
る。The infrared light emitting diodes 5a and 5b are connected to the optical axis (X
It is arranged at a position shifted in the column direction (Z-axis direction) of the photoelectric element column 6 with respect to the axis) and in a direction orthogonal to the column direction.
第３図（Ａ）において光電素子列６の列方向（Ｚ軸方
向）に分離して配置された赤外発光ダイオード5a,5bか
らの光束はＺ軸方向に分離した位置に角膜反射像e,dを
それぞれ形成する。この時、角膜反射像ｅ及びｄの中点
のＺ座標は角膜21の曲率中心ｏのＺ座標と一致してい
る。また角膜反射像ｅ及びｄの間隔は赤外発光ダイオー
ドと観察者の眼球との距離に対応して変化するため、光
電素子列６上に再結像した角膜反射像の位置ｅ′,d′を
検出することにより眼球からの反射像の結像倍率βを求
めることが可能となる。また第３図（Ｂ）において光電
素子列６の列方向と直交する方向に配置された赤外発光
ダイオード5a（5b）は観察者の眼球を斜め上から照明す
ることになり、そのため観察斜の眼球が垂直方向（Ｘ−
Ｙ平面内）に回転していない場合は角膜反射像ｅ（ｄ）
は角膜の曲率中心及び瞳孔の中心よりも図中＋Ｙ方向に
形成される。In FIG. 3 (A), the luminous fluxes from the infrared light emitting diodes 5a and 5b arranged separately in the column direction (Z-axis direction) of the photoelectric element array 6 are located at positions separated in the Z-axis direction. d is formed respectively. At this time, the Z coordinate of the middle point of the corneal reflection images e and d matches the Z coordinate of the center of curvature o of the cornea 21. Further, since the distance between the corneal reflection images e and d changes in accordance with the distance between the infrared light emitting diode and the eyeball of the observer, the positions e ′ and d ′ of the corneal reflection images re-imaged on the photoelectric element array 6. , It is possible to determine the imaging magnification β of the reflected image from the eyeball. In addition, in FIG. 3 (B), the infrared light emitting diodes 5a (5b) arranged in a direction orthogonal to the row direction of the photoelectric element rows 6 illuminate the observer's eyeball from obliquely above. The eyeball is in the vertical direction (X-
When the image is not rotated (in the Y plane), the corneal reflection image e (d)
Are formed in the + Y direction in the figure with respect to the center of curvature of the cornea and the center of the pupil.
第４図（Ａ）は本実施例において光電素子列６の複数
の光電素子列面上に投影された眼球からの反射像を示す
説明図である。FIG. 4A is an explanatory diagram showing a reflection image from an eyeball projected on a plurality of photoelectric element row surfaces of the photoelectric element row 6 in the present embodiment.
第４図（Ａ）は光電素子列６上に投影された眼球から
の反射像を示したものである。同図において角膜反射像
ｅ′,d′は光電素子列（第１の光電変換素子列）Yp′上
に再結像している。このとき光電素子列Yp′より得られ
る出力信号を第４図（Ｂ）に示す。FIG. 4A shows a reflected image from the eyeball projected on the photoelectric element array 6. In the figure, the corneal reflection images e 'and d' are re-imaged on the photoelectric element array (first photoelectric conversion element array) Yp '. FIG. 4B shows an output signal obtained from the photoelectric element array Yp 'at this time.
次に本実施例における視線検出方法を第５図のフロー
チャートを用いて順次説明する。Next, the line-of-sight detection method in the present embodiment will be sequentially described with reference to the flowchart of FIG.
まず信号処理回路109に含まれる眼球光軸検出回路に
おいて眼球光軸の回転角が検出される。次いで光電素子
列６の像信号の読み出しを第４図（Ａ）で示す−Ｙ方向
より順次行い角膜反射像ｅ′,d′が形成される光電素子
列（ライン）Yp′を検出する（＃１）。同時に角膜反射
像ｅ′,d′の列方向の発生位置Zd′,Ze′を検出し（＃
２）、該角膜反射像の間隔|Zd′−Ze′｜より光学系の
結像倍率βを求める（＃３）。さらに該光電素子列（ラ
イン）Yp′上に虹彩23と瞳孔24との境界点Z2b′,Z2a′
を検出し（＃４）、該光電素子列Yp′上の瞳孔長|Z2a′
−Z2b′｜を算出する（＃５）。First, an eyeball optical axis detection circuit included in the signal processing circuit 109 detects the rotation angle of the eyeball optical axis. Next, the image signals of the photoelectric element array 6 are sequentially read from the -Y direction shown in FIG. 4A, and the photoelectric element array (line) Yp 'on which the corneal reflection images e' and d 'are formed is detected (# 1). At the same time, the generation positions Zd ', Ze' of the corneal reflection images e ', d' in the column direction are detected (#
2), the imaging magnification β of the optical system is determined from the interval | Zd'-Ze '| of the corneal reflection image (# 3). Further, the boundary points Z2b 'and Z2a' between the iris 23 and the pupil 24 on the photoelectric element row (line) Yp '.
(# 4), and the pupil length | Z2a 'on the photoelectric element row Yp'
-Z2b '| is calculated (# 5).
第４図（Ａ）に示すように通常、角膜反射像が形成さ
れる光電素子列Yp′は瞳孔中心Ｃ′の存在する光電素子
列Y0′より図中−Ｙ方向に発生し、像信号の読み出しを
行うべきもう一つの光電素子列（第２の光電変換素子
列）Y1′は前記結像倍率β及び瞳孔長の値より算出され
る（＃６）。この時該光電素子列Y1′は光電素子列Yp′
に対して十分な間隔を有するように設定される。同様に
光電素子列Y1′上の虹彩23と瞳孔24との境界点Z1a′,Z1
b′が検出されると（＃７）、該境界点（Z1a′,Y
1′）、（Z1b′,Y1′）及び前記境界点（Z2a′,Y
2′）、（Z2b′,Y2′）の内の少なくとも３点を用いて
瞳孔の中心位置Ｃ′（Zc′,Yc′）が求められる。さら
に前記角膜反射像の位置（Zd′,Yp′），（Ze′,Yp′）
を用いて前記（２）式を変形すると眼球光軸の回転角θ
z,θｙは β＊▲▼＊SINθｙ≒Yc′−Yp′＋δＹ′ ……
（４） を満足する（＃８）。但しδＹ′は赤外発光ダイオード
が受光レンズ４に対して光電素子列６の列方向と直交す
る方向に配置されていることにより、角膜反射像の再結
像位置ｅ′,d′が光電素子列６上で角膜21の曲率中心の
Ｙ座標に対してＹ軸方向にシフトしている分を補正する
値である。As shown in FIG. 4 (A), the photoelectric element array Yp 'on which a corneal reflection image is formed is normally generated in the -Y direction in the figure from the photoelectric element array Y0' where the pupil center C 'is present. Another photoelectric element array (second photoelectric conversion element array) Y1 'to be read out is calculated from the values of the imaging magnification β and the pupil length (# 6). At this time, the photoelectric element row Y1 'is
Is set to have a sufficient interval with respect to. Similarly, the boundary points Z1a ', Z1 between the iris 23 and the pupil 24 on the photoelectric element row Y1'
When b 'is detected (# 7), the boundary point (Z1a', Y
1 '), (Z1b', Y1 ') and the boundary point (Z2a', Y
2 ') and (Z2b', Y2 '), the pupil center position C' (Zc ', Yc') is obtained using at least three points. Further, the positions (Zd ', Yp') and (Ze ', Yp') of the corneal reflection image
When the above equation (2) is transformed by using
z and θy are β * ▲ ▼ * SINθy ≒ Yc'-Yp '+ δY' ...
(4) is satisfied (# 8). However, δY ′ is such that the infrared light emitting diode is arranged in the direction orthogonal to the row direction of the photoelectric element array 6 with respect to the light receiving lens 4, and the re-imaging positions e ′ and d ′ of the corneal reflection image are This is a value for correcting the shift in the Y-axis direction with respect to the Y coordinate of the center of curvature of the cornea 21 on the column 6.
さらに、信号処理回路109に含まれる眼球判別回路に
おいては、例えば算出される眼球光軸の回転角の分布よ
りファインダーをのぞいている観察者の目が右目から左
目かを判別し（＃９）、さらに視軸補正回路において該
眼球判別情報と前記眼球光軸の回転角に基づいて視軸の
補正が行われる（＃10）。また注視点検出回路において
は、ファインダー光学系の光学定数に基づいて注視点を
算出される（＃11）。Further, the eyeball discriminating circuit included in the signal processing circuit 109 discriminates whether the eye of the observer looking through the finder is from the right eye to the left eye, for example, from the calculated distribution of the rotation angle of the optical axis of the eyeball (# 9). Further, the visual axis correction circuit corrects the visual axis based on the eyeball discrimination information and the rotation angle of the eyeball optical axis (# 10). In the gazing point detection circuit, the gazing point is calculated based on the optical constant of the finder optical system (# 11).
尚、観察者によってはまぶたによって瞳孔がけられる
状態が発生する。このような場合、第10図に示したよう
な光電素子列の選択を行なわなければならないが、その
際、眼球光軸の回転角の検出誤差を小さくするような補
正を眼球光軸検出回路によって行うのが望ましい。Note that, depending on the observer, a state occurs in which the pupil is opened by the eyelid. In such a case, it is necessary to select the photoelectric element row as shown in FIG. 10, but at this time, correction such as to reduce the detection error of the rotation angle of the eyeball optical axis is performed by the eyeball optical axis detection circuit. It is desirable to do.
また本実施例において垂直方向の回転角を求める際、
角膜反射像のＹ軸方向の位置をδＹ′だけ補正して回転
角を求めているが、カメラの姿勢に関係せず照明用の赤
外発光ダイオード5a,5b,5cを常時点灯し、光電素子列６
の列方向と平行なＺ−Ｘ平面に関して対称な赤外発光ダ
イオード（5b,5c）の組を用いて該赤外発光ダイオード
による角膜反射像の中点（Ｙ座標）を求めてから回転角
を算出しても構わない。この時該赤外発光ダイオード5c
の角膜反射像の光電素子列６上への投影像の位置（Ｙ座
標）を検知する必要があるが、角膜反射像は比較的光強
度が強いため容易に検知可能である。Also, when obtaining the rotation angle in the vertical direction in this embodiment,
The rotation angle is obtained by correcting the position of the corneal reflection image in the Y-axis direction by δY ', but the infrared light emitting diodes 5a, 5b, 5c for illumination are always turned on regardless of the posture of the camera, and the photoelectric element is turned on. Row 6
Using a set of infrared light emitting diodes (5b, 5c) symmetrical with respect to a ZX plane parallel to the column direction, the midpoint (Y coordinate) of the corneal reflection image by the infrared light emitting diodes is determined, and then the rotation angle is calculated. It may be calculated. At this time, the infrared light emitting diode 5c
It is necessary to detect the position (Y coordinate) of the projected image of the corneal reflection image on the photoelectric element array 6, but the corneal reflection image can be easily detected because of its relatively high light intensity.
また観察者がカメラを縦位置に構えた場合においても
同様の方法で観察者の注視点を精度よく検知することが
可能である。Further, even when the observer holds the camera in the vertical position, the gazing point of the observer can be accurately detected by the same method.
第６図は本発明の第２実施例において光電素子列６面
上に投影された眼球からの反射像を示す説明図である。
本実施例では角膜反射像ｅ′,d′は光電素子列Yp′面上
に形成されている。FIG. 6 is an explanatory view showing a reflected image from an eyeball projected on the surface of the photoelectric element array 6 in the second embodiment of the present invention.
In this embodiment, the corneal reflection images e 'and d' are formed on the surface of the photoelectric element row Yp '.
次に本実施例における視線検出方法を第７図のフロー
チャートを用いて順次説明する。Next, the line-of-sight detection method according to the present embodiment will be sequentially described with reference to the flowchart of FIG.
まず演算手段である信号処理回路109に含まれる眼球
光軸検出回路において眼球光軸の回転角が検出される。
次いで光電素子列６の像信号の読み出しを第６図で示す
−Ｙ方向より順次行い角膜反射像ｅ′,d′が形成され
る。光電素子列（第１の光電変換素子列）（ライン）Y
p′を検出する（＃21）。同時に角膜反射像ｅ′,d′の
発生位置Zd′,Ze′を検出し（＃22）、該角膜反射像の
間隔|Zd′−Ze′｜より結像倍率βを求める（＃23）。
また観察者の眼球の垂直方向（Ｘ−Ｙ平面内）の回転が
無いとすると、光電素子列６上に発生する前記角膜反射
像と瞳孔中心Ｃ′とのＹ軸方向の間隔は前記視線検出系
と観察者の眼球との距離によって決まっているため、前
記角膜反射像の間隔より瞳孔中心Ｃ′の存在する光電素
子列（ライン）Y0′が算出される（＃24）。First, the rotation angle of the optical axis of the eyeball is detected in the eyeball optical axis detection circuit included in the signal processing circuit 109 which is the arithmetic means.
Next, the reading of the image signals of the photoelectric element array 6 is sequentially performed from the −Y direction shown in FIG. 6 to form corneal reflection images e ′ and d ′. Photoelectric element row (first photoelectric conversion element row) (line) Y
Detect p '(# 21). At the same time, the generation positions Zd ', Ze' of the corneal reflection images e ', d' are detected (# 22), and the imaging magnification β is obtained from the interval | Zd'-Ze '| of the corneal reflection images (# 23).
If there is no rotation of the observer's eyeball in the vertical direction (within the XY plane), the distance between the corneal reflection image generated on the photoelectric element array 6 and the pupil center C 'in the Y-axis direction is determined by the line-of-sight detection. Since it is determined by the distance between the system and the eyeball of the observer, the photoelectric element row (line) Y0 'where the pupil center C' exists is calculated from the interval between the corneal reflection images (# 24).
さらに観察者の瞳孔径の最小値を想定することにより
検出すべき２つの光電素子列（ライン）Y1′とY2′との
間隔が決定し、瞳孔中心Ｃ′の存在する光電素子列Y0′
に関してＹ軸方向に略対称な２つの光電素子列Y1′,Y
2′が選択される（＃25）。ここで光電素子列Y1′は第
２の光電変換素子列に相当している。Further, by assuming the minimum value of the pupil diameter of the observer, the interval between the two photoelectric element rows (lines) Y1 'and Y2' to be detected is determined, and the photoelectric element row Y0 'where the pupil center C' exists is determined.
, Two photoelectric element arrays Y1 ′, Y that are substantially symmetric in the Y-axis direction.
2 'is selected (# 25). Here, the photoelectric element row Y1 'corresponds to the second photoelectric conversion element row.
本実施例において光電素子列Y2′と光電素子列Yp′と
が一致している例を示しているが、光電素子列Y2′が光
電素子列Yp′に対して−Ｙ方向に存在する場合がある。
そのため角膜反射像ｅ′,d′が存在する光電素子列Yp′
を検出する以前に読み出した光電素子列の像情報を記憶
しておく記憶装置が必要となるがその記憶容量は小さい
ため不都合はない。In the present embodiment, an example is shown in which the photoelectric element row Y2 'and the photoelectric element row Yp' coincide with each other, but the photoelectric element row Y2 'is present in the -Y direction with respect to the photoelectric element row Yp'. is there.
Therefore, the photoelectric element array Yp 'in which the corneal reflection images e' and d 'exist
A storage device for storing the image information of the photoelectric element array read out before detecting is required, but there is no inconvenience because its storage capacity is small.
次に眼球光軸検出回路は前記光電素子列Y2′（＝Y
p′）上の虹彩23と瞳孔24との境界点Z2a′,Z2b′及び前
記光電素子列（第２の光電変換素子列）Y1′上の虹彩23
と瞳孔24との境界点Z1a′,Z1b′を検出する（＃26,＃2
7）。該境界点（Z1a′,Y1′）、（Z1b′,Y1′）及び前
記境界点（Z2a′,Y2′）、（Z2b′,Y2′）の内の少なく
とも３点を用いて瞳孔の中心位置Ｃ′（Zc′,Yc′）が
求められる。さらに前記角膜反射像の位置（Zd′,Y
p′）、（Ze′,Yp′）を用いて前記（３），（４）式よ
り眼球光軸の回転角が算出される（＃28）。Next, the eyeball optical axis detection circuit performs the photoelectric element row Y2 '(= Y
p ′), the boundary points Z2a ′ and Z2b ′ between the iris 23 and the pupil 24 and the iris 23 on the photoelectric element array (second photoelectric conversion element array) Y1 ′.
And the pupil 24 and the boundary points Z1a 'and Z1b' are detected (# 26, # 2
7). Using at least three of the boundary points (Z1a ', Y1') and (Z1b ', Y1') and the boundary points (Z2a ', Y2') and (Z2b ', Y2'), the center position of the pupil C ′ (Zc ′, Yc ′) is obtained. Further, the position of the corneal reflection image (Zd ', Y
Using (p ') and (Ze', Yp '), the rotation angle of the optical axis of the eyeball is calculated from the equations (3) and (4) (# 28).
さらに信号処理回路109に含まれる眼球判別回路にお
いては、例えば算出される眼球光軸の回転角の分布より
ファインダーをのぞいている観察者の目が右目か左目か
を判別し（＃29）、さらに視軸補正回路において該眼球
判別情報と前記眼球光軸の回転角に基づいて視軸の補正
が行なわれる（＃30）。また注視点検出回路において
は、ファインダー光学系の光学定数に基づいて注視点が
算出される（＃31）。Further, the eyeball discriminating circuit included in the signal processing circuit 109 discriminates whether the eye of the observer looking through the finder is the right eye or the left eye from the calculated distribution of the rotation angle of the optical axis of the eyeball (# 29). The visual axis correction circuit corrects the visual axis based on the eyeball discrimination information and the rotation angle of the optical axis of the eyeball (# 30). In the gazing point detection circuit, the gazing point is calculated based on the optical constant of the finder optical system (# 31).
尚、本実施例において観察者の眼球の垂直方向（Ｘ−
Ｙ平面内）の回転が無いと前提して瞳孔中心の存在する
光電素子列Y0′を求めているが、眼球の垂直方向の回転
がある場合、算出される光電素子列Y0′の位置と実際に
瞳孔中心が存在する光電素子列の位置が若干ずれるが実
用上問題はない。In the present embodiment, the vertical direction (X-
Assuming that there is no rotation (within the Y plane), the photoelectric element row Y0 'in which the center of the pupil exists is obtained. However, if there is a vertical rotation of the eyeball, the calculated position of the photoelectric element row Y0' Although the position of the photoelectric element row where the center of the pupil exists is slightly shifted, there is no practical problem.
また本実施例において垂直方向の回転角を求める際
（４）式に示すよに角膜反射像のＹ軸方向の位置をδ
Ｙ′だけ補正して回転角を求めているが、カメラの姿勢
に関係せず照明用の赤外発光ダイオード5a,5b,5cを常時
点灯し、光電素子列６の列方向と平行なＺ−Ｘ平面に関
して対称な赤外発光ダイオード（5b,5c）の組を用いて
該赤外発光ダイオードによる角膜反射像の中点（Ｙ座
標）を求めてから回転角を算出しても構わない。この
時、該赤外発光ダイオード5cの角膜反射像の光電素子列
６上への投影像の位置（Ｙ座標）を検知する必要がある
が、角膜反射像は比較的光強度が強いため容易に検知可
能である。Further, in the present embodiment, when obtaining the rotation angle in the vertical direction, the position of the corneal reflection image in the Y-axis direction is represented by δ as shown in Expression (4).
The rotation angle is obtained by correcting only Y ', but the infrared light emitting diodes 5a, 5b, 5c for illumination are always turned on regardless of the attitude of the camera, and the Z-axis parallel to the column direction of the photoelectric element array 6 is turned on. The rotation angle may be calculated after obtaining the midpoint (Y coordinate) of the corneal reflection image by the infrared light emitting diode using a set of infrared light emitting diodes (5b, 5c) symmetric with respect to the X plane. At this time, it is necessary to detect the position (Y coordinate) of the projected image of the corneal reflection image of the infrared light emitting diode 5c on the photoelectric element array 6, but the corneal reflection image has a relatively high light intensity and thus can be easily detected. Can be detected.
また観察者がカメラを縦位置に構えた場合においても
同様の方法で観察者の注視点を精度よく検知することが
可能である。Further, even when the observer holds the camera in the vertical position, the gazing point of the observer can be accurately detected by the same method.
（発明の効果） 本発明の視線検出装置は、 （１−１）眼球を照明手段により照明し、前記照明手段
によって照明されることで生じる角膜反射像と、虹彩と
瞳孔の境界を複数の光電変換素子列からなる受光手段に
て受光して、前記受光手段の所定面上における前記角膜
反射像と前記境界の位置から視線を演算手段により演算
する視線検出装置において、 前記演算手段は、前記複数の光電変換素子列のなかか
ら前記角膜反射像を受光する第１の光電変換素子列を選
出するとともに、前記複数の光電変換素子列のなかから
前記第１の光電変換素子列から所定の間隔にある第２の
光源変換素子列を選出し、前記第１の光電変換素子列か
ら検出される前記角膜反射像の位置と前記境界の位置お
よび前記第２の光電変換素子列から検出される前記境界
の位置から視線を演算することを特徴としている。(Effects of the Invention) A gaze detection device according to the present invention includes: (1-1) illuminating an eyeball with an illuminating unit, and a plurality of photoelectric conversion units for a corneal reflection image generated by being illuminated by the illuminating unit and a boundary between the iris and the pupil. A gaze detection device that receives light with a light receiving unit formed of a conversion element array and calculates a line of sight from the position of the corneal reflection image and the boundary on a predetermined surface of the light receiving unit by a calculation unit; A first photoelectric conversion element array that receives the corneal reflection image is selected from among the photoelectric conversion element arrays, and at a predetermined interval from the first photoelectric conversion element array from among the plurality of photoelectric conversion element arrays. A certain second light source conversion element array is selected, and the position of the corneal reflection image detected from the first photoelectric conversion element array, the position of the boundary, and the boundary detected from the second photoelectric conversion element array of It is characterized by calculating a laid et gaze.
特に、 （１−１−１）前記照明手段は一対の発光素子で前記眼
球を照明し、前記演算手段は前記受光手段の所定面上に
おける一対の角膜反射像の位置の間隔から、前記演算手
段が前記第１の光電変換素子列から前記第２の光電変換
素子列を選出する際の前記所定の間隔を変化させること
を特徴としている。In particular, (1-1-1) the illuminating means illuminates the eyeball with a pair of light emitting elements, and the calculating means calculates the calculating means based on an interval between a pair of corneal reflection images on a predetermined surface of the light receiving means. Is characterized in that the predetermined interval when selecting the second photoelectric conversion element row from the first photoelectric conversion element row is changed.
請求項１に記載した発明は、眼球を照明手段により照
明し、前記照明手段によって照明されることで生じる角
膜反射像と、虹彩と瞳孔の境界を複数の光電変換素子列
からなる受光手段にて受光して、前記受光手段の所定面
上における前記角膜反射像と前記境界の位置から視線を
演算手段により演算する視線検出装置において、 前記演算手段は、前記複数の光電変換素子列のなかか
ら前記角膜反射像を受光する第１の光電変換素子列を選
出するとともに、前記複数の光電変換素子列のなかから
前記第１の光電変換素子列から所定の間隔にある第２の
光源変換素子列を選出し、前記第１の光電変換素子列か
ら検出される前記角膜反射像の位置と前記境界の位置お
よび前記第２の光電変換素子列から検出される前記境界
の位置から視線を演算することによって、視線検出の精
度を向上させることができる。According to the invention described in claim 1, the eyeball is illuminated by the illuminating unit, and the corneal reflection image generated by being illuminated by the illuminating unit and the boundary between the iris and the pupil are detected by the light receiving unit including a plurality of photoelectric conversion element arrays. A gaze detection device that receives light, and calculates a line of sight from the position of the corneal reflection image and the boundary on a predetermined surface of the light receiving unit by a calculating unit, wherein the calculating unit is configured to calculate the line of sight from among the plurality of photoelectric conversion element arrays; A first photoelectric conversion element array for receiving a corneal reflection image is selected, and a second light source conversion element array at a predetermined interval from the first photoelectric conversion element array is selected from the plurality of photoelectric conversion element arrays. And calculating a line of sight from the position of the corneal reflection image detected from the first photoelectric conversion element row and the position of the boundary and the position of the boundary detected from the second photoelectric conversion element row. By doing so, the accuracy of gaze detection can be improved.
又、請求項２に記載した発明は、前記照明手段は一対
の発光素子で前記眼球を照明し、前記演算手段は前記受
光手段の所定面上における一対の角膜反射像の位置の間
隔から、前記演算手段が前記第１の光電変換素子列から
前記第２の光電変換素子列を選出する際の前記所定の間
隔を変化させることによって、視線検出の精度を向上さ
せることができる。Further, according to the invention described in claim 2, the illuminating means illuminates the eyeball with a pair of light-emitting elements, and the arithmetic means determines a distance between a pair of corneal reflection images on a predetermined surface of the light receiving means. By changing the predetermined interval when the calculating means selects the second photoelectric conversion element row from the first photoelectric conversion element row, it is possible to improve the accuracy of gaze detection.
第１図（Ａ）は本発明を一眼レフカメラに適用したとき
の第１実施例の要部概略図、第１図（Ｂ）は第１図
（Ａ）の一部分の説明図、第２図は第１図の視線検出系
の要部斜視図、第３図（Ａ），（Ｂ）は本発明の視線検
出系の光学原理説明図、第４図（Ａ），（Ｂ）は本発明
に係る光電素子列面上の眼球からの反射像と光電素子列
からの出力信号の説明図、第５図は本発明の第１実施例
のフローチャート、第６図は本発明の第２実施例に係る
光電素子列面上の眼球の反射像の説明図、第７図は本発
明の第２実施例のフローチャート、第８図（Ａ），
（Ｂ）は従来の視線検出装置の要部概略図と出力信号の
説明図、第９図、第10図（Ａ），（Ｂ）は光電素子列面
上の眼球からの反射像の説明図、第11図は眼球の回転角
の検出特性図である。 図中、１は接眼レンズ、４は受光レンズ、5a,5b,5cは照
明手段、６は光電素子列、109は演算手段、101は撮影レ
ンズ、102は跳ね上げミラー、103は表示素子、104はピ
ント板、105はコンデンサーレンズ、106はペンタダハプ
リズム、21は角膜、23は虹彩、24は瞳孔である。FIG. 1A is a schematic view of a main part of a first embodiment when the present invention is applied to a single-lens reflex camera, FIG. 1B is an explanatory view of a part of FIG. 1A, and FIG. FIG. 3 is a perspective view of a main part of the visual axis detection system of FIG. 1, FIGS. 3A and 3B are explanatory diagrams of the optical principle of the visual axis detection system of the present invention, and FIGS. 4A and 4B are the present invention. FIG. 5 is an explanatory diagram of a reflected image from an eyeball on the photoelectric element array surface and an output signal from the photoelectric element array according to the first embodiment, FIG. 5 is a flowchart of the first embodiment of the present invention, and FIG. 6 is a second embodiment of the present invention. FIG. 7 is an explanatory view of a reflection image of an eyeball on a photoelectric element array surface according to the first embodiment, FIG. 7 is a flowchart of a second embodiment of the present invention, and FIG.
9 (B) is a schematic diagram of a main part of a conventional eye gaze detecting device and an explanatory diagram of an output signal, and FIGS. 9, 10 (A) and (B) are explanatory diagrams of a reflected image from an eyeball on a photoelectric element array surface. FIG. 11 is a graph showing the detection characteristics of the rotation angle of the eyeball. In the figure, 1 is an eyepiece, 4 is a light receiving lens, 5a, 5b, 5c are illumination means, 6 is a photoelectric element array, 109 is arithmetic means, 101 is a photographing lens, 102 is a flip-up mirror, 103 is a display element, 104 Is a focusing plate, 105 is a condenser lens, 106 is a penta roof prism, 21 is a cornea, 23 is an iris, and 24 is a pupil.
フロントページの続き (58)調査した分野(Int.Cl.6，ＤＢ名) A61B 3/113 ＪＩＣＳＴファイル（ＪＯＩＳ)Continuation of the front page (58) Field surveyed (Int.Cl. 6 , DB name) A61B 3/113 JICST file (JOIS)
Claims (2)
段によって照明されることで生じる角膜反射像と、虹彩
と瞳孔の境界を複数の光電変換素子列からなる受光手段
にて受光して、前記受光手段の所定面上における前記角
膜反射像と前記境界の位置から視線を演算手段により演
算する視線検出装置において、 前記演算手段は、前記複数の光電変換素子列のなかから
前記角膜反射像を受光する第１の光電変換素子列を選出
するとともに、前記複数の光電変換素子列のなかから前
記第１の光電変換素子列から所定の間隔にある第２の光
源変換素子列を選出し、前記第１の光電変換素子列から
検出される前記角膜反射像の位置と前記境界の位置およ
び前記第２の光電変換素子列から検出される前記境界の
位置から視線を演算することを特徴とする視線検出装
置。An eyeball is illuminated by an illuminating means, and a corneal reflection image generated by being illuminated by the illuminating means and a boundary between an iris and a pupil are received by a light receiving means comprising a plurality of photoelectric conversion element arrays. In a visual line detection device that calculates a line of sight from a position of the boundary between the corneal reflection image and the boundary on a predetermined surface of the light receiving unit, the calculation unit calculates the corneal reflection image from among the plurality of photoelectric conversion element arrays. While selecting a first photoelectric conversion element row to receive light, a second light source conversion element row at a predetermined interval from the first photoelectric conversion element row is selected from the plurality of photoelectric conversion element rows, A gaze is calculated from the position of the corneal reflection image detected from the first photoelectric conversion element array and the position of the boundary, and the position of the boundary detected from the second photoelectric conversion element array. Eye gaze detection device.
を照明し、前記演算手段は前記受光手段の所定面上にお
ける一対の角膜反射像の位置の間隔から、前記演算手段
が前記第１の光電変換素子列から前記第２の光電変換素
子列を選出する際の前記所定の間隔を変化させることを
特徴とする請求項１に記載の視線検出装置。2. The illuminating means illuminates the eyeball with a pair of light emitting elements, and the calculating means determines the first corneal reflection image on the predetermined surface of the light receiving means based on an interval between the positions of the pair of corneal reflection images. The line-of-sight detection device according to claim 1, wherein the predetermined interval when the second photoelectric conversion element row is selected from the photoelectric conversion element rows is changed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP1247333A JP2803222B2 (en) | 1989-09-22 | 1989-09-22 | Eye gaze detection device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP1247333A JP2803222B2 (en) | 1989-09-22 | 1989-09-22 | Eye gaze detection device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| JPH03109029A JPH03109029A (en) | 1991-05-09 |
| JP2803222B2 true JP2803222B2 (en) | 1998-09-24 |
Family
ID=17161848
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| JP1247333A Expired - Fee Related JP2803222B2 (en) | 1989-09-22 | 1989-09-22 | Eye gaze detection device |
Country Status (1)
| Country | Link |
|---|---|
| JP (1) | JP2803222B2 (en) |
Families Citing this family (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2911686B2 (en) * | 1992-07-31 | 1999-06-23 | キヤノン株式会社 | Optical device |
| JPH0694978A (en) * | 1992-09-14 | 1994-04-08 | Nikon Corp | Device for detecting line of sight |
| JP3327651B2 (en) * | 1993-11-25 | 2002-09-24 | キヤノン株式会社 | Optical device with gaze detection function |
| US7262919B1 (en) | 1994-06-13 | 2007-08-28 | Canon Kabushiki Kaisha | Head-up display device with curved optical surface having total reflection |
| EP0687932B1 (en) | 1994-06-13 | 2005-05-25 | Canon Kabushiki Kaisha | Display device |
| JPH08136798A (en) * | 1994-11-08 | 1996-05-31 | Canon Inc | Automatic focusing camera |
| JP3309615B2 (en) * | 1994-12-21 | 2002-07-29 | キヤノン株式会社 | Image observation apparatus and binocular image observation apparatus using the same |
| JP3683934B2 (en) * | 1995-02-28 | 2005-08-17 | キヤノン株式会社 | Image observation device |
| JPH09274144A (en) | 1996-04-02 | 1997-10-21 | Canon Inc | Image display device |
| JP2000098293A (en) | 1998-06-19 | 2000-04-07 | Canon Inc | Image observing device |
| US6490095B2 (en) | 2000-03-23 | 2002-12-03 | Canon Kabushiki Kaisha | Image display apparatus |
| JP2001311904A (en) | 2000-04-28 | 2001-11-09 | Canon Inc | Device and system for image display |
| JP4387554B2 (en) * | 2000-04-28 | 2009-12-16 | キヤノン株式会社 | Image display device and image display system |
| JP4194218B2 (en) | 2000-04-28 | 2008-12-10 | キヤノン株式会社 | Image display device and image display system |
| US7019909B2 (en) | 2001-11-14 | 2006-03-28 | Canon Kabushiki Kaisha | Optical system, image display apparatus, and image taking apparatus |
| US7012756B2 (en) | 2001-11-14 | 2006-03-14 | Canon Kabushiki Kaisha | Display optical system, image display apparatus, image taking optical system, and image taking apparatus |
| JP2004233425A (en) * | 2003-01-28 | 2004-08-19 | Mitsubishi Electric Corp | Image display device |
| CN110751093B (en) * | 2019-10-20 | 2023-10-27 | 赵俊杰 | Method for automatically identifying pupil and calculating pupil diameter |
-
1989
- 1989-09-22 JP JP1247333A patent/JP2803222B2/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| JPH03109029A (en) | 1991-05-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP2803222B2 (en) | Eye gaze detection device | |
| US6014524A (en) | Camera with visual axis detecting device | |
| JP3166179B2 (en) | Eye gaze detection device | |
| JP3186072B2 (en) | Equipment with gaze detection device | |
| JP2920940B2 (en) | Equipment with gaze detection means | |
| US5608489A (en) | Eye direction detecting device | |
| JP2749818B2 (en) | Eye gaze detection device | |
| JP2939988B2 (en) | Eye gaze detection device | |
| JP2861349B2 (en) | Optical device having line-of-sight detecting means | |
| JPH06230271A (en) | Line of sight detector | |
| JP3109870B2 (en) | Video camera | |
| JP2939982B2 (en) | Eye gaze detection device | |
| JPH04347131A (en) | Visual axis detector | |
| JP2939989B2 (en) | Eye gaze detection device | |
| JP2803223B2 (en) | Eye gaze detection device | |
| JP2995878B2 (en) | Optical device having line-of-sight detection device | |
| JPH04138431A (en) | Camera having detection means for line of sight | |
| JP2794752B2 (en) | Optical device having visual axis detecting means | |
| JP3225628B2 (en) | Eye gaze detection device | |
| JP3391892B2 (en) | Eye gaze detecting device, optical device, and eye gaze detecting method | |
| JPH04236935A (en) | Optical device having visual line detector device | |
| JP2744417B2 (en) | Equipment with gaze detection device | |
| JP2744406B2 (en) | Eye gaze detection device | |
| JPH05232372A (en) | Camera provided with device for detecting line of sight | |
| JP2744407B2 (en) | Eye gaze detection device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| LAPS | Cancellation because of no payment of annual fees |