KR101745077B1 - Gaze tracker and method for tracking graze thereof - Google Patents

Gaze tracker and method for tracking graze thereof Download PDF

Info

Publication number
KR101745077B1
KR101745077B1 KR1020150049785A KR20150049785A KR101745077B1 KR 101745077 B1 KR101745077 B1 KR 101745077B1 KR 1020150049785 A KR1020150049785 A KR 1020150049785A KR 20150049785 A KR20150049785 A KR 20150049785A KR 101745077 B1 KR101745077 B1 KR 101745077B1
Authority
KR
South Korea
Prior art keywords
eye
center point
point
illumination
dimensional illumination
Prior art date
Application number
KR1020150049785A
Other languages
Korean (ko)
Other versions
KR20160120559A (en
Inventor
김양신
Original Assignee
현대자동차주식회사
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 현대자동차주식회사 filed Critical 현대자동차주식회사
Priority to KR1020150049785A priority Critical patent/KR101745077B1/en
Publication of KR20160120559A publication Critical patent/KR20160120559A/en
Application granted granted Critical
Publication of KR101745077B1 publication Critical patent/KR101745077B1/en

Links

Images

Classifications

    • G06K9/00604—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30041—Eye; Retina; Ophthalmic

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Eye Examination Apparatus (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The present invention relates to a line-of-sight tracking apparatus and a line-of-sight tracking method thereof, and more particularly, to an eye-tracking apparatus and an eye- A three-dimensional illumination reflection point reconstruction unit for reconstructing a three-dimensional illumination reflection point with the at least one two-dimensional illumination reflection point, an eye center point detection unit for removing a noise illumination reflection point from the three-dimensional illumination reflection point and detecting a final eye center point, And a line-of-sight calculating unit for calculating a line-of-sight vector using the final center-of-eye point and the pupil center-point.

Description

TECHNICAL FIELD [0001] The present invention relates to a gaze tracking device and a gaze tracking method thereof,

The present invention relates to a line-of-sight tracking apparatus and a line-of-sight tracking method for minimizing an error of an eye center point caused by an error in extracting an illumination reflection point, thereby improving the reliability of eye-

Eye tracking is applied to various fields such as monitoring the driver's condition, understanding intentions, and operating the device in the eyesight interlocked vehicle. Recently, it has been applied to monitor the dispersion of driver's attention.

In order to track the user's gaze, the conventional gaze tracking apparatus uses a vector connecting the pupil center point and the eye center point to obtain a gaze vector. It is very likely to calculate incorrect line of sight vectors due to many errors such as errors of illumination reflex points, pupil errors, and face position errors during the calculation process of finding the line of sight vector, which makes it difficult to accurately track the driver's eyes.

In particular, in the case of a line-of-sight tracking apparatus for obtaining an eye center point using illumination, an error of an eye center point occurs due to noise such as blurring of an illumination reflection point and reflection of a spectacle lens. Since the shape and brightness of the reflection point are similar, there is a high possibility that the reflection point of the illumination is erroneously extracted. If a two-dimensional illumination reflex point is erroneously extracted, the three-dimensional illumination reflex point is erroneously extracted, which causes an error of the center point of the eye, thereby making accurate eye tracking difficult.

In order to solve the above problems, the present invention provides a line-of-sight tracking apparatus and a line-of-sight tracking method for minimizing an error of an eye center point caused by an error of extraction of an illumination point, thereby improving the reliability of eye-

According to an aspect of the present invention, there is provided an eye tracking apparatus comprising: an image obtaining unit that obtains a user's eye image using two or more cameras; A three-dimensional illumination reflection point restoration unit for restoring a three-dimensional illumination reflection point by the at least one two-dimensional illumination reflection point; an eye center point detection unit for detecting a final eye center point by removing a noise illumination reflection point from the three- A pupil center detecting unit for detecting a pupil center point from the eye image, and a gaze calculating unit for calculating a gaze vector using the final eye center point and the pupil center point.

The image acquiring unit may include a first camera having an infrared ray filter, a second camera, and an illumination unit for irradiating infrared light toward the user.

Further, the eye center point is an intersection point of straight lines connecting the illumination and the third illumination reflection point.

The eye center point detection unit may calculate the distance from the eye center point to the three-dimensional illumination reflection point, and detect the three-dimensional illumination reflection point having the calculated distance as a noise illumination reflection point.

The line-of-sight calculating unit may calculate a straight line connecting the final eye center point and the pupil center point as a line-of-sight vector.

According to another aspect of the present invention, there is provided a line-of-sight tracing method for a line-of-sight tracing apparatus, comprising: acquiring a user's eye image; detecting one or more illumination points of refraction in the eye image; Detecting a center point of the eye using the eye center point and the detected illumination reflection point; detecting a noise illumination reflection point in the detected illumination reflection point using the eye center point and the detected illumination reflection point; Detecting a final eye center point, and calculating a line-of-sight vector using the final eye center point.

The eye image acquiring step may acquire eye images using a stereoscopic camera.

The eye center point detecting step detects an intersection point of straight lines connecting the illumination reflection point and the illumination.

The noise illumination reflection point detection step may include calculating a distance from the eye center point to the detected illumination reflection point and detecting the illumination reflection point with the calculated distance as the noise illumination reflection point .

The line-of-sight vector calculating step calculates a line-of-sight vector, which is a straight line connecting the final eye center point and the pupil center point.

According to the present invention, it is possible to improve the reliability of the eye tracking by minimizing the error of the center of the eye due to the error of extraction of the illumination refracting point by removing the illumination refracting point erroneously extracted from the eye image when detecting the eye center point for eye tracking.

1 is a block diagram showing the configuration of a gaze tracking apparatus according to an embodiment of the present invention;
FIGS. 2A and 2B are diagrams for explaining a normal eye center point and an eye center point with an error; FIG.
3 is a flowchart illustrating a gaze tracking method according to an embodiment of the present invention.

The terms "comprises", "comprising", "having", and the like are used herein to mean that a component can be implanted unless otherwise specifically stated, Quot; element ".

Also, the terms " part, "" module, " and" module ", as used herein, refer to a unit that processes at least one function or operation and may be implemented as hardware or software or a combination of hardware and software . It is also to be understood that the articles "a", "an", "an" and "the" Can be used.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram illustrating a configuration of a gaze tracking apparatus according to an embodiment of the present invention. FIGS. 2A and 2B are diagrams for explaining a normal eye center point and an eye center point with an error.

The gaze tracking device includes an image acquisition unit 110, an illumination reflection point detection unit 120, a three-dimensional illumination reflection point reconstruction unit 130, an eye center point detection unit 140, a pupil center point detection unit 150, and a gaze calculation unit 160 do.

The image acquiring unit 110 acquires a user's eye image using a stereoscopic camera (stereo camera). The image obtaining unit 110 includes a first camera 111, a second camera 113, and an illumination 115. The first camera 111 and the second camera 113 may be implemented with an infrared camera having an infrared filter. The illumination 115 may be realized by a lamp for emitting infrared light, a light emitting diode (LED), or the like. In addition, the illumination 115 may be composed of more than one.

The first camera 111 and the second camera 113 simultaneously photograph the same subject (eye of the user) at different positions. At this time, the illumination 115 irradiates the infrared rays toward the user's eyes.

The illumination reflection point detection unit 120 detects a two-dimensional illumination reflection point (two-dimensional illumination reflection point) generated by the illumination 115 in the eye image. At this time, the illumination reflection point detection unit 120 detects a two-dimensional illumination reflection point in the eye image photographed by the first camera 111, and detects a two-dimensional illumination reflection point in the eye image photographed by the second camera 113 . For example, when the left eye image is input from the first camera 111, the illumination reflection point detection unit 120 detects two illumination reflection points in the left eye image. The illumination reflection point detection unit 120 detects two illumination reflection points in the left eye image input from the second camera 113. [

The three-dimensional illumination reflection point restoration unit 130 restores the three-dimensional illumination reflection point (three-dimensional illumination reflection point) using two or more two-dimensional illumination reflection points detected by the illumination reflection point detection unit 120.

The eye center point detection unit 140 detects an eye center point using two or more three-dimensional illumination reflection points. The eye center point is an intersection point where straight lines connecting the detected three-dimensional illumination reflection point and the illumination 115 intersect.

The eye center point detection unit 140 determines whether there is an error in the detected eye center point. At this time, the center-of-eye-point detecting unit 140 checks the distance between the center-of-eye point and the three-dimensional illumination-reflection point to check whether an error occurs at the center-of-eye point.

When assuming that the eye is spherical, top Jung (error-free) eye center point (P EC) is illuminated reflection point P1 and the eyeball center distance and light reflection point P2 and the eyeball center point between P EC as shown in Figure 2a P EC one, the distance between the same and the failed (abnormal) eye center point P EC is the distance between the illumination reflection point P3 and the eyeball center distance and light reflection point P4 and the eyeball center point between P EC P EC, as illustrated in Figure 2b They are different.

As described above, distances from the eyeball central point to the three-dimensional illumination reflection points must be equal to each other.

The eye center point detection unit 140 calculates and compares distances between the three-dimensional illumination reflection points and the eye center point. The eye center point detection unit 140 extracts a three-dimensional illumination reflection point having a different distance from the eye center point among the three-dimensional illumination reflection points detected based on the calculated distance as a noise reflection point. The eye center point detection unit 140 removes the noise reflection points extracted from the detected three-dimensional illumination reflection points and detects (calculates) the final eye center point using the remaining three-dimensional illumination reflection points.

The pupil center-of-sight detection unit 150 detects the pupil center point by performing image processing on the eye image. At this time, the pupil center detecting unit 150 detects the eye region in the image acquired through the image obtaining unit 110, and detects the pupil center point in the detected eye region. The pupil center-of-gravity point detection unit 150 restores the three-dimensional pupil center point using the two-dimensional pupil center point detected from the eye image.

The gaze calculation unit 160 calculates a gaze vector that the user's eyes gaze by using the pupil center point and the final eye center point. That is, the line-of-sight calculating unit 160 calculates a straight line connecting the pupil center point and the final eye center point as a line-of-sight vector.

3 is a flowchart illustrating a gaze tracking method according to an exemplary embodiment of the present invention.

First, the gaze tracking device acquires a user's eye image through the image acquisition unit 110 (S101). At this time, the image acquiring unit 110 acquires a snow image through two or more cameras 111 and 113.

The gaze tracking apparatus detects one or more two-dimensional illumination reflection points in the eye image through the illumination reflection point detection unit 120 (S102). The illumination reflection point detection unit 120 detects two-dimensional illumination reflection points in each eye image input from two or more cameras 111 and 113.

The three-dimensional illumination reflection point restoration unit 130 of the gaze tracking apparatus restores the three-dimensional illumination reflection point using the two-dimensional illumination reflection points output from the illumination reflection point detection unit 120 (S105). In other words, the gaze tracking device detects a three-dimensional illumination reflex point from the eye image photographed through the stereoscopic camera.

The eye center point detecting unit 140 of the eye gaze tracking apparatus detects an eye center point for the detected three-dimensional illumination reflex point (S107).

When the eye center point is detected, the eye center point detection unit 140 calculates the distance between the three-dimensional illumination reflection point and the eye center point (S109).

The eye center point detection unit 140 recognizes an errory three-dimensional illumination reflection point among three-dimensional illumination reflection points detected based on the calculated distance as a noise reflection point, and removes the noise reflection point to detect a final eye center point (S111).

The line of sight calculation unit 160 calculates a line-of-sight vector using the final eye center point and the pupil center point (S113). The gaze calculation unit 160 receives the pupil center point output from the pupil center point detection unit 150. [ The pupil center point detection unit 150 detects the pupil center point in the eye image output from the image acquisition unit 110. [

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention.

110:
111: First camera
113: Second camera
115: Lighting
120: Illumination reflection point detector
130: Three-dimensional illumination reflection point restoration unit
140: eye center point detecting unit
150: pupil center point detection unit
160: line of sight calculation unit

Claims (10)

An image acquiring unit acquiring a user's eye image using two or more cameras;
An illumination reflex point detector for detecting one or more two-dimensional illumination reflex points in the eye image,
A three-dimensional illumination reflection point restoration unit for restoring a three-dimensional illumination reflection point with the at least one two-
An eye center point detecting unit for detecting a center point of the final eye by removing a noise illumination reflection point from the three-
A pupil center-of-gravity point detecting unit for detecting a pupil center point from the eye image,
And a line-of-sight calculating unit for calculating a line-of-sight vector using the final eye center point and the pupil center point,
Wherein the eye center point detecting unit comprises:
The center point of the eye is detected using two or more three-dimensional illumination reflex points, and the distance from the detected center point of the eye to the two or more three-dimensional illumination reflex points is calculated. If the calculated distances are different from each other, The three-dimensional illumination reflection points having different distances from the center point of the detected eye among the two or more three-dimensional illumination reflection points are extracted and removed as noise illumination reflection points, and the final eye center point is detected using the remaining three-dimensional illumination reflection points Wherein the eye tracking device comprises:
The method according to claim 1,
The image acquiring unit may acquire,
A first camera and a second camera having an infrared filter,
And a light for irradiating infrared rays toward the user.
3. The method of claim 2,
The eye center point,
Wherein the line of sight is an intersection of straight lines connecting the illumination and the third illumination reflection point.
delete The method according to claim 1,
The gaze-
And a straight line connecting the final eye center point and the pupil center point is calculated as a line-of-sight vector.
Acquiring a user's eye image;
Detecting two or more three-dimensional illumination reflex points in the eye image,
Detecting an eye center point using the at least two three-dimensional illumination reflex points;
Calculating a distance from the eyeball central point to the two or more three-dimensional illumination reflection points,
Detecting a noise illumination reflex point of the three-dimensional illumination reflex points different in distance from the detected eye center point among the two or more three-dimensional illumination reflex points, if the calculated distances are different from each other;
Removing the noise illumination reflection point from the two or more three-dimensional illumination reflection points and using the remaining three-dimensional illumination reflection points to detect a final eye center point;
And calculating a line-of-sight vector using the final eye center point.
The method according to claim 6,
The eye image acquiring step includes:
Eye camera is used to acquire an eye image using a stereoscopic camera.
The method according to claim 6,
Wherein the eye center point detecting step comprises:
And detecting an intersection point of straight lines connecting the illumination reflection point and the illumination.
delete The method according to claim 6,
The line-of-
Wherein a straight line connecting the final eye center point and the pupil center point is calculated as a line-of-sight vector.
KR1020150049785A 2015-04-08 2015-04-08 Gaze tracker and method for tracking graze thereof KR101745077B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020150049785A KR101745077B1 (en) 2015-04-08 2015-04-08 Gaze tracker and method for tracking graze thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020150049785A KR101745077B1 (en) 2015-04-08 2015-04-08 Gaze tracker and method for tracking graze thereof

Publications (2)

Publication Number Publication Date
KR20160120559A KR20160120559A (en) 2016-10-18
KR101745077B1 true KR101745077B1 (en) 2017-06-08

Family

ID=57244208

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020150049785A KR101745077B1 (en) 2015-04-08 2015-04-08 Gaze tracker and method for tracking graze thereof

Country Status (1)

Country Link
KR (1) KR101745077B1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101721214B1 (en) * 2016-11-07 2017-03-30 주식회사 큐비스 Management system for judicial affairs using electronic commerce
KR101917600B1 (en) * 2016-12-30 2018-11-13 주식회사 펀진 Pupil tracking device

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011172853A (en) * 2010-02-25 2011-09-08 Kobe Univ Device, method and program for measuring visual axis

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011172853A (en) * 2010-02-25 2011-09-08 Kobe Univ Device, method and program for measuring visual axis

Also Published As

Publication number Publication date
KR20160120559A (en) 2016-10-18

Similar Documents

Publication Publication Date Title
KR101745140B1 (en) Gaze tracker and method for tracking graze thereof
JP6083761B2 (en) Pupil detection method, corneal reflection detection method, face posture detection method, and pupil tracking method
US10698482B2 (en) Gaze tracking using non-circular lights
US7682026B2 (en) Eye location and gaze detection system and method
JP6327753B2 (en) Pupil detection light source device, pupil detection device, and pupil detection method
WO2019036751A1 (en) Enhanced video-based driver monitoring using phase detect sensors
KR102471370B1 (en) Information processing apparatus, information processing method, and program
KR101628493B1 (en) Apparatus and method for tracking gaze of glasses wearer
US11243607B2 (en) Method and system for glint/reflection identification
JP2008513168A (en) Apparatus and method for specifying a gaze direction without contact
CN104434022A (en) optical array system and method for performing pupil tracking using retro-reflectivity
JP2009254525A (en) Pupil detecting method and apparatus
CN109964230B (en) Method and apparatus for eye metric acquisition
US11675429B2 (en) Calibration, customization, and improved user experience for bionic lenses
WO2018173314A1 (en) Face recognition device
KR101745077B1 (en) Gaze tracker and method for tracking graze thereof
US10288879B1 (en) Method and system for glint/reflection identification
KR101755883B1 (en) Gaze tracker and method for distinguishing detection error of eye feature thereof
WO2019228634A1 (en) Method and system for glint/reflection identification
KR101628543B1 (en) Gaze tracking apparatus and method for measuring tracking accuracy of the same
KR20160061691A (en) Gaze Tracker and Method for Detecting Pupil thereof
CN203759720U (en) Image acquisition module
JP2006087751A (en) Line of sight measuring method and line of sight measuring apparatus
JP2023063760A (en) identification device
CN117442156A (en) Cornea reflection image determining device and sight line detecting device

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
AMND Amendment
AMND Amendment
X701 Decision to grant (after re-examination)
GRNT Written decision to grant