CN106778641A - Gaze estimation method and device - Google Patents
Gaze estimation method and device Download PDFInfo
- Publication number
- CN106778641A CN106778641A CN201611207525.8A CN201611207525A CN106778641A CN 106778641 A CN106778641 A CN 106778641A CN 201611207525 A CN201611207525 A CN 201611207525A CN 106778641 A CN106778641 A CN 106778641A
- Authority
- CN
- China
- Prior art keywords
- hot spot
- light source
- feature point
- eye image
- extracting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/19—Sensors therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/20—Drawing from basic elements, e.g. lines or circles
- G06T11/203—Drawing of straight lines or curves
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/193—Preprocessing; Feature extraction
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Ophthalmology & Optometry (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Eye Examination Apparatus (AREA)
- Image Analysis (AREA)
Abstract
It is an object of the invention to provide a kind of gaze estimation method and device, wherein method includes：Eye image is obtained, if extracting at least three hot spots in the eye image, the position according to each hot spot for extracting is fitted hot spot distribution curve, wherein, the eye image correspondence multiple light sources, each described hot spot is an imaging for the light source；The characteristic point of the hot spot distribution curve is defined as fisrt feature point, wherein, the fisrt feature point includes midpoint, the center of circle, focus, central point；Position and presupposed information according to the fisrt feature point determine the corresponding blinkpunkt information of the eye image.By gaze estimation method and device in the embodiment of the present invention, the quick estimation of blinkpunkt information can be realized.
Description
Technical field
The present invention relates to human-computer interaction technique field, in particular to a kind of gaze estimation method and device.
Background technology
Sight line estimates to refer to obtain subject currently " direction of gaze " using the various detection means such as machinery, electronics, optics
Technology.With developing rapidly for computer vision, artificial intelligence technology and digitizing technique, sight line estimation technique has turned into works as
Preceding hot research field, in field of human-computer interaction extensive application, for example, can be applied to vehicle auxiliary sail, virtual reality,
Cognitive disorder diagnosis etc..
In correlation technique, the blinkpunkt information of eyeball is generally determined using P-CR methods, P-CR methods are divided into based on recurrence
P-CR methods and two kinds of the P-CR methods based on 3D, both approaches are required to exist using multiple light sources when implementing
Multiple hot spots are formed in user eyeball, and obtains the eye image of user.
, it is necessary to be examined on the eye image of user when obtaining the blinkpunkt information of eyeball using the P-CR methods based on recurrence
The corresponding hot spot of all light sources is measured, however, when the eye movement amplitude of user is larger, the off-center meeting of eyeball is more, this
Shi Huiyou parts light source cannot form hot spot in human eye, lead to not determine the blinkpunkt information of eyeball.
Using based on 3D P-CR methods obtain eyeball blinkpunkt information when, it is necessary to know hot spot and light in eye image
The corresponding relation in source, however, when light source is more, and only part light source is imaged when forming hot spot in human eye, hot spot and light source
Between corresponding relation be difficult to estimate, lead to not determine eyeball blinkpunkt information.
The content of the invention
In view of this, it is an object of the invention to provide a kind of gaze estimation method and device, to solve above-mentioned at least one
Individual technical problem.
In a first aspect, a kind of gaze estimation method is the embodiment of the invention provides, including：Eye image is obtained, if in institute
State and extract at least three hot spots in eye image, then the position fitting hot spot distribution according to each hot spot for extracting is bent
Line, wherein, the eye image correspondence multiple light sources, each described hot spot is an imaging for the light source；By the hot spot
The characteristic point of distribution curve is defined as fisrt feature point, wherein, the fisrt feature point includes midpoint, the center of circle, focus, center
Point；Position and presupposed information according to the fisrt feature point determine the corresponding blinkpunkt information of the eye image.
With reference in a first aspect, the embodiment of the invention provides first aspect the first possible implementation method, wherein, it is described
Position and presupposed information according to the fisrt feature point determine the corresponding blinkpunkt information of the eye image, including：According to
The position of the pupil center's point in the position and the eye image of the fisrt feature point, determines that the eye image is corresponding
Blinkpunkt information；Or, the characteristic point of the figure of all light source compositions is defined as second feature point, according to described first
The position of characteristic point, the position of second feature point, the position of each hot spot of extraction, the position of each light source
Put, it is determined that the corresponding light source of each described hot spot for extracting, position and its correspondence according to each hot spot for extracting
The light source position, determine the corresponding blinkpunkt information of the eye image, wherein, during the second feature point includes
Point, the center of circle, focus, central point.
The first possible implementation method with reference to first aspect, the embodiment of the invention provides second possibility of first aspect
Implementation method, wherein, position, the position of second feature point according to the fisrt feature point, extraction each
The position of the hot spot, the position of each light source, it is determined that the corresponding light source of each described hot spot for extracting, including：
The position of position and second feature point according to the fisrt feature point, calculates the fisrt feature point and second spy
Evolution relation between levying a little；According to the evolution relation and the position of each hot spot for extracting, calculating is carried
The corresponding standard sources position of each described hot spot for taking；The light source that position is matched with the standard sources position, it is determined that
It is the corresponding light source of each described hot spot for extracting.
The first possible implementation method with reference to first aspect, the third may to the embodiment of the invention provides first aspect
Implementation method, wherein, position, the position of second feature point according to the fisrt feature point, extraction each
The position of the hot spot, the position of each light source, it is determined that the corresponding light source of each described hot spot for extracting, including：
The position of position and each hot spot for extracting according to the fisrt feature point, calculates the fisrt feature point with extraction
The first relative position between each described hot spot；The position of position and each light source according to the second feature point,
Calculate the second relative position between the second feature point and each described light source；It is determined that with the first relative position phase
The second relative position matched somebody with somebody, the light source corresponding to second relative position that will be determined is defined as each the described light for extracting
The corresponding light source of spot.
With reference in a first aspect, the embodiment of the invention provides the 4th kind of possible implementation method of first aspect, wherein, own
The light source is rounded, oval, approximate circle or approximate ellipsoidal is arranged, and the hot spot distribution curve is circle or ellipse
Circle.
With reference to the above-mentioned implementation method of first aspect, the 5th kind of possible implementation of first aspect is the embodiment of the invention provides
Mode, wherein, also include：If extracting two hot spots in the eye image, according to each the described light for extracting
The position of spot, the position of each light source, the position of human eye feature in the eye image, it is determined that each the described light for extracting
The corresponding light source of spot；According to the position and its position of the corresponding light source of each hot spot for extracting, institute is determined
The corresponding blinkpunkt information of eye image is stated, or, the characteristic point of the figure of all light source compositions is defined as second special
Levy a little, position, the position of second feature point according to the corresponding light source of each described hot spot for extracting, extract it is every
The position of the individual hot spot, determines mapping point of the second feature point in the eye image, according to the mapping point
The position of the pupil center's point in position and the eye image, determines the corresponding blinkpunkt information of the eye image, wherein,
The second feature point includes midpoint, the center of circle, focus, central point.
With reference to the 5th kind of possible implementation method of first aspect, the 6th kind of possibility of first aspect is the embodiment of the invention provides
Implementation method, wherein, it is described according to the position of each hot spot, the position of each light source extracted, the human eye
The position of human eye feature in image, it is determined that the corresponding light source of each described hot spot for extracting, including：According to each extracted
The position of the hot spot calculates third phase between each the described hot spot for extracting to position, according to the position of each light source
Calculate the 4th relative position between each described light source；Position and the human eye feature according to each hot spot for extracting
Position, calculate extract each described hot spot and the human eye feature between the 5th relative position, according to each light
The position in source and the position of the human eye feature, calculate the 6th relative position between each described light source and the human eye feature
Put；Determine that the 4th relative position matches with the third phase to position, and the 6th relative position and the described 5th
The light source that relative position matches, is the corresponding light source of each described hot spot for extracting.
With reference to the above-mentioned implementation method of first aspect, the 7th kind of possible implementation of first aspect is the embodiment of the invention provides
Mode, wherein, also include：If extracting a hot spot in the eye image, each described light source is controlled one by one
Light, determine the corresponding light source of the hot spot；The characteristic point of the figure of all light source compositions is defined as second special
Levy a little, position, the position of second feature point, the light of extraction according to the corresponding light source of the hot spot for extracting
The position of spot, determines mapping point of the second feature point in the eye image；Position and institute according to the mapping point
The position of the pupil center's point in eye image is stated, the corresponding blinkpunkt information of the eye image is determined, wherein, described second
Characteristic point includes midpoint, the center of circle, focus, central point.
Second aspect, the embodiment of the invention provides a kind of line-of-sight estimation device, including：Curve determining module, for obtaining
Eye image is taken, if extracting at least three hot spots in the eye image, according to the position of each hot spot for extracting
Fitting hot spot distribution curve is put, wherein, the eye image correspondence multiple light sources, each described hot spot is a light source
Imaging；Characteristic point determining module, for the characteristic point of the hot spot distribution curve to be defined as into fisrt feature point, wherein, it is described
Fisrt feature point includes midpoint, the center of circle, focus, central point；Information determination module, for the position according to the fisrt feature point
Determine the corresponding blinkpunkt information of the eye image with presupposed information.
With reference to second aspect, second aspect the first possible implementation method is the embodiment of the invention provides, wherein, it is described
Information determination module includes：First determination sub-module, in the position and the eye image according to the fisrt feature point
Pupil center's point position, determine the corresponding blinkpunkt information of the eye image；Or, the second determination sub-module is used for
The characteristic point of the figure of all light source compositions is defined as second feature point, according to position, the institute of the fisrt feature point
State the position of second feature point, the position of each hot spot of extraction, the position of each light source, it is determined that each extracted
The corresponding light source of the hot spot, according to the position and its position of the corresponding light source of each hot spot for extracting,
Determine the corresponding blinkpunkt information of the eye image, wherein, the second feature point includes midpoint, the center of circle, focus, center
Point.
In the embodiment of the present invention, if extracting at least three hot spots in eye image, according to each hot spot for extracting
Position fitting hot spot distribution curve, wherein, eye image correspondence multiple light sources, each hot spot is an imaging for light source, will
The characteristic point of hot spot distribution curve is defined as fisrt feature point, wherein, fisrt feature point includes midpoint, the center of circle, focus, center
Point, position and presupposed information according to fisrt feature point determine the corresponding blinkpunkt information of eye image.Implemented by the present invention
Gaze estimation method and device in example, are imaged in user eyeball without whole light sources and form hot spot, without estimation light
Corresponding relation between spot and light source, when only light source such as three light sources in part form hot spot, it becomes possible to determine eye image
Corresponding blinkpunkt information, so as to realize the quick estimation of blinkpunkt.
To enable the above objects, features and advantages of the present invention to become apparent, preferred embodiment cited below particularly, and coordinate
Appended accompanying drawing, is described in detail below.
Brief description of the drawings
Technical scheme in order to illustrate more clearly the embodiments of the present invention, below will be attached to what is used needed for embodiment
Figure is briefly described, it will be appreciated that the following drawings illustrate only certain embodiments of the present invention, thus be not construed as it is right
The restriction of scope, for those of ordinary skill in the art, on the premise of not paying creative work, can also be according to this
A little accompanying drawings obtain other related accompanying drawings.
Fig. 1 is the top view of eyeball tracking module provided in an embodiment of the present invention；
Fig. 2 is the side view of eyeball tracking module provided in an embodiment of the present invention；
The schematic flow sheet of the gaze estimation method that Fig. 3 is provided for inventive embodiments；
Fig. 4 is the module composition schematic diagram of line-of-sight estimation device provided in an embodiment of the present invention.
Reference：
10- eyeballs, 20- input paths, 21- reflected light paths, 22- transmitted light paths, 31- light sources, 30- eyepiece stalks, 40- mesh
Mirror, 50- circuit boards, 60- image collecting devices, 70- infrared fileters, 80- cutoff filters, 90- display screens.
Specific embodiment
To make the purpose, technical scheme and advantage of the embodiment of the present invention clearer, below in conjunction with the embodiment of the present invention
Middle accompanying drawing, is clearly and completely described to the technical scheme in the embodiment of the present invention, it is clear that described embodiment is only
It is a part of embodiment of the invention, rather than whole embodiments.The present invention generally described and illustrated in accompanying drawing herein is real
The component for applying example can be arranged and designed with a variety of configurations.Therefore, it is of the invention to what is provided in the accompanying drawings below
The detailed description of embodiment is not intended to limit the scope of claimed invention, but is merely representative of selected reality of the invention
Apply example.Based on embodiments of the invention, the institute that those skilled in the art are obtained on the premise of creative work is not made
There is other embodiment, belong to the scope of protection of the invention.
, it is necessary to all light sources form light in eyeball during in view of the blinkpunkt information that eyeball is obtained in correlation technique
Spot, or need to know hot spot and the corresponding relation of light source in eye image, cause blinkpunkt information to be in some cases difficult to really
It is fixed, a kind of gaze estimation method and device are the embodiment of the invention provides, it is specifically described below by embodiment.
In order to clearly describe the gaze estimation method in the embodiment of the present invention and device, the embodiment of the present invention is introduced first
In the eyeball tracking module that is related to, gaze estimation method and device in the present embodiment can be carried out using the eyeball tracking module
Sight line is estimated.
Eyeball tracking module in the present embodiment is arranged in virtual reality glasses device, by virtual reality glasses
The design of internal structure, completes the realization of eyeball tracking technology, makes user when virtual reality glasses are watched, by eyeball tracking
Technology carries out operational control to display terminal, and realizes that man-machine interaction and blinkpunkt such as render at the function.
Fig. 1 is the top view of eyeball tracking module provided in an embodiment of the present invention, as shown in figure 1, the eyeball tracking module
Including eyepiece stalk 30, at least two light sources 31 are provided with eyepiece stalk 30, light source 31 is preferably LED (Light Emitting
Diode, light emitting diode) infrared light supply, the eyeball tracking module also including circuit board 50 and image collecting device 60, adopt by image
Acquisition means 60 include image-forming objective lens and imageing sensor.
Fig. 2 is the side view of eyeball tracking module provided in an embodiment of the present invention, as shown in Fig. 2 being set in eyepiece stalk 30
There is eyepiece 40, eyepiece 40 is located at the front of eyeball 10, and light source 31 launches infrared light to eyeball 10, and the reflection infrared light of eyeball 10 comes from
The input path 20 of eyeball 10 propagates to cutoff filter 80 by eyepiece 40, is reflected to form by cutoff filter 80
Reflected light path 21, reflected light path 21 enters image collecting device 60 by infrared fileter 70, and image collecting device 60 is fixed on
On circuit board 50.Input path 20 from eyeball 10 also forms transmitted light path 22, transmitted light through cutoff filter 80
Road 22 is propagated at display screen 90, and display screen 90 is VR (Virtual Reality, virtual reality) display screen.
With reference to shown in Fig. 1 and Fig. 2, in the present embodiment, eyepiece 40, can through eyepiece 40 equivalent to a function for magnifying glass
To obtain the overall picture of eyes.Light source 31 is fixedly installed on the side front region of eyeball 10, for launching to eyeball 10
Infrared light, the reflection infrared light of eyeball 10, so as to form hot spot on eyeball 10.Image collecting device 60 is fixedly installed on eyeball and chases after
The edge or outside of visible angle in track module, cutoff filter 80 are used to for eyeball image to reflex to image collecting device
60 imageing sensor, so that imageing sensor collects eyeball image.
With reference to Fig. 2, the imaging optical path of eyeball tracking module is：
(1) light source 31 launches infrared light, the reflection infrared light of eyeball 10, so as to form hot spot on eyeball 10 to eyeball 10；
Specifically, the acquisition of hot spot is emitted beam by light source 31, beat on eyeball 10, by the different layers surface reflection line of eyeball 10,
The imageing sensor of image collecting device 60 receives the light of reflection, and hot spot can be formed in the picture.Because human eye can be felt
The wavelength known is 380nm-780nm, in order to not influence people normally to watch display terminal, the selection human eye sensitivity substantially of light source 31
The small near-infrared light source of low, harmfulness；
(2) after the hot spot formed on eyeball 10 reflects through cutoff filter 80, image collecting device 60 is imaged onto, is
The influence to image of other wavelength light is eliminated, it is necessary to add infrared fileter 70 before image collecting device 60, so that only
The light for allowing light source 31 to send enters image collecting device 60.
Further to reduce the occupancy volume of whole equipment, cutoff filter 80 can be integrated directly into display screen 90
Screen surfaces, and/or, cutoff filter 80 is the infrared reflecting coating or infrared reflecting pad pasting of display screen 90.
Cutoff filter 80 is fixedly installed on the region between eyepiece 40 and display screen 90, also, IR-cut is filtered
The outer rim of mating plate 80 is located at the outside of module visible angle, it is ensured that the hot spot that the optional position of eyeball 10 is formed can be by infrared section
Only optical filter 80 is imaged onto image collecting device 60.
To avoid display screen 90 from influenceing user's sight line, in the present embodiment, by adjusting the angle of cutoff filter 80,
The near infrared light of eyeball image is reflexed into eyepiece stalk side, is received by image collecting device 60, what display screen 90 was launched can
See that light reaches eyeball 10 by cutoff filter 80.The size of cutoff filter 80 is chosen to be needed comprising eyepiece 40
Maximum field of view's rim ray.
Eyeball tracking module as depicted in figs. 1 and 2 in actual applications, can be integrated in headset equipment, such as virtual
In Reality glasses, user after equipment starts, is once aligned first when using the headset equipment, and detailed process is：Profit
Alignment mark is shown with display screen 90, while showing position of the pupil in camera view, user's movable head-wearing formula equipment makes
Position of the pupil (or pupil center) in camera view is obtained to be in appropriate area.Wherein, camera is integrated in image and adopts
In acquisition means 60.After the completion of alignment, sight line estimation is carried out based on the eyeball tracking module in the headset equipment.
As shown in figure 3, the gaze estimation method in the embodiment of the present invention specifically includes following steps：
Step S302, obtains eye image, every according to what is extracted if extracting at least three hot spots in eye image
The position fitting hot spot distribution curve of individual hot spot, wherein, eye image correspondence multiple light sources, each hot spot be light source into
Picture.
Eye image is obtained using eyeball tracking module as depicted in figs. 1 and 2, specifically, user is worn with such as Fig. 1
With the virtual reality glasses of the eyeball tracking module shown in Fig. 2, the light source 31 in eyeball tracking module launches infrared to eyeball 10
Light, the reflection infrared light of eyeball 10, so as to form hot spot on eyeball 10, the hot spot formed on eyeball 10 is through cutoff filter
After 80 reflections, image collecting device 60 is imaged onto, so as to collect eye image by image collecting device 60.
The eye image correspondence multiple light sources collected using eyeball tracking module as depicted in figs. 1 and 2, that is, corresponded to many
Individual light source 31, and each hot spot is an imaging for light source in eye image, specially one light source is anti-on eye cornea surface
The picture penetrated projection in the picture.
In the present embodiment, as soon as image collecting device 60 often obtains frame eye image, image procossing is carried out (including pre-
Treatment, Information Statistics, parameter updates, Pupil diameter, pupil edge positioning, pupil center's positioning, spot location, hot spot screening
Deng), to extract the hot spot in eye image, if extracting at least three hot spots in eye image, such as extract three, four
Deng hot spot, then the position fitting hot spot distribution curve according to each hot spot for extracting, such as, and when three hot spots are extracted,
Using three position fitting circles of hot spot, the circle is hot spot distribution curve, when five hot spots are extracted, using five hot spots
Position fitted ellipse, the ellipse be hot spot distribution curve, certainly, in other embodiments, the quantity of hot spot is not limited to three
It is individual or five, be more than or equal to three as long as meeting, the shape of the hot spot distribution curve that fitting is obtained be not limited to circle or
Person is oval, depending on specific fitting result.
It should be noted that due to the imaging that hot spot is each light source, therefore the shape of hot spot distribution curve and all light
The setting shape correspondence in source, hot spot distribution curve is theoretically the affine transformation of the setting shape of all light sources, mathematically, imitates
Penetrate conversion and straight line is changed into straight line, circle is changed into oval, and ellipse is changed into oval, and trapezoidal/rectangular/square is all changed into any four side
Shape, but under some constraintss, when the setting of all light sources is shaped as justifying, the major axis of hot spot distribution curve (ellipse)
It is closer to short axle, so can also be approximate with circle.Therefore in the present embodiment, when all light sources are rounded or approximate circle
During setting, the shape of hot spot distribution curve can be circular or oval, when all light source ovalizes or sub-elliptical
When shape is set, hot spot distribution curve is shaped as ellipse, when shaped formations such as all light sources are trapezoidal, rectangle, squares,
Hot spot distribution curve corresponds to arbitrary quadrilateral, and when all light sources are arranged in line segment, hot spot distribution curve corresponds to line segment, when
So, the shape of the arrangement mode of all light sources and hot spot distribution curve can also have other situations, according to radiation mathematically
Shift theory, repeats no more here.
In a kind of specific embodiment, the corresponding all light sources of eye image are rounded, oval, approximate circle or near
Like all light sources 31 in the eyeball tracking module in oval shaped arrangements, namely Fig. 1 Fig. 2 it is rounded, oval, approximate circle or
Person's approximate ellipsoidal is arranged, and, hot spot distribution curve is circle or oval.
As depicted in figs. 1 and 2, when the structure precision of eyeball tracking module is higher, all light sources 31 are preferably provided with circle
Shape or oval shaped arrangements, it is contemplated that during practical application scene, it is also possible to which all light sources 31 are set in approximate circle or approximate
Oval shaped arrangements, the approximate circle distance for referring to all light source distance centers of circle in the first predeterminable range threshold range, such as 5
In millimeter to 6 millimeters of distance range, the difference of the approximate circle distance for referring to the same focus of all light source distances is second
In predeterminable range threshold range, such as in 1 millimeter to 3 millimeters of distance range.In the present embodiment, according to each light for extracting
The hot spot distribution curve that the position fitting of spot is obtained is circle or oval.
Step S304, fisrt feature point is defined as by the characteristic point of hot spot distribution curve, wherein, during fisrt feature point includes
Point, the center of circle, focus, central point.
After fitting obtains hot spot distribution curve, the characteristic point of hot spot distribution curve is defined as fisrt feature point.Hot spot point
The shape of cloth curve is corresponding with the setting shape of all light sources, when hot spot distribution curve is bowlder, the spy of hot spot distribution curve
Levy a little and fisrt feature point be the center of circle, when hot spot distribution curve for it is oval when, the characteristic point of hot spot distribution curve and first special
Levy and be a little focus, be specifically as follows any one in two focuses, when hot spot distribution curve is line segment, hot spot distribution is bent
Characteristic point and the fisrt feature point of line are the midpoint of line segment, when hot spot distribution curve is polygon, hot spot distribution curve
Characteristic point and fisrt feature point are polygonal central point.
Step S306, position and presupposed information according to fisrt feature point determine the corresponding blinkpunkt information of eye image.
It is determined that after fisrt feature point, position and presupposed information according to fisrt feature point determine the corresponding note of eye image
View information.
A kind of specific embodiment is：Presupposed information is the position of the pupil center's point in eye image, according to first
The position of the pupil center's point in the position and eye image of characteristic point, determines the corresponding blinkpunkt information of eye image, specifically
Ground, determines the position of pupil center's point, the position of position and pupil center's point according to fisrt feature point, profit in eye image
Blinkpunkt information is obtained with based on the P-CR algorithms for returning, blinkpunkt information there can be various presentation modes, including but not limited to
Coordinate, drift angle and spin matrix etc..In this kind of embodiment, without knowing the corresponding relation between hot spot and light source, calculated
Journey is simple, and computational efficiency is high.
Another specific embodiment is that the characteristic point of the figure for constituting all light sources is defined as second feature point,
The position of position, the second feature point according to fisrt feature point, the position of each hot spot of extraction, the position of each light source, really
The fixed corresponding light source of each hot spot for extracting, according to the position and its position of corresponding light source of each hot spot for extracting, it is determined that
The corresponding blinkpunkt information of eye image, wherein, second feature point includes midpoint, the center of circle, focus, central point.This kind of embodiment party
In formula, presupposed information is the position of second feature point, the position and the position of each light source of each hot spot of extraction.
In the present embodiment, the figure of all light sources composition can for circular, oval, approximate circle, approximate ellipsoidal, just
Square, rectangle, line segment, trapezoidal etc., correspondingly, second feature point can be the center of circle, midpoint, focus, central point etc..Due to hot spot
It is imaging of each light source in human eye, therefore the shape of hot spot distribution curve is corresponding with the figure that all light sources are constituted, the
One characteristic point is corresponding with second feature point, and each light source is corresponding with each hot spot, based on this, according to the position of fisrt feature point
Put, the position of second feature point, the position of each hot spot of extraction, the position of each light source, it is determined that extract each hot spot pair
The detailed process of the light source answered is：
(1) position and the position of second feature point according to fisrt feature point, calculate fisrt feature point and second feature point
Between evolution relation.The evolution relation can be represented by way of coordinate, drift angle, spin matrix；
(2) according to the evolution relation and the position of each hot spot for extracting, each hot spot for extracting is calculated corresponding
Standard sources position.Wherein, calculation can add evolution relation for the position of hot spot, obtain corresponding standard sources
Position, the normal place of the corresponding light source of standard sources positional representation hot spot；
(3) light source that position and standard sources position match, is defined as the corresponding light source of each hot spot for extracting.By
In the influence such as computational accuracy error, may there is error in the position of light source, with standard sources position by the error in the mistake for allowing
Light source in difference scope is defined as the light source matched with standard sources position, and position and standard sources position are matched
Light source, is defined as the corresponding light source of each hot spot for extracting.
Because the shape of hot spot distribution curve is corresponding with the figure that all light sources are constituted, fisrt feature point and second feature
Point is corresponding, and each light source is corresponding with each hot spot, therefore, the position of position, the second feature point according to fisrt feature point,
Position, the position of each light source of each hot spot for extracting, it is determined that the detailed process of the corresponding light source of each hot spot for extracting is also
Can be：
(1) position of the position according to fisrt feature point and each hot spot for extracting, calculates fisrt feature point with extraction
The first relative position between each hot spot.Specific calculation can be position and each hot spot of extraction of fisrt feature point
Position subtract each other, obtain multiple first relative positions, each hot spot of extraction is to that should have first relative position, the first phase
Position can be represented by way of coordinate, drift angle, spin matrix；
(2) position of the position according to second feature point and each light source, calculates between second feature point and each light source
The second relative position.Specific calculation can be that the position of second feature point and the position of each light source are subtracted each other, and obtain many
Individual second relative position, each light source to that should have second relative position, the second relative position can by coordinate, partially
Angle, the mode of spin matrix are represented；
(3) the second relative position matched with the first relative position is determined, corresponding to the second relative position that will be determined
Light source, be defined as extract the corresponding light source of each hot spot.Influenceed in view of computational accuracy etc., by each the first relative position
Subtract each other with each second relative position, obtain difference, when difference is in the difference range for allowing, determine the first relative position and the
Two relative positions match.For the first relative position and the second relative position that match, first relative position is corresponding
Hot spot light source corresponding with second relative position is mutually corresponded to.
After the corresponding light source of each hot spot for extracting is determined using above-mentioned two ways, according to each hot spot for extracting
Position and its corresponding light source position, determine the corresponding blinkpunkt information of eye image using the P-CR algorithms based on 3D,
Blinkpunkt information can have various presentation modes, including but not limited to coordinate, drift angle and spin matrix etc..In the embodiment,
Need to know the corresponding relation between hot spot and light source, at the position of the pupil center's point in cannot determining eye image, energy
Blinkpunkt information is enough determined using the embodiment.
In the embodiment of the present invention, if extracting at least three hot spots in eye image, according to each hot spot for extracting
Position fitting hot spot distribution curve, wherein, eye image correspondence multiple light sources, each hot spot is an imaging for light source, will
The characteristic point of hot spot distribution curve is defined as fisrt feature point, wherein, fisrt feature point includes midpoint, the center of circle, focus, center
Point, position and presupposed information according to fisrt feature point determine the corresponding blinkpunkt information of eye image.Implemented by the present invention
Method in example, is imaged in user eyeball without whole light sources and forms hot spot, without between estimation hot spot and light source
Corresponding relation, when only light source such as three light sources in part form hot spot, it becomes possible to determine the corresponding blinkpunkt letter of eye image
Breath, so as to realize the quick estimation of blinkpunkt.
In view of two situations of hot spot are extracted in eye image, the embodiment of the present invention additionally provides following steps：
(a1), if extracting two hot spots in eye image, according to position, each light source of each hot spot for extracting
Position, the position of human eye feature in eye image, it is determined that extract the corresponding light source of each hot spot；
Human eye feature in eye image includes but is not limited to pupil, iris etc..In this step, with human eye in eye image
The position of feature is reference, it is determined that the corresponding light source of each hot spot for extracting, detailed process is：
(a11) third phase between each hot spot that the position calculating according to each hot spot for extracting is extracted is to position, root
The 4th relative position between each light source is calculated according to the position of each light source.Assuming that extracting N number of hot spot, then calculate and extract
Third phase of each hot spot relative to other any one hot spots for extracting to position, therefore for light that each is extracted
For spot, all with N-1 third phase to position, it is assumed that one has M light source, then calculate each light source any relative to other
One the 4th relative position of light source, therefore for each light source, all with M-1 the 4th relative position；
(a12) position according to each hot spot for extracting and the position of human eye feature, calculate each hot spot and people for extracting
The 5th relative position between eye feature, position and the position of human eye feature according to each light source, calculates each light source and people
The 6th relative position between eye feature.Each hot spot that calculating is extracted is relative to human eye feature (such as pupil or iris)
5th relative position, for each hot spot for extracting, is respectively provided with the 5th relative position, calculates each light source relative
In the 6th relative position of human eye feature (such as pupil or iris), for each light source for extracting, one is respectively provided with
6th relative position.It should be noted that when calculating the 5th relative position and calculating six relative positions, human eye feature is same
Human eye feature；
(a13) determine that the 4th relative position matches with third phase to position, and the 6th relative position position relative with the 5th
The light source for matching is put, is the corresponding light source of each hot spot for extracting.It is determined that during the corresponding light source of the hot spot for extracting, both examining
Consider the relative position between light source, it is also considered that to the relative position between light source and human eye feature, when relative between light source
Relative position between position and hot spot is matched, and relative position between light source and human eye feature and hot spot and human eye feature it
Between relative position match when, determine that light source is corresponding with hot spot.Signified two positions match in the present embodiment, refer to two positions
Between difference allow error range in.
In step (a11) to step (a13), third phase is to position, the 4th relative position, the 5th relative position and the 6th phase
Position can be represented by forms such as coordinate, drift angle and spin matrixs.
(a2), according to the position and its position of corresponding light source of each hot spot for extracting, determine that eye image is corresponding
Blinkpunkt information, or, the characteristic point of the figure that all light sources are constituted is defined as second feature point, according to each light for extracting
The position of the corresponding light source of spot, the position of second feature point, the position of each hot spot of extraction, determine second feature point in human eye
Mapping point in image, the position of the pupil center's point in the position and eye image of the mapping point, determines eye image
Corresponding blinkpunkt information, wherein, second feature point includes midpoint, the center of circle, focus, central point.
In this step, after the corresponding light source of each hot spot for knowing extraction, can be according to the position of each hot spot for extracting
Put and its corresponding light source position, determine the corresponding blinkpunkt information of eye image using the P-CR algorithms based on 3D, watch attentively
Point information can have various presentation modes, including but not limited to coordinate, drift angle and spin matrix etc..
In this step, after the corresponding light source of each hot spot for knowing extraction, it is contemplated that hot spot is light source in human eye
Imaging, each light source is corresponding with each hot spot, the image that the characteristic point of the figure that all light sources are constituted is constituted with all hot spots
Characteristic point it is corresponding, therefore can also in the following ways determine main view information：
(a21) characteristic point of the figure for constituting all light sources is defined as second feature point.The figure of all light source compositions
Can be circular, oval, approximate circle, approximate ellipsoidal, square, rectangle, line segment, trapezoidal etc., correspondingly, second feature
Point can be the center of circle, midpoint, focus, central point etc..
(a22) position according to the corresponding light source of each hot spot for extracting, the position of second feature point, each light of extraction
The position of spot, determines mapping point of the second feature point in eye image.The determination mode of mapping point can be：Calculate what is extracted
Evolution relation between corresponding each light source of each hot spot and second feature point, in eye image, using the position
The position of transformation relation and each hot spot for extracting, obtains the position of mapping point, so that it is determined that second feature point is in eye image
In mapping point.
(a23) position of the pupil center's point in the position and eye image of the mapping point, using based on recurrence
P-CR algorithms, determine the corresponding blinkpunkt information of eye image, and blinkpunkt information can have various presentation modes, including but not limit
In coordinate, drift angle and spin matrix etc..
By step (a1) and step (a2), when two hot spots can be extracted in eye image, eye image is determined
Corresponding blinkpunkt information, so as to determine the blinkpunkt information of human eye when number of spots is less, realizes that the quick of blinkpunkt is estimated
Meter.
In view of a situation for hot spot is extracted in eye image, the embodiment of the present invention additionally provides following steps：
(b1) if extracting a hot spot in eye image, control each light source to light one by one, determine the hot spot pair
The light source answered；Wherein, control each light source to light one by one, if in certain light source igniting hot spot occur, it is determined that the light source with
Hot spot correspondence.
(b2) characteristic point of the figure for constituting all light sources is defined as second feature point, corresponding according to the hot spot for extracting
The position of light source, the position of second feature point, the position of the hot spot of extraction, determine mapping of the second feature point in eye image
Point.
Specifically, the figure of all light source compositions can be circular, oval, approximate circle, approximate ellipsoidal, pros
Shape, rectangle, line segment, trapezoidal etc., correspondingly, second feature point can be the center of circle, midpoint, focus, central point etc..According to what is extracted
The position of the corresponding light source of hot spot, the position of second feature point, the position of the hot spot of extraction, determine second feature point in people's eye pattern
Mapping point as in, concrete mode is that the evolution calculated between the corresponding light source of hot spot and second feature point for extracting is closed
System, in eye image, using the evolution relation and the position of the hot spot for extracting, obtains the position of mapping point, so that really
Determine mapping point of the second feature point in eye image.
(b3) position of the pupil center's point in the position and eye image of the mapping point, determines eye image correspondence
Blinkpunkt information, wherein, second feature point include midpoint, the center of circle, focus, central point.
The position of the pupil center's point in the position and eye image of the mapping point, is calculated using based on the P-CR for returning
Method, determines the corresponding blinkpunkt information of eye image, and blinkpunkt information can have various presentation modes, including but not limited to sit
Mark, drift angle and spin matrix etc..
By step (b1) to step (b3), when a hot spot can be extracted in eye image, eye image is determined
Corresponding blinkpunkt information, so as to determine the blinkpunkt information of human eye when number of spots is minimum, realizes that the quick of blinkpunkt is estimated
Meter.
To sum up, by the method in the embodiment of the present invention, when can have three and above hot spot in eye image, really
Determine the corresponding blinkpunkt information of eye image, additionally it is possible to when there is two or a hot spot in eye image, determine people's eye pattern
As corresponding blinkpunkt information, so as to realize the determination of blinkpunkt information during the imaging of light source part, the fast of human eye fixation point is realized
Speed is estimated.
The above-mentioned gaze estimation method of correspondence, the embodiment of the present invention additionally provides a kind of line-of-sight estimation device, and Fig. 4 is this hair
The module composition schematic diagram of the line-of-sight estimation device that bright embodiment is provided, as shown in figure 4, the device includes：
Curve determining module 41, for obtaining eye image, if extracting at least three hot spots, root in eye image
Hot spot distribution curve is fitted according to the position of each hot spot for extracting, wherein, eye image correspondence multiple light sources, each hot spot is one
The imaging of individual light source；
Characteristic point determining module 42, for the characteristic point of hot spot distribution curve to be defined as into fisrt feature point, wherein, first
Characteristic point includes midpoint, the center of circle, focus, central point；
Information determination module 43, the corresponding note of eye image is determined for the position according to fisrt feature point and presupposed information
View information.
Wherein, information determination module 43 includes：First determination sub-module, for the position according to fisrt feature point and human eye
The position of the pupil center's point in image, determines the corresponding blinkpunkt information of eye image；Or, the second determination sub-module is used
It is defined as second feature point in the characteristic point of the figure for constituting all light sources, position, second feature according to fisrt feature point
The position of point, the position of each hot spot of extraction, the position of each light source, it is determined that the corresponding light source of each hot spot for extracting, root
According to the position and its position of corresponding light source of each hot spot for extracting, the corresponding blinkpunkt information of eye image is determined, wherein,
Second feature point includes midpoint, the center of circle, focus, central point.
In a kind of implementation method, the second determination sub-module determines the corresponding light of each hot spot for extracting in the following manner
Source：The position of position and second feature point according to fisrt feature point, calculates the position between fisrt feature point and second feature point
Put transformation relation；According to the evolution relation and the position of each hot spot for extracting, each hot spot for extracting is calculated corresponding
Standard sources position；The light source that position and standard sources position are matched, is defined as the corresponding light source of each hot spot for extracting.
In a kind of implementation method, the second determination sub-module determines the corresponding light of each hot spot for extracting in the following manner
Source：The position of position and each hot spot for extracting according to fisrt feature point, calculates fisrt feature point with each hot spot for extracting
Between the first relative position；Position according to second feature point and the position of each light source, calculate second feature point and each
The second relative position between light source；It is determined that the second relative position matched with the first relative position, the second phase that will be determined
To the light source corresponding to position, it is defined as the corresponding light source of each hot spot for extracting.
In the present embodiment, all light sources are rounded, oval, approximate circle or approximate ellipsoidal is arranged, hot spot distribution
Curve is circle or oval.
In the embodiment of the present invention, if extracting at least three hot spots in eye image, according to each hot spot for extracting
Position fitting hot spot distribution curve, wherein, eye image correspondence multiple light sources, each hot spot is an imaging for light source, will
The characteristic point of hot spot distribution curve is defined as fisrt feature point, wherein, fisrt feature point includes midpoint, the center of circle, focus, center
Point, position and presupposed information according to fisrt feature point determine the corresponding blinkpunkt information of eye image.Implemented by the present invention
Device in example, is imaged in user eyeball without whole light sources and forms hot spot, without between estimation hot spot and light source
Corresponding relation, when only light source such as three light sources in part form hot spot, it becomes possible to determine the corresponding blinkpunkt letter of eye image
Breath, so as to realize the quick estimation of blinkpunkt.
Further, the device in the present embodiment also includes：
First light source determining module, if for extracting two hot spots in eye image, according to each light for extracting
The position of spot, the position of each light source, the position of human eye feature in eye image, it is determined that the corresponding light of each hot spot for extracting
Source；
First blinkpunkt information determination module, for position and its position of corresponding light source according to each hot spot for extracting
Put, determine the corresponding blinkpunkt information of eye image, or, the characteristic point of the figure that all light sources are constituted is defined as the second spy
Levy a little, position, the position of second feature point, the position of each hot spot of extraction according to the corresponding light source of each hot spot for extracting
Put, mapping point of the second feature point in eye image is determined, according to the pupil center in the position and eye image of mapping point
The position of point, determines the corresponding blinkpunkt information of eye image, wherein, second feature point includes midpoint, the center of circle, focus, center
Point.
Wherein, the first blinkpunkt information determination module determines the corresponding light source of each hot spot for extracting in the following manner：
According to extract each hot spot position calculate extract each hot spot between third phase to position, according to the position of each light source
Put the 4th relative position calculated between each light source；Position and the position of human eye feature according to each hot spot for extracting, count
The 5th relative position between each hot spot and human eye feature for extracting is calculated, the position of position and human eye feature according to each light source
Put, calculate the 6th relative position between each light source and human eye feature；Determine the 4th relative position with third phase to position phase
Matching, and the light source that the 6th relative position matches with the 5th relative position, are the corresponding light source of each hot spot for extracting.
By the first light source determining module and the first blinkpunkt information determination module, two can be extracted in eye image
During individual hot spot, the corresponding blinkpunkt information of eye image is determined, so as to determine the blinkpunkt letter of human eye when number of spots is less
Breath, realizes the quick estimation of blinkpunkt.
Further, the device in the present embodiment also includes：
Secondary light source determining module, if for extracting a hot spot in eye image, controlling each light source one by one
Light, determine the corresponding light source of hot spot；
Mapping point determining module, the characteristic point of the figure for all light sources to be constituted is defined as second feature point, according to
The position of the corresponding light source of hot spot of extraction, the position of second feature point, the position of the hot spot of extraction, determine that second feature point exists
Mapping point in eye image；
Second blinkpunkt information determination module, for the pupil center's point in the position and eye image according to mapping point
Position, determines the corresponding blinkpunkt information of eye image, wherein, second feature point includes midpoint, the center of circle, focus, central point.
By secondary light source determining module, mapping point determining module and the second blinkpunkt information determination module, can be in people
When a hot spot is extracted in eye pattern picture, the corresponding blinkpunkt information of eye image is determined, so as to true when number of spots is minimum
Determine the blinkpunkt information of human eye, realize the quick estimation of blinkpunkt.
The line-of-sight estimation device that the embodiment of the present invention is provided can be equipment on specific hardware or be installed on equipment
On software or firmware etc..The technique effect of the device that the embodiment of the present invention is provided, its realization principle and generation and foregoing side
Method embodiment is identical, is to briefly describe, and device embodiment part does not refer to part, refers in corresponding in preceding method embodiment
Hold.It is apparent to those skilled in the art that, for convenience and simplicity of description, system, device described above
With the specific work process of unit, the corresponding process in above method embodiment is may be referred to, will not be repeated here.
In embodiment provided by the present invention, it should be understood that disclosed apparatus and method, can be by other sides
Formula is realized.Device embodiment described above is only schematical, for example, the division of the unit, only one kind are patrolled
Collect function to divide, there can be other dividing mode when actually realizing, but for example, multiple units or component can combine or can
To be integrated into another system, or some features can be ignored, or not perform.It is another, it is shown or discussed each other
Coupling or direct-coupling or communication connection can be the INDIRECT COUPLING or communication link of device or unit by some communication interfaces
Connect, can be electrical, mechanical or other forms.
The unit that is illustrated as separating component can be or may not be it is physically separate, it is aobvious as unit
The part for showing can be or may not be physical location, you can with positioned at a place, or can also be distributed to multiple
On NE.Some or all of unit therein can be according to the actual needs selected to realize the mesh of this embodiment scheme
's.
In addition, during each functional unit in the embodiment that the present invention is provided can be integrated in a processing unit, also may be used
Being that unit is individually physically present, it is also possible to which two or more units are integrated in a unit.
If the function is to realize in the form of SFU software functional unit and as independent production marketing or when using, can be with
Storage is in a computer read/write memory medium.Based on such understanding, technical scheme is substantially in other words
The part contributed to correlation technique or the part of the technical scheme can be embodied in the form of software product, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used to so that a computer equipment (can be individual
People's computer, server, or network equipment etc.) perform all or part of step of each embodiment methods described of the invention.
And foregoing storage medium includes：USB flash disk, mobile hard disk, read-only storage (ROM, Read-Only Memory), arbitrary access are deposited
Reservoir (RAM, Random Access Memory), magnetic disc or CD etc. are various can be with the medium of store program codes.
It should be noted that：Similar label and letter represents similar terms in following accompanying drawing, therefore, once a certain Xiang Yi
It is defined in individual accompanying drawing, then it need not be further defined and explained in subsequent accompanying drawing, additionally, term " the
One ", " second ", " the 3rd " etc. are only used for distinguishing description, and it is not intended that indicating or implying relative importance.
Finally it should be noted that：Embodiment described above, specific embodiment only of the invention, is used to illustrate the present invention
Technical scheme, rather than its limitations, protection scope of the present invention is not limited thereto, although with reference to the foregoing embodiments to this hair
It is bright to be described in detail, it will be understood by those within the art that：Any one skilled in the art
The invention discloses technical scope in, it can still modify to the technical scheme described in previous embodiment or can be light
Change is readily conceivable that, or equivalent is carried out to which part technical characteristic；And these modifications, change or replacement, do not make
The essence of appropriate technical solution departs from the spirit and scope of embodiment of the present invention technical scheme.Should all cover in protection of the invention
Within the scope of.Therefore, protection scope of the present invention described should be defined by scope of the claims.
Claims (10)
1. a kind of gaze estimation method, it is characterised in that including：
Eye image is obtained, if extracting at least three hot spots in the eye image, according to each the described light for extracting
The position fitting hot spot distribution curve of spot, wherein, the eye image correspondence multiple light sources, each described hot spot is described in one
The imaging of light source；
The characteristic point of the hot spot distribution curve is defined as fisrt feature point, wherein, the fisrt feature point includes midpoint, circle
The heart, focus, central point；
Position and presupposed information according to the fisrt feature point determine the corresponding blinkpunkt information of the eye image.
2. method according to claim 1, it is characterised in that the position according to the fisrt feature point and default letter
Breath determines the corresponding blinkpunkt information of the eye image, including：
The position of the pupil center's point in position and the eye image according to the fisrt feature point, determines people's eye pattern
As corresponding blinkpunkt information；Or,
The characteristic point of the figure of all light source compositions is defined as second feature point, according to the position of the fisrt feature point
Put, the position of second feature point, the position of each hot spot of extraction, the position of each light source, it is determined that extract
The corresponding light source of each described hot spot, position and its corresponding light source according to each hot spot for extracting
Position, determines the corresponding blinkpunkt information of the eye image, wherein, the second feature point include midpoint, the center of circle, focus,
Central point.
3. method according to claim 2, it is characterised in that position, described according to the fisrt feature point
The position of two characteristic points, the position of each hot spot of extraction, the position of each light source, it is determined that described in each extracted
The corresponding light source of hot spot, including：
The position of position and second feature point according to the fisrt feature point, calculates the fisrt feature point and described the
Evolution relation between two characteristic points；
According to the evolution relation and the position of each hot spot for extracting, each extracted hot spot correspondence is calculated
Standard sources position；
The light source that position is matched with the standard sources position, is defined as the corresponding light of each described hot spot for extracting
Source.
4. method according to claim 2, it is characterised in that position, described according to the fisrt feature point
The position of two characteristic points, the position of each hot spot of extraction, the position of each light source, it is determined that described in each extracted
The corresponding light source of hot spot, including：
The position of position and each hot spot for extracting according to the fisrt feature point, calculates the fisrt feature point and carries
The first relative position between each the described hot spot for taking；
Position according to the second feature point and the position of each light source, calculate the second feature point with described in each
The second relative position between light source；
It is determined that the second relative position matched with first relative position, corresponding to second relative position that will be determined
Light source, be defined as the corresponding light source of each described hot spot for extracting.
5. method according to claim 1, it is characterised in that all light sources it is rounded, oval, approximate circle or
Person's approximate ellipsoidal is arranged, and the hot spot distribution curve is circle or oval.
6. the method according to any one of claim 1 to 5, it is characterised in that also include：
If extracting two hot spots in the eye image, according to extract each hot spot position, each
The position of the light source, the position of human eye feature in the eye image, it is determined that each the described hot spot for extracting is corresponding described
Light source；
According to the position and its position of the corresponding light source of each hot spot for extracting, the eye image correspondence is determined
Blinkpunkt information, or, the characteristic point of the figure of all light sources compositions is defined as second feature point, according to what is extracted
The position of the corresponding light source of each described hot spot, the position of second feature point, the position of each hot spot for extracting
Put, determine mapping point of the second feature point in the eye image, position and the human eye according to the mapping point
The position of the pupil center's point in image, determines the corresponding blinkpunkt information of the eye image, wherein, the second feature point
Including midpoint, the center of circle, focus, central point.
7. method according to claim 6, it is characterised in that the position according to each hot spot for extracting, every
The position of the individual light source, the position of human eye feature in the eye image, it is determined that the corresponding institute of each described hot spot for extracting
Light source is stated, including：
Position according to each hot spot for extracting calculates third phase between each the described hot spot for extracting to position, according to
The position of each light source calculates the 4th relative position between each described light source；
According to extract each hot spot position and the human eye feature position, calculate extract each described hot spot with
The 5th relative position between the human eye feature, position and the position of the human eye feature according to each light source, counts
Calculate the 6th relative position between each described light source and the human eye feature；
Determine that the 4th relative position matches with the third phase to position, and the 6th relative position and the described 5th
The light source that relative position matches, is the corresponding light source of each described hot spot for extracting.
8. the method according to any one of claim 1 to 5, it is characterised in that also include：
If extracting a hot spot in the eye image, each described light source is controlled to light one by one, it is determined that described
The corresponding light source of hot spot；
The characteristic point of the figure of all light source compositions is defined as second feature point, it is corresponding according to the hot spot for extracting
The position of the light source, the position of second feature point, the position of the hot spot for extracting, determine that the second feature point exists
Mapping point in the eye image；
The position of the pupil center's point in the position and the eye image according to the mapping point, determines the eye image pair
The blinkpunkt information answered, wherein, the second feature point includes midpoint, the center of circle, focus, central point.
9. a kind of line-of-sight estimation device, it is characterised in that including：
Curve determining module, for obtaining eye image, if extracting at least three hot spots, basis in the eye image
The position fitting hot spot distribution curve of each hot spot for extracting, wherein, the eye image correspondence multiple light sources, each institute
It is an imaging for the light source to state hot spot；
Characteristic point determining module, for the characteristic point of the hot spot distribution curve to be defined as into fisrt feature point, wherein, described
One characteristic point includes midpoint, the center of circle, focus, central point；
Information determination module, determines that the eye image is corresponding for the position according to the fisrt feature point and presupposed information
Blinkpunkt information.
10. device according to claim 9, it is characterised in that described information determining module includes：
First determination sub-module, for the pupil center's point in the position and the eye image according to the fisrt feature point
Position, determines the corresponding blinkpunkt information of the eye image；Or,
Second determination sub-module, for the characteristic point of the figure of all light source compositions to be defined as into second feature point, according to
The position of the fisrt feature point, the position of second feature point, the position of each hot spot of extraction, each described light
The position in source, it is determined that the corresponding light source of each described hot spot for extracting, position according to each hot spot for extracting and
The position of its corresponding light source, determines the corresponding blinkpunkt information of the eye image, wherein, the second feature point bag
Include midpoint, the center of circle, focus, central point.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611207525.8A CN106778641B (en) | 2016-12-23 | 2016-12-23 | Sight estimation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611207525.8A CN106778641B (en) | 2016-12-23 | 2016-12-23 | Sight estimation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106778641A true CN106778641A (en) | 2017-05-31 |
| CN106778641B CN106778641B (en) | 2020-07-03 |
Family
ID=58919295
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611207525.8A Active CN106778641B (en) | 2016-12-23 | 2016-12-23 | Sight estimation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106778641B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107767421A (en) * | 2017-09-01 | 2018-03-06 | 北京七鑫易维信息技术有限公司 | Hot spot light source matching process and device in Eye-controlling focus equipment |
| CN108510542A (en) * | 2018-02-12 | 2018-09-07 | 北京七鑫易维信息技术有限公司 | The method and apparatus for matching light source and hot spot |
| CN108898572A (en) * | 2018-04-19 | 2018-11-27 | 北京七鑫易维信息技术有限公司 | Hot spot extracting method |
| WO2019085939A1 (en) * | 2017-11-01 | 2019-05-09 | 北京七鑫易维信息技术有限公司 | Calibration method and apparatus, storage medium, and processor |
| CN110032277A (en) * | 2019-03-13 | 2019-07-19 | 北京七鑫易维信息技术有限公司 | A kind of eyeball tracking device and intelligent terminal |
| CN110123267A (en) * | 2019-03-22 | 2019-08-16 | 重庆康华瑞明科技股份有限公司 | Additional flood projection device and image analysis system based on ophthalmic slit lamp |
| WO2021063155A1 (en) * | 2019-09-30 | 2021-04-08 | Oppo广东移动通信有限公司 | Human eye region positioning method and related device |
| CN114428547A (en) * | 2020-10-29 | 2022-05-03 | 北京七鑫易维信息技术有限公司 | Sight tracking method, device, equipment and storage medium |
| WO2023116571A1 (en) * | 2021-12-20 | 2023-06-29 | 华为技术有限公司 | Eye tracking apparatus and eye tracking method |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102930252A (en) * | 2012-10-26 | 2013-02-13 | 广东百泰科技有限公司 | Sight tracking method based on neural network head movement compensation |
| CN103761519A (en) * | 2013-12-20 | 2014-04-30 | 哈尔滨工业大学深圳研究生院 | Non-contact sight-line tracking method based on self-adaptive calibration |
| CN103838378A (en) * | 2014-03-13 | 2014-06-04 | 广东石油化工学院 | Head wearing type eye control system based on pupil recognition positioning |
| WO2014186347A1 (en) * | 2013-05-13 | 2014-11-20 | River Point, Llc | Medical headlamp assembly |
| CN104732191A (en) * | 2013-12-23 | 2015-06-24 | 北京七鑫易维信息技术有限公司 | Device and method for achieving eye-tracking of virtual display screens by means of crossratio invariability |
| CN105094300A (en) * | 2014-05-16 | 2015-11-25 | 北京七鑫易维信息技术有限公司 | Standardized eye image based eye gaze tracking system and method |
| CN205594581U (en) * | 2016-04-06 | 2016-09-21 | 北京七鑫易维信息技术有限公司 | Module is tracked to eyeball of video glasses |
| CN106168853A (en) * | 2016-06-23 | 2016-11-30 | 中国科学技术大学 | A kind of free space wear-type gaze tracking system |
-
2016
- 2016-12-23 CN CN201611207525.8A patent/CN106778641B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102930252A (en) * | 2012-10-26 | 2013-02-13 | 广东百泰科技有限公司 | Sight tracking method based on neural network head movement compensation |
| WO2014186347A1 (en) * | 2013-05-13 | 2014-11-20 | River Point, Llc | Medical headlamp assembly |
| CN103761519A (en) * | 2013-12-20 | 2014-04-30 | 哈尔滨工业大学深圳研究生院 | Non-contact sight-line tracking method based on self-adaptive calibration |
| CN104732191A (en) * | 2013-12-23 | 2015-06-24 | 北京七鑫易维信息技术有限公司 | Device and method for achieving eye-tracking of virtual display screens by means of crossratio invariability |
| CN103838378A (en) * | 2014-03-13 | 2014-06-04 | 广东石油化工学院 | Head wearing type eye control system based on pupil recognition positioning |
| CN105094300A (en) * | 2014-05-16 | 2015-11-25 | 北京七鑫易维信息技术有限公司 | Standardized eye image based eye gaze tracking system and method |
| CN205594581U (en) * | 2016-04-06 | 2016-09-21 | 北京七鑫易维信息技术有限公司 | Module is tracked to eyeball of video glasses |
| CN106168853A (en) * | 2016-06-23 | 2016-11-30 | 中国科学技术大学 | A kind of free space wear-type gaze tracking system |
Non-Patent Citations (3)
| Title |
|---|
| WANG CHANGYUAN 等: "The line of sight to estimate method based on stereo vision", 《MULTIMEDIA TOOLS AND APPLICATIONS》 * |
| 杨瑞: "人机交互中视线跟踪技术的研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 金纯 等: "视线追踪***中注视点估计算法研究", 《科学技术与工程》 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107767421B (en) * | 2017-09-01 | 2020-03-27 | 北京七鑫易维信息技术有限公司 | Light spot light source matching method and device in sight tracking equipment |
| CN107767421A (en) * | 2017-09-01 | 2018-03-06 | 北京七鑫易维信息技术有限公司 | Hot spot light source matching process and device in Eye-controlling focus equipment |
| US11003243B2 (en) | 2017-11-01 | 2021-05-11 | Beijing 7Invensun Technology Co., Ltd. | Calibration method and device, storage medium and processor |
| WO2019085939A1 (en) * | 2017-11-01 | 2019-05-09 | 北京七鑫易维信息技术有限公司 | Calibration method and apparatus, storage medium, and processor |
| CN108510542A (en) * | 2018-02-12 | 2018-09-07 | 北京七鑫易维信息技术有限公司 | The method and apparatus for matching light source and hot spot |
| WO2019154012A1 (en) * | 2018-02-12 | 2019-08-15 | 北京七鑫易维信息技术有限公司 | Method and apparatus for matching light sources with light spots |
| CN108510542B (en) * | 2018-02-12 | 2020-09-11 | 北京七鑫易维信息技术有限公司 | Method and device for matching light source and light spot |
| CN108898572B (en) * | 2018-04-19 | 2020-11-13 | 北京七鑫易维信息技术有限公司 | Light spot extraction method |
| CN108898572A (en) * | 2018-04-19 | 2018-11-27 | 北京七鑫易维信息技术有限公司 | Hot spot extracting method |
| CN110032277A (en) * | 2019-03-13 | 2019-07-19 | 北京七鑫易维信息技术有限公司 | A kind of eyeball tracking device and intelligent terminal |
| CN110123267A (en) * | 2019-03-22 | 2019-08-16 | 重庆康华瑞明科技股份有限公司 | Additional flood projection device and image analysis system based on ophthalmic slit lamp |
| CN110123267B (en) * | 2019-03-22 | 2022-02-08 | 重庆康华瑞明科技股份有限公司 | Additional floodlight projection device based on ophthalmic slit lamp and image analysis system |
| WO2021063155A1 (en) * | 2019-09-30 | 2021-04-08 | Oppo广东移动通信有限公司 | Human eye region positioning method and related device |
| CN114428547A (en) * | 2020-10-29 | 2022-05-03 | 北京七鑫易维信息技术有限公司 | Sight tracking method, device, equipment and storage medium |
| WO2023116571A1 (en) * | 2021-12-20 | 2023-06-29 | 华为技术有限公司 | Eye tracking apparatus and eye tracking method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106778641B (en) | 2020-07-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106778641A (en) | Gaze estimation method and device | |
| US11755102B2 (en) | User interface interaction paradigms for eyewear device with limited field of view | |
| KR102062658B1 (en) | Sphere tracking of the cornea to create an ocular model | |
| JP6106684B2 (en) | System and method for high resolution gaze detection | |
| US9961335B2 (en) | Pickup of objects in three-dimensional display | |
| CN113808160B (en) | Sight direction tracking method and device | |
| CN106445167B (en) | Simple eye visual field is adaptive to match method of adjustment and device, wear-type visual device | |
| JP2019500660A (en) | Eye tracking using structured light | |
| CN109558012A (en) | Eyeball tracking method and device | |
| CN107533362A (en) | Eye-tracking device and the method for operating eye-tracking device | |
| CN108354584A (en) | Eyeball tracking module and its method for tracing, virtual reality device | |
| US11422620B2 (en) | Display systems and methods for determining vertical alignment between left and right displays and a user's eyes | |
| WO2018076202A1 (en) | Head-mounted display device that can perform eye tracking, and eye tracking method | |
| CN103458770A (en) | Optical measuring device and method for capturing at least one parameter of at least one eye wherein an illumination characteristic is adjustable | |
| CN103605208A (en) | Content projection system and method | |
| CN108205374A (en) | Eyeball tracking module and its method, the video glass of a kind of video glass | |
| CN107991775B (en) | Head-mounted visual equipment capable of tracking human eyes and human eye tracking method | |
| WO2015051605A1 (en) | Image collection and locating method, and image collection and locating device | |
| CN108697321B (en) | Device for gaze tracking within a defined operating range | |
| US11860375B2 (en) | Virtual reality display device and method for presenting picture | |
| EP4038441A2 (en) | Compact retinal scanning device for tracking movement of the eye's pupil and applications thereof | |
| US20190236355A1 (en) | Gaze-tracking system using curved photo-sensitive chip | |
| CN108537103B (en) | Living body face detection method and device based on pupil axis measurement | |
| CN106293100A (en) | The determination method of sight line focus and virtual reality device in virtual reality device | |
| WO2016142489A1 (en) | Eye tracking using a depth sensor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |