CN108259887A - Watch point calibration method and device, blinkpunkt scaling method and device attentively - Google Patents
Watch point calibration method and device, blinkpunkt scaling method and device attentively Download PDFInfo
- Publication number
- CN108259887A CN108259887A CN201810330258.6A CN201810330258A CN108259887A CN 108259887 A CN108259887 A CN 108259887A CN 201810330258 A CN201810330258 A CN 201810330258A CN 108259887 A CN108259887 A CN 108259887A
- Authority
- CN
- China
- Prior art keywords
- eye
- blinkpunkt
- scene
- calibration
- right eye
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Eye Examination Apparatus (AREA)
Abstract
Watch point calibration method and device attentively the present invention provides one kind and provide a kind of blinkpunkt scaling method and device.Described to watch point calibration method attentively and need using multigroup binocular parallax images, the left-eye image and eye image in every group of binocular parallax images are obtained by same scene, and each scene includes a calibration point known to coordinate in scene coordinate system；During calibration, multiple left-eye images are played successively to left eye, carry out left eye blinkpunkt calibration, obtain the left eye mapping relations about left eye eye feature and calibration point coordinates；It plays multiple eye images successively to right eye, carries out right eye blinkpunkt calibration, obtain the right eye mapping relations about right eye eye feature and calibration point coordinates.Since binocular parallax images are stereo-picture, left eye and right eye are calibrated respectively using the left-eye image and eye image of binocular parallax images, obtained left eye mapping relations and right eye mapping relations and Stereo image matching precision higher.
Description
Technical field
The present invention relates to visual pursuit technical field, in particular to one kind watch attentively point calibration method and device and
A kind of blinkpunkt scaling method and device.
Background technology
Eye movement analysis instrument is that a kind of important instrument of current virtual reality, hot spot capture and the detection of mankind's daily behavior is set
Standby, tool has been widely used.Before eye movement analysis instrument is used, i.e., blinkpunkt is demarcated using eye movement analysis instrument
Before, it needs to calibrate blinkpunkt using eye movement analysis instrument.So-called blinkpunkt, during referring to visual perception, sight pair
The certain point of accurate object.So-called calibration, i.e., calibration point known to displaing coordinate, eye video camera shoot eye successively in the plane
Eye feature of the eyeball when observing each calibration point finds eye feature and calibrates the mapping relations between point coordinates.It is so-called right
Blinkpunkt is demarcated, i.e., when using eye movement analysis instrument, the eye feature and when calibration shot according to eye video camera is true
Fixed mapping relations determine that coordinate is demarcated in the position of blinkpunkt in the plane to get to blinkpunkt.
Currently used calibration method includes linear scaled, polar coordinates calibration, fitting and difference scaling method etc..However at present
Any calibration method does not account for the visual characteristic of people, and when calibration is carried out using single sheet of planar image, and is not had
In view of the left and right vision difference of human body, it can not accurately reflect the correspondence between human eye vision and actual object, have
Certain limitation.
As shown in Figure 1a, when right and left eyes see an object simultaneously, because eyes see that the angle of object is different, can cause
The position of object difference in the different eyeball visuals field.As shown in Figure 1 b, the field-of-view angle range of people also has certain limitation,
The semicircle of theoretically one 240 degree of approximation, that is to say, that things of the normal person to oneself body side and below is without solid
Sense, there is only at the moment 150 degree or so of relief range.As illustrated in figure 1 c, from people it is observed that in face of from the point of view of image, one
As the image observed of people nor complete 360 degree of border circular areas, but the similar heart-shaped region that dotted line is included in figure.
In conclusion the image and real image observed by people only can be overlapped more in a part of area of space, and
And the image observed by two not fully overlaps, therefore the single sheet of planar image of traditional use either rectangular coordinate system
Square region or polar circular image cannot all find accurately mapping relations, therefore can not completely accurate terrestrial reference
Determine blinkpunkt.
Invention content
The purpose of the present invention is to provide one kind watch attentively point calibration method and device and a kind of blinkpunkt scaling method and
Device carries out blinkpunkt calibration to right and left eyes respectively based on binocular vision, finds right and left eyes eye feature respectively and is sat with calibration point
Mapping relations between mark improve blinkpunkt stated accuracy.
To achieve these goals, an embodiment of the present invention provides following technical schemes：
On the one hand, watch point calibration method attentively an embodiment of the present invention provides one kind, it is described to watch point calibration method attentively and need profit
With multigroup binocular parallax images, every group of binocular parallax images include left-eye image and eye image, in every group of binocular parallax images
Left-eye image and eye image obtained by same scene, each scene includes one in scene coordinate system known to coordinate
Point, the known point of the coordinate is as calibration point；
It is described to watch point calibration method attentively and specifically include：
During the calibration point watched attentively in every left-eye image in tester's left eye, the eye feature of left eye is obtained, according to more
A calibration point coordinates and more parts of left eye eye features of acquisition obtain the mapping about left eye eye feature and calibration point coordinates
Relationship, as left eye mapping relations；
During the calibration point watched attentively in every eye image in tester's right eye, the eye feature of right eye is obtained, according to more
A calibration point coordinates and more parts of right eye eye features of acquisition obtain the mapping about right eye eye feature and calibration point coordinates
Relationship, as right eye mapping relations.
On the other hand, the embodiment of the present invention provides a kind of blinkpunkt calibrating installation simultaneously, including：
Virtual Reality system, for the left-eye image of the binocular parallax images described in power 1 and eye image to be shown respectively
Show to the left eye and right eye of tester；
Eye feature harvester, during left eye blinkpunkt calibration is carried out, the eye feature figure of collecting test person's left eye
Piece；During right eye blinkpunkt calibration is carried out, the eye feature picture of collecting test person's right eye；And
Middle control module, for the coordinate and multiple left eye eye features according to multiple calibration points in scene coordinate system
Picture calculates left eye mapping relations；It is special according to coordinate of multiple calibration points in scene coordinate system and multiple right eye eyes
Picture is levied, calculates right eye mapping relations.
Further, the blinkpunkt calibrating installation further includes binocular camera shooting device, for acquiring the binocular parallax figure
Picture.
Further, blinkpunkt calibrating installation further includes communication module, and the middle control module can pass through the communication module
Receive by the binocular camera shooting device teletransmission and Lai the binocular parallax images, and the binocular parallax images are transmitted
To the Virtual Reality system.
In another aspect, the embodiment of the present invention provides a kind of blinkpunkt scaling method simultaneously, specifically include：
The left-eye image and eye image of one group of binocular parallax images are observed respectively simultaneously in the left eye and right eye of tester
Period obtains the left eye eye feature of tester and right eye eye feature；The left-eye image and right eye of the binocular parallax images
Image is obtained by same scene, and the scene has a scene coordinate system；
According to left eye eye feature and left eye mapping relations, seat of the left eye blinkpunkt in scene coordinate system is calculated
Mark；
According to right eye eye feature and right eye mapping relations, seat of the right eye blinkpunkt in scene coordinate system is calculated
Mark；
According to coordinate and right eye blinkpunkt seat in scene coordinate system of the left eye blinkpunkt in scene coordinate system
Mark, the coordinate of the blinkpunkt of discriminating test person in scene coordinate system be located at left eye watch point coordinates attentively and right eye watch attentively point coordinates it
Between.
In another aspect, the embodiment of the present invention provides a kind of blinkpunkt caliberating device simultaneously, including：
Virtual Reality system, for the left-eye image of binocular parallax images and eye image to be shown to tester respectively
Left eye and right eye；
Eye feature harvester, in left eye observation left-eye image right eye observation eye image simultaneously, for adopting respectively
Collect the eye feature picture of left eye and the eye feature picture of right eye；And
Middle control module, for according to left eye eye feature and left eye mapping relations, calculating left eye blinkpunkt in scene
Coordinate in coordinate system；According to right eye eye feature and right eye mapping relations, right eye blinkpunkt is calculated in scene coordinate system
In coordinate；And coordinate of the blinkpunkt of discriminating test person in scene coordinate system is located at left eye and watches point coordinates attentively and watch attentively with right eye
Between point coordinates.
Further, the blinkpunkt caliberating device further includes binocular camera shooting device, for acquiring the binocular parallax figure
Picture.
Further, the blinkpunkt caliberating device further includes communication module, and the middle control module can pass through the communication
Module receive by the binocular camera shooting device teletransmission and Lai the binocular parallax images, and by the binocular parallax images
Send the Virtual Reality system to.
Compared with prior art, the beneficial effects of the present invention are：Since binocular parallax images are stereo-picture, using double
The left-eye image and eye image of visual difference image respectively calibrate left eye and right eye, obtained left eye mapping relations and the right side
Eye mapping relations and Stereo image matching precision higher.Therefore it is noted using the left eye mapping relations and right eye mapping relations
When viewpoint is demarcated, blinkpunkt calibrated error smaller.
Description of the drawings
It in order to illustrate the technical solution of the embodiments of the present invention more clearly, below will be to needed in the embodiment attached
Figure is briefly introduced, it should be understood that the following drawings illustrates only certain embodiments of the present invention, therefore is not construed as to model
The restriction enclosed, for those of ordinary skill in the art, without creative efforts, can also be according to these
Attached drawing obtains other relevant drawings.
Fig. 1 a show the reference chart about background technology part.
Fig. 1 b show the reference chart about background technology part.
Fig. 1 c show the reference chart about background technology part.
Fig. 2 show the flow chart for watching point calibration method in present pre-ferred embodiments attentively.
Fig. 3 show the image pickup method schematic diagram of binocular camera shooting device in scene in kind.
Fig. 4 show the schematic diagram of scene and calibration point.
Fig. 5 show the schematic diagram of another scene and calibration point.
Fig. 6 show the structure diagram of blinkpunkt calibrating installation provided in embodiment.
Fig. 7 show the structure diagram of another blinkpunkt calibrating installation provided in embodiment.
Fig. 8 show the structure diagram of another blinkpunkt calibrating installation provided in embodiment.
Fig. 9 show the structure diagram of left eye eye feature harvester provided in embodiment.
Figure 10 show the flow chart of blinkpunkt scaling method in present pre-ferred embodiments.
Figure 11 a show a kind of binocular fixation point provided in embodiment and determine method schematic diagram.
Figure 11 b show another binocular fixation point provided in embodiment and determine method schematic diagram.
Figure label explanation：
10- infrared light supplies；20- eye video cameras；The first optical filters of 30-；The second optical filters of 40-.
Specific embodiment
Below in conjunction with attached drawing in the embodiment of the present invention, the technical solution in the embodiment of the present invention is carried out clearly complete
Description.It should be appreciated that specific embodiment described herein is used only for explaining the present invention, it is not intended to limit the present invention.Base
In the embodiment of the present invention, every other implementation that those skilled in the art are obtained under the premise of no creative work
Example, belongs to protection scope of the present invention.
It please refers to shown in Fig. 2, present embodiments provides one kind and watch point calibration method attentively.
Described to watch point calibration method attentively and need using multigroup binocular parallax images, every group of binocular parallax images include left eye figure
Picture and eye image, left-eye image and eye image in every group of binocular parallax images are obtained by same scene, each scene
Include a point known to coordinate in scene coordinate system, the known point of the coordinate is as calibration point；
It is described to watch point calibration method attentively and specifically include：
During the calibration point watched attentively in every left-eye image in tester's left eye, the eye feature of left eye is obtained, according to more
A calibration point coordinates and more parts of left eye eye features of acquisition obtain the mapping about left eye eye feature and calibration point coordinates
Relationship, as left eye mapping relations；
During the calibration point watched attentively in every eye image in tester's right eye, the eye feature of right eye is obtained, according to more
A calibration point coordinates and more parts of right eye eye features of acquisition obtain the mapping about right eye eye feature and calibration point coordinates
Relationship, as right eye mapping relations.
Based on the above method, some specific embodiments are will be given below, it is each to be embodied under the premise of mutually reconcilable
Mode can be combined with each other, and form specific method.
For example, it please refers to shown in Fig. 3.The scene can be scene in kind, the left-eye image of binocular parallax images and the right side
Eye pattern picture can be acquired from scene in kind using binocular camera shooting device and be obtained.Binocular camera shooting device during multiple series of images is acquired,
Position of the binocular camera shooting device in scene coordinate system is fixed.And calibration point is beated according to artificial control in scene coordinate system,
Coordinate after calibration point is beated every time is known.Calibration point is often beated once, and binocular camera shooting device acquires an image, described
Image includes left-eye image and eye image, and calibration point should be fallen into left-eye image and eye image simultaneously.It is listed in Fig. 3 several
Group shooting schematic diagram, every group of schematic diagram correspond to primary shooting.
For example, the scene can be virtual scene again, the virtual scene can utilize existing virtual reality figure to give birth to
It is synthesized into technology.Therefore the left-eye image of binocular parallax images and eye image are directly to be generated by computer manufacture.Every group
Coordinate in virtual scene coordinate system of calibration point corresponding to binocular parallax images it is known that and calibration point should fall into a left side simultaneously
In eye pattern picture and eye image.
For example, it please refers to shown in Fig. 4.The scene is scene in kind or virtual scene, can be in the scene
Including a plane, the plane is divided equally by cell, and coordinate of the calibration point in scene coordinate system is different.The calibration
Point is located in the plane, and the calibration point is located at the vertex of cell.For the scene corresponding to each group binocular parallax images,
Position of the plane in scene coordinate system is all identical, and is divided equally by identical cell.It is each unlike but
In scene, calibration point is located at different cell vertex in the planes.
For example, it please refers to shown in Fig. 5.The scene is scene in kind or virtual scene, the scene alignment
The bounce position of point may not necessarily be defined in a certain plane.But even if the bounce position of calibration point is not limited to a certain plane
It is interior, it should be understood that, the coordinate after calibration point is beated every time still should be known.
For example, left-eye image can be played to left eye respectively by Virtual Reality system and play eye image to right eye.
For example multigroup binocular parallax images are stored in the Virtual Reality system, the left-eye image of every group of binocular parallax images and the right side
Eye pattern picture respectively enters the left eye and right eye of tester by Virtual Reality system.During left eye blinkpunkt calibration is carried out,
Virtual Reality system puts right eye visual field black, and Virtual Reality system plays multiple left-eye images to left eye successively.Carry out
During right eye blinkpunkt is calibrated, Virtual Reality system puts left eye visual field black, and Virtual Reality system plays successively to left eye
Multiple left-eye images.For example virtual reality head-mounted display apparatus can be selected in the Virtual Reality system, such as VR glasses.
For example, can left eye eye feature be obtained by existing method-corneal reflection method.If entire eyeball is a standard
Sphere is fixed with center of circle rotation, then reflective spot.But the truth of eyeball, which is cornea, to be protruded from eyeball surface
, therefore when eye movement, light is mapped to cornea with the angle changed, obtains the reflected light of different directions.Based on corneal reflection
Method can set camera and infrared light supply in drawing axis, and the infrared light of infrared light supply transmitting is by corneal reflection and passes through one
A little speculums, lens are input to camera.Then image procossing eye pattern and corneal reflection is determined using some corresponding algorithms
The position of light.
For example, can also by other existing methods obtain left eye eye feature, as pupil-corneal reflection vector method, sclera-
Iris edge method, double Pooles are admired as method etc..Since the above method is existing method, the present embodiment is no longer to the above method
It is further elaborated.
For example, according to the left eye eye feature of acquisition, the specific method of extraction left eye eye feature data includes but unlimited
In existing methods such as bright dark pupil method, Starburst algorithms, circumference difference algorithms.
For example, when solving left eye mapping relations, using classical pupil-corneal reflection hot spot difference fitting process.Its
In establish quadratic regression model xv=a0+a1xe+a2ye+a3xeye+a4xe 2+a5ye 2；yv=b0+b1ye+b2xe+b3xeye+b4ye 2+
b5xe 2.Wherein, (xv, yv) for the blinkpunkt coordinate position in scene coordinate system；(xe, ye) formed for pupil-cornea coordinate value
Vector, for the vector in eeye-coordinate system or in eye cameras coordinate system, which can pass through aforementioned corneal reflection
Method obtains.Before using the quadratic regression model as left eye mapping relations, need that a is obtained0~a5And b0~b5This 12 ginsengs
Number.The method for solving this 12 parameters is to play 12 left-eye images to left eye successively, the school in left eye observes left-eye image
When on schedule, 12 groups of (x are calculated using infrared light supply and eye video camera and based on corneal reflection methode, ye) coordinate.According to 12
Open (x known to 12 groups of respective alignment point in left-eye imagev, yv) coordinate and above 12 groups of (xe, ye) coordinate, calculate 12
Parameter.12 parameters are brought into quadratic regression model, which can be used as left eye mapping relations.It should be understood that
, for right eye mapping relations, the above method can also be used and be obtained.Simultaneously it should be appreciated that right and left eyes calibrate sequencing not
It is construed as limiting.
For example, when carrying out the calibration of left eye blinkpunkt, other existing calibration methods, such as least square can also be utilized
Fitting process, the human eye sight angle detecting method based on Purkinje image point etc..It should be appreciated that for right eye mapping relations,
It is obtained using the above method.Simultaneously it should be appreciated that right and left eyes calibration sequencing is not construed as limiting.
For existing conventional calibration methods, right and left eyes are calibrated using single sheet of planar image, are not accounted for
To right and left eyes parallax, therefore there are larger matching errors between its obtained mapping relations and three-dimensional scenic.And the present embodiment carries
In the method for confession, since binocular parallax images are stereo-picture, the left-eye image and eye image point of binocular parallax images are utilized
Other that left eye and right eye are calibrated, obtained left eye mapping relations and right eye mapping relations and Stereo image matching precision are more
It is high.
It please refers to shown in Fig. 6, watches point calibration method attentively based on above-mentioned, the present embodiment provides a kind of blinkpunkt calibration simultaneously
Device.
The blinkpunkt calibrating installation includes：
Virtual Reality system is shown to test respectively for the left-eye image and eye image of above-mentioned binocular parallax images
The left eye and right eye of person；
Eye feature harvester, during left eye blinkpunkt calibration is carried out, the eye feature figure of collecting test person's left eye
Piece；During right eye blinkpunkt calibration is carried out, the eye feature picture of collecting test person's right eye；And
Middle control module, for the coordinate and multiple left eye eye features according to multiple calibration points in scene coordinate system
Picture calculates left eye mapping relations；It is special according to coordinate of multiple calibration points in scene coordinate system and multiple right eye eyes
Picture is levied, calculates right eye mapping relations.
Based on above device, some specific embodiments are will be given below, it is each to be embodied under the premise of mutually reconcilable
Mode can be combined with each other, and form specific device.
For example, virtual reality head-mounted display apparatus can be selected in the Virtual Reality system, such as VR glasses.
For example, multigroup binocular parallax images or the merging virtual reality are stored in the Virtual Reality system
Multigroup binocular parallax images are stored in other electronic equipments of VR systems, the binocular parallax images can derive from shooting in advance
Scene in kind or from prior computer synthesis virtual scene.It is virtual existing during left eye or right eye blinkpunkt are calibrated
Real VR systems directly call the left-eye image and eye image of binocular parallax images from itself, are played respectively to left eye and right eye.
For example, please referring to shown in Fig. 7, the blinkpunkt calibrating installation can also include binocular camera shooting device, for acquiring
The binocular parallax images.The binocular parallax images of acquisition are transferred to middle control by the binocular camera shooting device by image pick-up card
Module, then binocular parallax images are sent into Virtual Reality system by middle control module.During left eye blinkpunkt is calibrated, calibration point
It beats successively in scene in kind, calibration point is often beated once, and binocular camera shooting device shoots a left eye picture, and by left eye figure
Piece is transferred to Virtual Reality system, is allowed to play to the left eye of tester.During right eye blinkpunkt is calibrated, calibration point is in reality
According to similarly sequentially beating when being calibrated with left eye in object field scape, calibration point is often beated once, and binocular camera shooting device shoots one
Right eye picture, and give right eye picture transfer to Virtual Reality system, it is allowed to play to the right eye of tester.It should be appreciated that
Right and left eyes calibration sequencing is not construed as limiting.
For example, please referring to shown in Fig. 8, the blinkpunkt calibrating installation can also include communication module.Binocular camera shooting at this time
Device can be connect by a variety of wireless transmission methods with middle control module, can carry out left eye and right eye using long-range scene in kind
Blinkpunkt is calibrated.For example special calibration laboratory is set, it is disposed with what is calibrated for left eye and right eye blinkpunkt in laboratory
Scene in kind and binocular camera shooting device, during calibration, the middle control module of user remotely receives calibration laboratory by communication module
In binocular camera shooting device shooting binocular parallax images, calibrated.Such as optional 3G, 4G or 5G communication mould of communication module
Block.
For example, the eye feature harvester includes left eye eye feature harvester and right eye eye feature acquisition dress
It puts.It please refers to shown in Fig. 9, the left eye eye feature harvester includes infrared light supply 10, eye video camera 20, first filters
30 and second optical filter 40 of piece.Wherein the first optical filter 30 has properties of infrared reflection, positioned at drawing axis and positioned at infrared
40 lower section of light source 10 and eye video camera, will accomplish neither to interfere observation of the left eye to left-eye image, and cover eye video camera
20 range of exposures makes eye video camera 20 collect clearly left eye feature image.Second optical filter 40 has infrared projection
Performance on 20 camera lens of eye video camera, can filter out other light into eye video camera 20, ensure that entering eye images
Machine 20 is mostly infrared light.In use, the infrared light that infrared light supply 10 is sent out reflects 30 through the first optical filter, it is irradiated to
Eyes region.Infrared light reaches camera lens, the second optical filter after 30 second secondary reflection of eye reflections and the first optical filter
After 40 filter out veiling glare, eye video camera 20 collects clearly left eye feature image.Eye video camera 20 passes through image pick-up card
Module is controlled during left eye feature image is sent into.
For example, the middle control module can select microprocessor, it is used to calculate left eye mapping relations in the microprocessor
With the program of right eye mapping relations, described program can write according to existing computational methods, for example bright dark pupil of existing computational methods
Method, Starburst algorithms, circumference difference algorithm etc..
It please refers to Fig.1 shown in 0, the present embodiment watches point calibration method attentively based on above-mentioned, additionally provides a kind of blinkpunkt calibration
Method.
The blinkpunkt scaling method specifically includes：
The left-eye image and eye image of one group of binocular parallax images are observed respectively simultaneously in the left eye and right eye of tester
Period obtains the left eye eye feature of tester and right eye eye feature；The left-eye image and right eye of the binocular parallax images
Image is obtained by same scene, and the scene has a scene coordinate system；
According to left eye eye feature and left eye mapping relations, seat of the left eye blinkpunkt in scene coordinate system is calculated
Mark；
According to right eye eye feature and right eye mapping relations, seat of the right eye blinkpunkt in scene coordinate system is calculated
Mark；
According to coordinate and right eye blinkpunkt seat in scene coordinate system of the left eye blinkpunkt in scene coordinate system
Mark, the coordinate of the blinkpunkt of discriminating test person in scene coordinate system be located at left eye watch point coordinates attentively and right eye watch attentively point coordinates it
Between.
Based on the above method, some specific embodiments are will be given below, it is each to be embodied under the premise of mutually reconcilable
Mode can be combined with each other, and form specific method.
For example, the binocular parallax images can derive from scene in kind, the material object scene has coordinate system.Binocular parallax
Binocular camera shooting device shooting in image and scene in kind obtains.Binocular camera shooting device can pass through wirelessly or non-wirelessly communication module
It is connect with Virtual Reality system, binocular parallax images are played to right and left eyes by Virtual Reality system.During calibration, field in kind
Binocular camera shooting device in scape is fixed, and be worn on tester head Virtual Reality system can with tester head with
Meaning movement.
For example, the binocular parallax images can derive from scene in kind, the material object scene has coordinate system.Binocular parallax
Binocular camera shooting device shooting in image and scene in kind obtains, and binocular camera shooting device is fixedly connected with Virtual Reality system,
Binocular parallax images are played to right and left eyes by Virtual Reality system.During calibration, tester head remains stationary as, and takes the photograph binocular
As device also relative physical scene coordinate system remains stationary.Or during calibration, binocular camera shooting device can follow the head of tester
Portion's voltuntary movement, but need to increase another fixing camera and computing unit at this time, it is relatively real to calculate binocular camera shooting device
The displacement of object field scape coordinate system.
For example, the binocular parallax images can derive from scene in kind, but the scene in kind is pre-stored within virtually now
In real VR systems.Therefore it does not need to use binocular camera shooting device during demarcating.
For example, the binocular parallax images can derive from the virtual image synthesized by computer again, using existing virtual
Realistic graphic generation technique synthesizes.There is a coordinate system, virtual image is by left-eye image and right eye figure in the scene of virtual image
As composition.The virtual image is pre-stored in Virtual Reality system.
For example, can by preceding method, as corneal reflection method, pupil-corneal reflection vector method, sclera-iris edge method,
Double Pooles are admired as method etc., obtain left eye eye feature.Pass through preceding method, such as bright dark pupil method, Starburst algorithms, circle
The methods of all difference algorithms, extracts left eye eye feature data.For right eye, the same left eye of method.
For example, the left eye eye feature data that will be extracted, i.e., the vector Z Y (x being made of pupil-cornea coordinate valuee,
ye), it is brought into left eye mapping relations, i.e. xv=a0+a1xe+a2ye+a3xeye+a4xe 2+a5ye 2；yv=b0+b1ye+b2xe+
b3xeye+b4ye 2+b5xe 2In, 12 parameters one therein are obtained.The blinkpunkt of left eye can be obtained in scene coordinate system
Coordinate ZY (xv, yv).For right eye, coordinate YY (x of the blinkpunkt of right eye in scene coordinate system is obtained in the same left eye of methodv,
yv)。
For example, ZY (x are obtained respectivelyv, yv) and YY (xv, yv) after.It please refers to Fig.1 shown in 1a, final binocular fixation point SY exists
Position in scene coordinate system can be chosen to be ZY (xv, yv)、YY(xv, yv) 2 lines midpoint.Or please refer to Fig.1 1b
Shown, positions of the final binocular fixation point SY in scene coordinate system can be selected：It falls with ZY (xv, yv)、YY(xv, yv) two
Point line is in the sphere of diameter.It should be appreciated that the determining side of final positions of the binocular fixation point SY in scene coordinate system
Formula is not limited to above two citing.
Based on above-mentioned blinkpunkt scaling method, the present embodiment provides a kind of blinkpunkt caliberating device simultaneously, please refers to Fig. 6
Shown hardware configuration.
The blinkpunkt calibrating installation includes：
Virtual Reality system, for the left-eye image of binocular parallax images and eye image to be shown to tester respectively
Left eye and right eye；
Eye feature harvester, in left eye observation left-eye image right eye observation eye image simultaneously, for adopting respectively
Collect the eye feature picture of left eye and the eye feature picture of right eye；And
Middle control module, for according to left eye eye feature and left eye mapping relations, calculating left eye blinkpunkt in scene
Coordinate in coordinate system；According to right eye eye feature and right eye mapping relations, right eye blinkpunkt is calculated in scene coordinate system
In coordinate；And coordinate of the blinkpunkt of discriminating test person in scene coordinate system is located at left eye and watches point coordinates attentively and watch attentively with right eye
Between point coordinates.
Based on above device, some specific embodiments are will be given below, it is each to be embodied under the premise of mutually reconcilable
Mode can be combined with each other, and form specific device.
For example, virtual reality head-mounted display apparatus can be selected in the Virtual Reality system, such as VR glasses.
For example, multigroup binocular parallax images or the merging virtual reality are stored in the Virtual Reality system
Multigroup binocular parallax images are stored in other electronic equipments of VR systems, the binocular parallax images can derive from shooting in advance
Scene in kind or from prior computer synthesis virtual scene.During calibration, Virtual Reality system for simultaneously to
Left eye and right eye play left-eye image and eye image respectively.
For example, please referring to hardware configuration shown in Fig. 7, the blinkpunkt caliberating device can also include binocular camera shooting and fill
It puts, for acquiring the binocular parallax images.The binocular camera shooting device is by image pick-up card by the binocular parallax figure of acquisition
Virtual Reality system is sent into as being transferred to middle control module, then by middle control module.
For example, please referring to hardware configuration shown in Fig. 8, the blinkpunkt caliberating device can also include communication module, this
When binocular camera shooting device can be connect by a variety of wireless transmission methods with middle control module, tester can be to remote objects and scenery
Deng progress blinkpunkt calibration, solve the problems, such as that people must reach scene and carry out blinkpunkt calibration.For the communication module, than
Such as optional 3G, 4G or 5G communication module.
For example, the eye feature harvester includes left eye eye feature harvester and right eye eye feature acquisition dress
It puts.The concrete structure of the left eye eye feature harvester and right eye eye feature harvester can be same as shown in Fig. 9 hard
Part structure.
For example, the middle control module can select microprocessor, it is stored in the microprocessor and utilizes above-mentioned blinkpunkt
The left eye mapping relations and right eye mapping relations that calibration method acquires.
The above description is merely a specific embodiment, but protection scope of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can readily occur in change or replacement, should
Cover within the scope of the present invention.
Claims (10)
1. one kind watches point calibration method attentively, which is characterized in that described to watch point calibration method attentively and need using multigroup binocular parallax figure
Picture, every group of binocular parallax images include left-eye image and eye image, left-eye image and right eye in every group of binocular parallax images
Image is obtained by same scene, and each scene includes a point known to coordinate in scene coordinate system, and the coordinate is
The point known is as calibration point；
It is described to watch point calibration method attentively and specifically include：
During the calibration point watched attentively in every left-eye image in tester's left eye, the eye feature of left eye is obtained, according to multiple schools
Coordinate and more parts of left eye eye features of acquisition on schedule obtain closing about the mapping of left eye eye feature and calibration point coordinates
System, as left eye mapping relations；
During the calibration point watched attentively in every eye image in tester's right eye, the eye feature of right eye is obtained, according to multiple schools
Coordinate and more parts of right eye eye features of acquisition on schedule obtain closing about the mapping of right eye eye feature and calibration point coordinates
System, as right eye mapping relations.
2. according to claim 1 watch point calibration method attentively, which is characterized in that for corresponding to multigroup binocular parallax images
Scene, there is between each scene identical content and different content；
Wherein identical content is, a plane, and each coordinate of the plane in scene coordinate system are respectively provided in each scene
Identical, each plane is divided equally by identical cell, and the calibration point is located in the plane, and the calibration point is located at cell
Vertex；
Wherein different content is, for each scene, the calibration point is located at different cell vertex in the planes.
3. according to claim 1 watch point calibration method attentively, which is characterized in that the left-eye image of every group of binocular parallax images
The left eye and right eye of tester is respectively enterd by Virtual Reality system with eye image, is carrying out left eye blinkpunkt alignment epoch
Between, Virtual Reality system puts right eye visual field black, and during right eye blinkpunkt calibration is carried out, Virtual Reality system is by left eye
Visual field is put black.
4. a kind of blinkpunkt calibrating installation, which is characterized in that including：
Virtual Reality system, for the left-eye image of the binocular parallax images described in power 1 and eye image to be shown to respectively
The left eye and right eye of tester；
Eye feature harvester, during left eye blinkpunkt calibration is carried out, the eye feature picture of collecting test person's left eye；
During carrying out right eye blinkpunkt calibration, the eye feature picture of collecting test person's right eye；And
Middle control module, for the coordinate and multiple left eye eye feature pictures according to multiple calibration points in scene coordinate system,
Calculate left eye mapping relations；According to coordinate and multiple right eye eye feature figures of multiple calibration points in scene coordinate system
Piece calculates right eye mapping relations.
5. blinkpunkt calibrating installation according to claim 4, which is characterized in that binocular camera shooting device is further included, for adopting
Collect the binocular parallax images.
6. blinkpunkt calibrating installation according to claim 5, which is characterized in that further include communication module, the middle control mould
Block can receive the binocular parallax images come by the binocular camera shooting device teletransmission by the communication module, and will
The binocular parallax images send the Virtual Reality system to.
7. a kind of blinkpunkt scaling method, which is characterized in that specifically include：
During the left eye and right eye of tester observe the left-eye image and eye image of one group of binocular parallax images respectively simultaneously,
Obtain the left eye eye feature of tester and right eye eye feature；The left-eye image and eye image of the binocular parallax images by
Same scene obtains, and the scene has a scene coordinate system；
According to left eye eye feature and left eye mapping relations, coordinate of the left eye blinkpunkt in scene coordinate system is calculated；
According to right eye eye feature and right eye mapping relations, coordinate of the right eye blinkpunkt in scene coordinate system is calculated；
According to coordinate and right eye blinkpunkt coordinate in scene coordinate system of the left eye blinkpunkt in scene coordinate system, sentence
Determine coordinate of the blinkpunkt of tester in scene coordinate system and be located at left eye to watch point coordinates attentively and between right eye watches point coordinates attentively.
8. a kind of blinkpunkt caliberating device, which is characterized in that including：
Virtual Reality system, for the left-eye image of binocular parallax images and eye image to be shown to the left side of tester respectively
Eye and right eye；
Eye feature harvester, in left eye observation left-eye image right eye observation eye image simultaneously, for acquiring a left side respectively
The eye feature picture of eye and the eye feature picture of right eye；And
Middle control module, for according to left eye eye feature and left eye mapping relations, calculating left eye blinkpunkt in scene coordinate
Coordinate in system；According to right eye eye feature and right eye mapping relations, right eye blinkpunkt is calculated in scene coordinate system
Coordinate；And coordinate of the blinkpunkt of discriminating test person in scene coordinate system is located at left eye and watches point coordinates and right eye blinkpunkt seat attentively
Between mark.
9. blinkpunkt caliberating device according to claim 8, which is characterized in that binocular camera shooting device is further included, for adopting
Collect the binocular parallax images.
10. blinkpunkt caliberating device according to claim 9, which is characterized in that further include communication module, the middle control mould
Block can receive the binocular parallax images come by the binocular camera shooting device teletransmission by the communication module, and will
The binocular parallax images send the Virtual Reality system to.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810330258.6A CN108259887B (en) | 2018-04-13 | 2018-04-13 | Method and device for calibrating fixation point and method and device for calibrating fixation point |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810330258.6A CN108259887B (en) | 2018-04-13 | 2018-04-13 | Method and device for calibrating fixation point and method and device for calibrating fixation point |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108259887A true CN108259887A (en) | 2018-07-06 |
| CN108259887B CN108259887B (en) | 2020-01-31 |
Family
ID=62748203
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810330258.6A Active CN108259887B (en) | 2018-04-13 | 2018-04-13 | Method and device for calibrating fixation point and method and device for calibrating fixation point |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108259887B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109032351A (en) * | 2018-07-16 | 2018-12-18 | 北京七鑫易维信息技术有限公司 | Watch point function attentively and determines that method, blinkpunkt determine method, apparatus and terminal device |
| CN109044263A (en) * | 2018-07-13 | 2018-12-21 | 北京七鑫易维信息技术有限公司 | Detection method, device, equipment and the storage medium of eye sight line |
| CN109240497A (en) * | 2018-08-28 | 2019-01-18 | 北京航空航天大学青岛研究院 | A kind of eye movement tracking automatic calibrating method in virtual reality scenario |
| CN110399930A (en) * | 2019-07-29 | 2019-11-01 | 北京七鑫易维信息技术有限公司 | A kind of data processing method and system |
| CN112101064A (en) * | 2019-06-17 | 2020-12-18 | 北京七鑫易维科技有限公司 | Sight tracking method, device, equipment and storage medium |
| CN112578901A (en) * | 2019-09-30 | 2021-03-30 | Oppo广东移动通信有限公司 | Eyeball tracking calibration method and related equipment |
| CN113283402A (en) * | 2021-07-21 | 2021-08-20 | 北京科技大学 | Differential two-dimensional fixation point detection method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010259605A (en) * | 2009-05-01 | 2010-11-18 | Nippon Hoso Kyokai <Nhk> | Visual line measuring device and visual line measuring program |
| CN106659380A (en) * | 2014-06-12 | 2017-05-10 | Sr实验室有限公司 | Device and method of calibration for an eye tracker and eye control equipment comprising said calibration device |
| CN107450720A (en) * | 2016-05-31 | 2017-12-08 | Fove股份有限公司 | Line-of-sight detection systems |
| US20170364149A1 (en) * | 2014-12-16 | 2017-12-21 | Koninklijke Philips N.V. | Gaze tracking system with calibration improvement, accuracy compensation, and gaze localization smoothing |
-
2018
- 2018-04-13 CN CN201810330258.6A patent/CN108259887B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010259605A (en) * | 2009-05-01 | 2010-11-18 | Nippon Hoso Kyokai <Nhk> | Visual line measuring device and visual line measuring program |
| CN106659380A (en) * | 2014-06-12 | 2017-05-10 | Sr实验室有限公司 | Device and method of calibration for an eye tracker and eye control equipment comprising said calibration device |
| US20170364149A1 (en) * | 2014-12-16 | 2017-12-21 | Koninklijke Philips N.V. | Gaze tracking system with calibration improvement, accuracy compensation, and gaze localization smoothing |
| CN107450720A (en) * | 2016-05-31 | 2017-12-08 | Fove股份有限公司 | Line-of-sight detection systems |
Non-Patent Citations (2)
| Title |
|---|
| DEVENDER KUMAR等: "Electrooculogram-based virtual reality game control using blink detection and gaze calibration", 《2016 INTERNATIONAL CONFERENCE ON ADVANCES IN COMPUTING, COMMUNICATIONS AND INFORMATICS (ICACCI)》 * |
| 张显勇: "眼动仪注视点定位技术研究", 《中国优秀硕士学位论文全文数据库》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109044263A (en) * | 2018-07-13 | 2018-12-21 | 北京七鑫易维信息技术有限公司 | Detection method, device, equipment and the storage medium of eye sight line |
| WO2020010868A1 (en) * | 2018-07-13 | 2020-01-16 | 北京七鑫易维信息技术有限公司 | Line-of-sight detection method and device, apparatus, and storage medium |
| CN109032351A (en) * | 2018-07-16 | 2018-12-18 | 北京七鑫易维信息技术有限公司 | Watch point function attentively and determines that method, blinkpunkt determine method, apparatus and terminal device |
| CN109032351B (en) * | 2018-07-16 | 2021-09-24 | 北京七鑫易维信息技术有限公司 | Fixation point function determination method, fixation point determination device and terminal equipment |
| CN109240497A (en) * | 2018-08-28 | 2019-01-18 | 北京航空航天大学青岛研究院 | A kind of eye movement tracking automatic calibrating method in virtual reality scenario |
| CN109240497B (en) * | 2018-08-28 | 2021-07-13 | 北京航空航天大学青岛研究院 | Automatic calibration method for eye tracking in virtual reality scene |
| CN112101064A (en) * | 2019-06-17 | 2020-12-18 | 北京七鑫易维科技有限公司 | Sight tracking method, device, equipment and storage medium |
| CN112101064B (en) * | 2019-06-17 | 2024-07-05 | 北京七鑫易维科技有限公司 | Sight tracking method, device, equipment and storage medium |
| CN110399930A (en) * | 2019-07-29 | 2019-11-01 | 北京七鑫易维信息技术有限公司 | A kind of data processing method and system |
| CN110399930B (en) * | 2019-07-29 | 2021-09-03 | 北京七鑫易维信息技术有限公司 | Data processing method and system |
| CN112578901A (en) * | 2019-09-30 | 2021-03-30 | Oppo广东移动通信有限公司 | Eyeball tracking calibration method and related equipment |
| CN113283402A (en) * | 2021-07-21 | 2021-08-20 | 北京科技大学 | Differential two-dimensional fixation point detection method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108259887B (en) | 2020-01-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108259887A (en) | Watch point calibration method and device, blinkpunkt scaling method and device attentively | |
| CN103429139B (en) | Spectacle device with an adjustable field of view and method | |
| CN105812778B (en) | Binocular AR wears display device and its method for information display | |
| US8708490B2 (en) | Method and a device for automatically measuring at least one refractive characteristic of both eyes of an individual | |
| Shih et al. | A novel approach to 3-D gaze tracking using stereo cameras | |
| US11854171B2 (en) | Compensation for deformation in head mounted display systems | |
| US7533989B2 (en) | Sight-line detection method and device, and three-dimensional view-point measurement device | |
| CN106168853A (en) | A kind of free space wear-type gaze tracking system | |
| Hennessey et al. | Noncontact binocular eye-gaze tracking for point-of-gaze estimation in three dimensions | |
| CN106199964B (en) | The binocular AR helmet and depth of field adjusting method of the depth of field can be automatically adjusted | |
| CN113808160B (en) | Sight direction tracking method and device | |
| US20130076884A1 (en) | Method and device for measuring an interpupillary distance | |
| CN104089606B (en) | A kind of free space eye tracking measuring method | |
| CN104090371B (en) | A kind of 3D glasses and 3D display systems | |
| WO2015051606A1 (en) | Locating method and locating system | |
| JP2013228557A (en) | Display device and control method thereof | |
| CN107427208A (en) | For providing the wear-type eye tracking equipment and method of rift-free eye tracking by lens combination | |
| WO2015051605A1 (en) | Image collection and locating method, and image collection and locating device | |
| JP2005185431A (en) | Line-of-sight detection method and line-of-sight detector | |
| CN107991775B (en) | Head-mounted visual equipment capable of tracking human eyes and human eye tracking method | |
| CN108369744A (en) | It is detected by the 3D blinkpunkts of binocular homography | |
| CN109008937A (en) | Method for detecting diopter and equipment | |
| CN112336301B (en) | Strabismus measuring equipment | |
| Wibirama et al. | 3D gaze tracking on stereoscopic display using optimized geometric method | |
| CN107765840A (en) | A kind of Eye-controlling focus method equipment of the general headset equipment based on binocular measurement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |