CN110286759A - The adaptive amplification display method of terminal, system and terminal based on eye tracking - Google Patents
The adaptive amplification display method of terminal, system and terminal based on eye tracking Download PDFInfo
- Publication number
- CN110286759A CN110286759A CN201910528214.9A CN201910528214A CN110286759A CN 110286759 A CN110286759 A CN 110286759A CN 201910528214 A CN201910528214 A CN 201910528214A CN 110286759 A CN110286759 A CN 110286759A
- Authority
- CN
- China
- Prior art keywords
- user
- terminal
- region
- adaptive
- interest
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/19—Sensors therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/048—Indexing scheme relating to G06F3/048
- G06F2203/04806—Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Ophthalmology & Optometry (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
A kind of adaptive amplification display method of terminal based on eye tracking provided by the invention, system and terminal, wherein, adaptive amplification display method includes the terminal with front camera, blinkpunkt of the user on terminal display screen is obtained in real time, user's region-of-interest is partitioned into according to the position of the blinkpunkt, the user's region-of-interest being partitioned into adaptively is amplified, and carries out real-time display in the specific position of terminal display screen.The present invention is to obtain user's region-of-interest according to the track that user pupil center moves using visual trace technology; user's region-of-interest is subjected to adaptive amplification display; and touching manipulation can be carried out in adaptive amplified region; so that user can more clearly see the content of display clearly; manipulation is also more flexible accurate; enhance visual experience when user's browsing terminal shows picture; the interaction being also beneficial between user and terminal simultaneously; especially for middle-aged and elderly users, usage experience is significantly promoted.
Description
Technical field
The invention belongs to the man-machine mutual friendship technical field of multimedia more particularly to a kind of terminal based on eye tracking are adaptive
Amplification display method, system and terminal.
Background technique
With the explosive growth of mobile Internet market, requirement of the user to application experience is also higher and higher.And move end
The characteristics of end is due to portability to be met and flexibility, the size for even touching its display screen of extra large screen terminal entirely also tend to
It is limited, therefore many inconvenience can be brought to user.For example, carrying out online Taobao with terminal device or checking network map
When, user need to see clearly commodity certain details or certain specific positions, and the terminal display screen of finite size is unable to satisfy
The demand that user sees clearly；In another example some users like seeing e-book, novel, the word of secret fiber crops fiber crops, it appears that take very much painstakingly are seen
Eyes.In addition, such as text inputs, user needs accurately and quickly to carry out text input, but due to the limitation of screen size
So that inputting in actual operation relatively slow and being easy error；Especially present smart phone middle-aged and elderly users also increasingly increase, right
It for them, can maximumlly show, them can be allowed to have better user experience.
In view of the above problems, a kind of display adapting oneself that regional area amplification is carried out according to user's focus is needed,
The demand for experience that people can preferably be met, preferably serves user.
Summary of the invention
Place in view of above-mentioned deficiencies of the prior art, the purpose of the present invention is to provide a kind of terminals based on eye tracking
Adaptive amplification display method, system and terminal, it is intended to which solution is in the prior art adaptively amplified user's region-of-interest
It has been shown that, in order to which user clearer can see the content of display clearly.
In order to achieve the above object, this invention takes following technical schemes:
A kind of adaptive amplification display method of terminal based on eye tracking leads to including the terminal with front camera
It crosses the front camera and obtains blinkpunkt of the user on terminal display screen in real time, be partitioned into according to the position of the blinkpunkt
User's region-of-interest is adaptively amplified the user's region-of-interest being partitioned into, and in the certain bits of terminal display screen
It sets and carries out real-time display and manipulation.
For improvement of the invention, the picture of front camera acquisition user's sight movement is calculated by eye tracking
Method analyzes the feature of user's sight motion profile, to obtain the position of user's blinkpunkt on terminal display screen.
For improvement of the invention, according to relative position of the blinkpunkt in display screen, institute after adaptively amplifying
User's region-of-interest real-time display is stated in suitable region.
For improvement of the invention, side of the display area without departing from display screen after user's region-of-interest adaptively amplifies
Boundary.
For improvement of the invention, adaptive amplified user's region-of-interest is manipulated, and is mapped to without adaptive
On the actual coordinate that should amplify.
In addition, the present invention also provides a kind of adaptive amplifying display systems of terminal, including
Analytical unit, the analytical unit are used to analyze the feature of user's sight motion profile；
Acquiring unit, the acquiring unit are used for the feature according to user's sight motion profile, obtain user in real time
Blinkpunkt on terminal display screen；
Cutting unit, the cutting unit are used to be partitioned into user's region-of-interest according to the position of the blinkpunkt；
Display unit, the display unit are used to adaptively be amplified the user's region-of-interest being partitioned into, and
Real-time display and manipulation are carried out in the specific position of terminal display screen.
Map unit, the map unit are mapped to for manipulating adaptive amplified user's region-of-interest
On not enlarged actual coordinate.
It further include adjustment unit for improvement of the invention, the adjustment unit is for adjusting adaptive amplified institute
State the coordinate position of user's region-of-interest.
In addition, the present invention also provides a kind of terminal, including terminal body, the terminal body includes display screen and preposition takes the photograph
As head, the terminal body adaptively amplifies display side using the terminal based on eye tracking described in any of the above-described technical solution
Method；Alternatively, the terminal body is built-in with the adaptive amplifying display system of terminal described in any of the above-described technical solution.
For improvement of the invention, the lamp that can emit infrared light supply is additionally provided on the terminal body, before the lamp is used for
It sets camera and obtains user's sight track in external insufficient light.
Compared with prior art, beneficial effects of the present invention:
A kind of adaptive amplification display method of terminal based on eye tracking provided by the invention, system and terminal, side
Method includes: to obtain blinkpunkt of the user on terminal display screen in real time by the front camera, according to the blinkpunkt
Position is partitioned into user's region-of-interest, and the user's region-of-interest being partitioned into adaptively is amplified, and shows in terminal
The specific position of screen carries out real-time display and manipulation.The present invention is the track moved using visual trace technology according to eyes of user
User's region-of-interest is obtained, user's region-of-interest is subjected to adaptive amplification display, and can be in adaptive amplified area
Domain carry out touching manipulation, so as to user can be clearer the content for seeing display clearly, manipulation also it is more flexible precisely, enhance user
Browsing terminal shows visual experience when picture, while being also beneficial to the interaction between user and terminal, especially to person in middle and old age
User significantly promotes usage experience.
Detailed description of the invention
Fig. 1 is the flow diagram of the adaptive amplification display method of terminal based on eye tracking in the present invention；
Fig. 2 is the schematic block diagram of the adaptive amplifying display system of terminal in the present invention；
Specific embodiment
To make the purpose of the present invention, technical solution and effect clearer, clear and definite, right as follows in conjunction with drawings and embodiments
The present invention is further described.It should be appreciated that described herein, specific examples are only used to explain the present invention, is not used to
Limit the present invention.
As shown in Fig. 1, the adaptive amplification display method of a kind of terminal based on eye tracking provided by the invention, including
Terminal with front camera, is achieved by the steps of:
S10 obtains blinkpunkt of the user on terminal display screen by the front camera in real time；
It is understood that the picture of front camera acquisition user's sight movement, passes through eye tracking algorithm point
The feature for analysing user's sight motion profile, to obtain the position of user's blinkpunkt on terminal display screen.
It should be noted that analysis user's sight motion profile is characterized in acquiring user's eye according to by front camera
The picture of eyeball, when eyes of user is look at different location, the position of pupil center is to be handled in variation by picture, according to
Pupil center location determines the position of user's sight blinkpunkt at the terminal.
S20 is partitioned into user's region-of-interest according to the position of the blinkpunkt；
It should be noted that the relative position according to the blinkpunkt in display screen, described after and adaptively amplifying
User's region-of-interest real-time display is in suitable region；
User feels the acquisition and segmentation of region-of-interest, we get the coordinate (x, y) of blinkpunkt first, and with this coordinate
For origin, rectangular, round or other irregular shapes region A are extended to the outside, and region A is adaptively amplified.
S30 is adaptively amplified the user's region-of-interest being partitioned into；
It is understood that user's region-of-interest adaptively amplify after display area without departing from display screen boundary.
S40 carries out real-time display and manipulation in the specific position of terminal display screen.
It is understood that it is adaptive to user's region-of-interest that one piece of specific region is arranged on the terminal display screen
It is shown after should amplifying, for example, be user's region-of-interest to several regions of external radiation using the blinkpunkt as basic point, then with
Certain point in user's region-of-interest is that basic point (can be the blinkpunkt, be also possible in user's region-of-interest
Any point) adaptively amplified；
For example, for example, to be shown three typesettings on screen female, first row is Q, W, E, R, T respectively from left to right, the
Two rows are A, S, D, F, G respectively from left to right, and third row is Z, X, C, V, B respectively from left to right, pass through the front camera
Obtain blinkpunkt of the user on terminal display screen in real time, and the blinkpunkt be on D, user's region-of-interest with D to
External radiation, user's region-of-interest can be set to W, E, R of first row, X, C, V of S, D, F and third row of second row,
That is centered on D, adaptive magnification region is a rectangle；Certainly, the adaptive magnification region is not limited to
For rectangle, it is also possible to circle or other irregular shapes, for example, using the certain point in some pattern as blinkpunkt, with this
Certain point is that basic point gives off user's region-of-interest outward, and is adaptively amplified user's region-of-interest.
It should be noted that after adaptively amplify to user's region-of-interest, it is adaptive amplified a certain
The coordinate and actual coordinate of point may not be overlapped, and at this moment, adaptive amplified certain point is mapped to reality by mapping techniques
On the coordinate of border.
The above method is obtained by terminal front camera, using visual trace technology according to the track that eyes of user moves
User's region-of-interest is carried out adaptive amplification display by user's region-of-interest, and can adaptive amplified region into
Row touching manipulation, so as to user can be clearer the content for seeing display clearly, manipulation also it is more flexible precisely, enhancing user browsing
Visual experience when terminal display, while being also beneficial to the interaction between user and terminal, especially to middle-aged and elderly users,
Significantly promote usage experience.
Referring to fig. 2, in addition, the present invention also provides a kind of adaptive amplifying display systems 100 of terminal, including
Analytical unit 101, the analytical unit 101 are used to analyze the feature of user's sight motion profile；
Acquiring unit 102, the acquiring unit 102 are used for the feature according to user's sight motion profile, obtain in real time
Take blinkpunkt of the family on terminal display screen；
It is understood that the picture of front camera acquisition user's sight movement, passes through eye tracking algorithm point
The feature for analysing user's sight motion profile, to obtain the position of user's blinkpunkt on terminal display screen；
Cutting unit 103, the cutting unit 103, which is used to be partitioned into user according to the position of the blinkpunkt, pays close attention to area
Domain；
It should be noted that be user's region-of-interest to several regions of external radiation using the blinkpunkt as basic point, then
(it can be the blinkpunkt by basic point of the certain point in user's region-of-interest, be also possible to user's region-of-interest
In any point) adaptively amplified.
Display unit 104, the display unit 104 is for adaptively being put the user's region-of-interest being partitioned into
Greatly, and in the specific position of terminal display screen real-time display is carried out.
Map unit 105, the map unit 105 are used to manipulate adaptive amplified user's region-of-interest, and
It is mapped on not enlarged actual coordinate, to be manipulated to terminal.
It should be noted that after adaptively amplify to user's region-of-interest, it is therein after adaptive amplification
The coordinate and actual coordinate of certain point may not be overlapped, and at this moment, be mapped adaptive amplified certain point by mapping techniques
Onto actual coordinate, the manipulation to adaptive magnification region is realized.
It preferably, further include adjustment unit 106, the adjustment unit 106 is for adjusting the adaptive amplified use
The coordinate position of family region-of-interest.
Position i.e. according to user's region-of-interest at the terminal adjusts the adaptive display of amplified region at the terminal
Position makes it be shown in more particularly suitable region.
For example, the focus got, on the side of the screen, adaptive amplification is shown on the screen, and
User's region-of-interest is adjusted among screen by the adjustment unit 106.
In addition, the present invention also provides a kind of terminal, including terminal body, the terminal body includes display screen and preposition takes the photograph
As head, the terminal body adaptively amplifies display side using the terminal based on eye tracking described in any of the above-described technical solution
Method；Alternatively, the terminal body is built-in with the adaptive amplifying display system 100 of terminal described in any of the above-described technical solution.
Preferably, the lamp that can emit infrared light supply is additionally provided on the terminal body, the lamp exists for front camera
User's sight track is obtained when extraneous light deficiency.
Increase by two lamps that can emit infrared light supply at the terminal, it is small to the eye injury of user, and before capable of accurately assisting
It sets camera and obtains user's sight track in external insufficient light, and assist being accurately positioned user's watching attentively on a display screen
Point.
It, can according to the technique and scheme of the present invention and its hair it is understood that for those of ordinary skills
Bright design is subject to equivalent substitution or change, and all these changes or replacement all should belong to the guarantor of appended claims of the invention
Protect range.
Claims (9)
1. a kind of adaptive amplification display method of terminal based on eye tracking, special including the terminal with front camera
Sign is: blinkpunkt of the user on terminal display screen is obtained in real time by the front camera, according to the blinkpunkt
Position is partitioned into user's region-of-interest, and the user's region-of-interest being partitioned into adaptively is amplified, and shows in terminal
The specific position of screen carries out real-time display and manipulation.
2. the adaptive amplification display method of the terminal according to claim 1 based on eye tracking, it is characterised in that: described
Front camera acquires the picture of user's sight movement, and the feature of user's sight motion profile is analyzed by eye tracking algorithm,
To obtain the position of user's blinkpunkt on terminal display screen.
3. the adaptive amplification display method of the terminal according to claim 1 based on eye tracking, it is characterised in that: according to
Relative position of the blinkpunkt in display screen, user's region-of-interest real-time display is suitable after adaptively amplifying
In region.
4. the adaptive amplification display method of the terminal according to claim 3 based on eye tracking, it is characterised in that: described
Boundary of the display area without departing from display screen after user's region-of-interest adaptively amplifies.
5. the adaptive amplification display method of the terminal according to claim 1 based on eye tracking, it is characterised in that: manipulation
Adaptive amplified user's region-of-interest, and be mapped on the actual coordinate without adaptively amplifying.
6. a kind of adaptive amplifying display system of terminal, it is characterised in that: including
Analytical unit, the analytical unit are used to analyze the feature of user's sight motion profile；
Acquiring unit, the acquiring unit are used for the feature according to user's sight motion profile, obtain user in real time at end
Hold the blinkpunkt on display screen；
Cutting unit, the cutting unit are used to be partitioned into user's region-of-interest according to the position of the blinkpunkt；
Display unit, the display unit are used to adaptively be amplified the user's region-of-interest being partitioned into, and at end
The specific position of display screen is held to carry out real-time display and manipulation.
Map unit, the map unit for manipulating adaptive amplified user's region-of-interest, and be mapped to without
On the actual coordinate of amplification.
7. the adaptive amplifying display system of terminal according to claim 6, it is characterised in that: it further include adjustment unit, institute
Adjustment unit is stated for adjusting the coordinate position of adaptive amplified user's region-of-interest.
8. a kind of terminal, including terminal body, the terminal body includes display screen and front camera, it is characterised in that: institute
It is adaptive to state the terminal that terminal body applies described in any one of claim 1-5 claim based on eye tracking
Amplification display method；Alternatively, the terminal body is built-in with the adaptive amplifying display system of terminal described in claim 6 or 7.
9. terminal according to claim 8, it is characterised in that: infrared light supply can be emitted by being additionally provided on the terminal body
Lamp, the lamp obtain user's sight track in external insufficient light for front camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910528214.9A CN110286759A (en) | 2019-06-18 | 2019-06-18 | The adaptive amplification display method of terminal, system and terminal based on eye tracking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910528214.9A CN110286759A (en) | 2019-06-18 | 2019-06-18 | The adaptive amplification display method of terminal, system and terminal based on eye tracking |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110286759A true CN110286759A (en) | 2019-09-27 |
Family
ID=68004075
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910528214.9A Pending CN110286759A (en) | 2019-06-18 | 2019-06-18 | The adaptive amplification display method of terminal, system and terminal based on eye tracking |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110286759A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111078044A (en) * | 2019-11-22 | 2020-04-28 | 深圳传音控股股份有限公司 | Terminal interaction method, terminal and computer storage medium |
| CN116844219A (en) * | 2023-06-29 | 2023-10-03 | 深圳卓创智能科技有限公司 | Display method, device, equipment and storage medium for preventing eyestrain |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102834789A (en) * | 2010-04-16 | 2012-12-19 | 高通股份有限公司 | Apparatus and methods for dynamically correlating virtual keyboard dimensions to user finger size |
| CN103164163A (en) * | 2013-03-27 | 2013-06-19 | 东莞宇龙通信科技有限公司 | Display adjustment method and display adjustment device |
| CN105446673A (en) * | 2014-07-28 | 2016-03-30 | 华为技术有限公司 | Screen display method and terminal device |
-
2019
- 2019-06-18 CN CN201910528214.9A patent/CN110286759A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102834789A (en) * | 2010-04-16 | 2012-12-19 | 高通股份有限公司 | Apparatus and methods for dynamically correlating virtual keyboard dimensions to user finger size |
| CN103164163A (en) * | 2013-03-27 | 2013-06-19 | 东莞宇龙通信科技有限公司 | Display adjustment method and display adjustment device |
| CN105446673A (en) * | 2014-07-28 | 2016-03-30 | 华为技术有限公司 | Screen display method and terminal device |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111078044A (en) * | 2019-11-22 | 2020-04-28 | 深圳传音控股股份有限公司 | Terminal interaction method, terminal and computer storage medium |
| CN111078044B (en) * | 2019-11-22 | 2024-02-02 | 深圳传音控股股份有限公司 | Terminal interaction method, terminal and computer storage medium |
| CN116844219A (en) * | 2023-06-29 | 2023-10-03 | 深圳卓创智能科技有限公司 | Display method, device, equipment and storage medium for preventing eyestrain |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| RU2672502C1 (en) | Device and method for forming cornea image | |
| US10671156B2 (en) | Electronic apparatus operated by head movement and operation method thereof | |
| US20180007255A1 (en) | Image capture systems, devices, and methods that autofocus based on eye-tracking | |
| CN105472174A (en) | Intelligent eye protecting method achieved by controlling distance between mobile terminal and eyes | |
| KR102172988B1 (en) | Device and method for eyeliner-wearing guide | |
| US20200387215A1 (en) | Physical input device in virtual reality | |
| USD856328S1 (en) | Wearable artificial intelligence (AI) data processing, augmented reality, virtual reality, and mixed reality communication eyeglass including mobile phone and mobile computing via virtual touch screen gesture and eye tracking control and neuron command | |
| US20190339922A1 (en) | Information processing apparatus, information processing method, and information processing system | |
| KR101502085B1 (en) | A gesture recognition input method for glass type display device | |
| CN109155053B (en) | Information processing apparatus, information processing method, and recording medium | |
| CN104036270B (en) | A kind of instant automatic translation device and method | |
| CN110286759A (en) | The adaptive amplification display method of terminal, system and terminal based on eye tracking | |
| CN109948435A (en) | Sitting posture prompting method and device | |
| CN107729871A (en) | Infrared light-based human eye movement track tracking method and device | |
| US20180004287A1 (en) | Method for providing user interface through head mounted display using eye recognition and bio-signal, apparatus using same, and computer readable recording medium | |
| Salunkhe et al. | A device controlled using eye movement | |
| US20240077937A1 (en) | Devices, methods, and graphical user interfaces for controlling avatars within three-dimensional environments | |
| US10398855B2 (en) | Augmented reality based injection therapy | |
| Chen et al. | Real-time eye localization, blink detection, and gaze estimation system without infrared illumination | |
| JP6483514B2 (en) | Wearable device, control method, and control program | |
| CN107133008A (en) | The method of mobile terminal adjust automatically output | |
| US11682045B2 (en) | Augmented reality advertisements on objects | |
| Kocejko et al. | Eye tracking within near-to-eye display | |
| CN107436675A (en) | A kind of visual interactive method, system and equipment | |
| CN103713387A (en) | Electronic device and acquisition method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190927 |