CN102339124A - Method for controlling cursor - Google Patents
Method for controlling cursor Download PDFInfo
- Publication number
- CN102339124A CN102339124A CN2010102356296A CN201010235629A CN102339124A CN 102339124 A CN102339124 A CN 102339124A CN 2010102356296 A CN2010102356296 A CN 2010102356296A CN 201010235629 A CN201010235629 A CN 201010235629A CN 102339124 A CN102339124 A CN 102339124A
- Authority
- CN
- China
- Prior art keywords
- cursor
- picture
- display device
- block
- blocks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- User Interface Of Digital Computer (AREA)
Abstract
The invention provides a method for controlling a cursor, which is suitable for controlling the cursor on a display device. In the method, a picture of the display device is firstly divided into a plurality of blocks; in addition, an image of a user is captured by utilizing an image capturing device; the position of a sight line is inferred according to the positions of pupils on the image; then a watching block corresponding to the position of the sight line is found out in the picture; and finally, the cursor is moved to the watching block.
Description
Technical field
The invention relates to a kind of method of controlling cursor, and particularly relevant for the method for the auxiliary control cursor of a kind of vision.
Background technology
In the society that scientific and technical information is flourishing now; Because computing machine can be widely used in many-sides such as storage data, multimedia, network, message transmission and document processing; Make computing machine almost become one of all trades and professions basic tool, even reach the stage that every household all has a personal computer at least soon.
In computer system, mouse control is the key player of system operation.The user is through the mouse beacon cursor position in the mobile system relatively, and then chooses the running of specific file archives.Generally speaking, when user's desire use mouse to come moving cursor and do not know cursor be positioned at picture where the time, the user can be earlier significantly rolling mouse use the cursor location of seeking in the picture.Afterwards, the user again rolling mouse cursor is moved to fixed point.
Yet under the monitor resolution condition with higher, the cursor in the picture can diminish, and the user will take the more time in picture, to search cursor.In addition, because the progress of science and technology, the size of display is increasing, and in large-sized display, the user also can relatively increase in the picture searching on-site time of cursor.Thus, the user will increase in the time of seeking cursor and moving cursor, and then reduced the convenience that uses mouse.
Summary of the invention
The present invention provides a kind of method of controlling cursor, imports vision and assists the method for improving the control cursor.
The present invention proposes a kind of method of controlling cursor, is suitable for controlling the cursor on the display device.In the method, the picture with display device is divided into a plurality of blocks earlier.And, utilize image capture unit to capture user's image.According to the pupil position of image, infer the sight line position.And in picture, find out and the corresponding block of watching in sight line position, wherein watch block to be a block wherein.At last, cursor is moved to watch block.
In one embodiment of this invention, above-mentioned picture with display device is divided in the step of a plurality of blocks, can judge the distance of viewing and admiring between user and the display device, again according to the number of viewing and admiring the block of dividing apart from the picture institute desire of decision display device.Perhaps; Can be from expansion display recognition data (the Extended Display Identification Data of display device; EDID) reading displayed device size in, the number of the block of dividing according to the picture institute desire of the size of display device decision display device again.
In one embodiment of this invention, above-mentioned with cursor move to watch block step for cursor is moved to the predeterminated position of watching in the block.
In one embodiment of this invention, when the method for above-mentioned control cursor also is included in the display reminding window, judge whether the corresponding block of watching in sight line position is the centerablock of picture.If watching block is the centerablock of picture, cursor is moved to the predeterminated position of prompt window.
In one embodiment of this invention, the method for above-mentioned control cursor also can be at the time point of carrying out predeterminable event, the track that the record cursor moves.This predeterminable event for example is start.
In one embodiment of this invention, the method for above-mentioned control cursor also can be divided into active area and invalid picture with picture, and active area is divided into a plurality of blocks when open preset file (for example handling file for document).
Based on above-mentioned, the present invention utilizes image capture unit to capture user's image, judges sight line corresponding block of watching in the picture of display device of user again through image processing technique, and cursor is moved viewing area piece so far.In view of the above, but moving cursor quickly not only shorten to be searched the time of cursor near the destination, also shorten the time of moving cursor.
For letting the above-mentioned feature and advantage of the present invention can be more obviously understandable, hereinafter is special lifts embodiment, and cooperates appended graphic elaborating as follows.
Description of drawings
Fig. 1 is the method flow diagram according to the control cursor that one embodiment of the invention illustrated.
Fig. 2 views and admires the synoptic diagram that distance and sized display decide number of blocks according to foundation that one embodiment of the invention illustrated.
Fig. 3 A～Fig. 3 C is the method synoptic diagram that moves according to the control cursor that one embodiment of the invention illustrated.
Fig. 4 is the synoptic diagram according to the control cursor related application that one embodiment of the invention illustrated.
[main element label declaration]
301,401: cursor
403: option
405: prompt window
S105～S125: each step of method of the control cursor of one embodiment of the invention
Embodiment
Generally speaking, the user controls cursor can look for earlier present cursor position on the picture of display device, again according to current position relatively rolling mouse come moving cursor, quite inconvenience for large scale or high-resolution display device.For this reason, the present invention proposes a kind of method of controlling cursor, imports vision and assists the method for improving tradition control cursor.
Fig. 1 is the method flow diagram according to the control cursor that one embodiment of the invention illustrated.Please, in the present embodiment, image capture unit is arranged at display apparatus side, uses the image that captures according to image capture unit and judge that the user watches the opposite position of display device, control moving of cursor by this with reference to Fig. 1.For example, a backup system is set in computer system, through grasping the pupil position of facial contour and human eye, the picture opposite position relation according to user's pupil position and display device goes out the picture position that the user watches with deducibility.Specify as follows.
In order to judge that the user watches the opposite position of display device, in step S105, the picture of display device is divided into a plurality of blocks so that with the user to watch sight line to carry out corresponding.And in step S110, utilize image capture unit acquisition user's image.Then, shown in step S115,, infer user's sight line position according to the pupil position in this image.
Generally speaking, utilize the sight line tracer technique can detect the direction that eyes are watched attentively.In the present embodiment, the pupil position in the image that utilizes image capture unit and captured is inferred the sight line position.For example, utilize technology such as heterochromatic boundary tracking method (limbus tracking), pupil tracing method (pupil tracking), pupil and corneal reflection back tracking method or artificial neural network (artificial neural network).
In step S120, in picture, find out and the corresponding block of watching in sight line position, wherein watch the block wherein block divided of picture for this reason.This is because under general environment for use, the sight line position that the user watched can be accompanied by the place that moving cursor to eyes are seen usually.
In order to increase the coupling accuracy of sight line position and display device, when sized display is big more, the number of blocks that picture is divided also can be many more.In view of the above; When carrying out the division block (step S105 just); Can be from expansion display recognition data (the Extended Display IdentificationData of display device; EDID) reading displayed device size in, the size according to display device determines the block number that the picture institute desire of display device is divided again.
In addition, the user's views and admires apart from the coupling accuracy that also can influence sight line position and display device.When viewing and admiring apart from far away more of user, the block of then dividing is many more; Otherwise, to view and admire apart near more, the block of division is then few more.In the present embodiment, user's judgement of viewing and admiring distance can be judged the distance of viewing and admiring between user and the display device through the image that image capture unit captures the user.For example, through the mode of process identification in the image processing, image comes the distance of viewing and admiring between identification user and the display device thus.Afterwards, just can be according to viewing and admiring the block number that distance decides the picture institute desire of display device to divide.
For instance, Fig. 2 views and admires the synoptic diagram that distance and sized display decide number of blocks according to foundation that one embodiment of the invention illustrated.Please with reference to Fig. 2, in the present embodiment, the trend of the block number among Fig. 2 is not to be to be used for limiting the relation of viewing and admiring between distance and the sized display.Present embodiment is merely the relation that illustrates between sized display and the block number, and the relation of viewing and admiring distance and block number, does not limit its scope with this.
At this, the number of number of blocks can decide by viewing and admiring distance or sized display.If with the size of display device, the size of display device is big more, then the quantity of picture division block is many more; Otherwise the size of display device is more little, and then the quantity of picture division block is few more.In addition, if to view and admire distance, view and admire apart from far away more, then the quantity of picture division block is many more; Otherwise, view and admire apart near more, then the quantity of picture division block is few more.
Obtaining corresponding the watching after the block in user's sight line position, shown in step S125, cursor moved to watch block.For example, can cursor be moved to the predeterminated position of watching in the block (for example middle position).
Below lifting an embodiment again explains.Fig. 3 A～Fig. 3 C is the method synoptic diagram that moves according to the control cursor that one embodiment of the invention illustrated.Please with reference to Fig. 3 A～Fig. 3 C, in Fig. 3 A, cursor 301 is located in optional position in the picture.At this moment, the user wants to click " beginning " of picture lower left.Processing through backup system is divided into some blocks with picture.Shown in Fig. 3 B, picture is divided into 16 blocks in the present embodiment.And backup system watches the pupil position of picture to infer corresponding sight line position according to the user again, and analyzes the corresponding relation of sight line position and display device.The sight line position of supposing the user falls within the lower left block.At this moment, just backup system moves to the lower left block with cursor, shown in Fig. 3 C, use and reduce the not convenient property that the user seeks cursor position.
The method of above-mentioned control cursor of arranging in pairs or groups can be improved the convenience that the user controls cursor, and can derive relevant application.For example, when the user in the operating operation systematic procedure, in misoperation or when message notifying is arranged, system can prompt window occur in the middle of picture, use and inform user's particular message.And, comprise in the prompt window " be/not " or options such as " confirming/cancellation ".At this moment, the user will click wherein an option earlier and close this prompt window.Because when prompt window occurs; The user can be interrupted previous operation, and need cursor be moved to the action that prompt window is closed, therefore in the present embodiment; When the display reminding window, can judge whether the corresponding block of watching in sight line position is the centerablock of picture earlier.If watching block is the centerablock of picture, just cursor is moved to the predeterminated position of prompt window, for example " confirm " the option place.
For instance, Fig. 4 is the synoptic diagram according to the control cursor related application that one embodiment of the invention illustrated.Please with reference to Fig. 4, in the present embodiment, after prompt window 405 occurred, user's first action promptly was the centre that the sight line of eyes is moved to picture.At this moment, backup system occurs and the visual picture of user's eyes central authorities at prompt window, then help the user that cursor 401 is moved to prompt window 405, and preset cursor 401 positions is positioned at option 403.In view of the above, the method for the above-mentioned control cursor of arranging in pairs or groups just can easily be closed prompt window, and then reduces the user and in picture, seek on cursor 401 and moving cursor 401 to the prompt window 405, increases the convenience that cursor is controlled.
Owing to may not be certain the sight line that the user moves eyes at every turn, just must moving cursor to opposite position.Therefore, the method for control cursor of the present invention can be done to combine with user's application program operating.For example, when the user watched film, if moving cursor will cause user's puzzlement, so moving of cursor can be left in the basket at this moment.For example, when the user carries out the full screen broadcast, can temporarily cursor control be closed automatically, make cursor can not change the sight line position and move according to the user.
In addition, when opening a certain preset file, can picture be divided into active area and invalid picture, and active area is carried out the division of block.With preset file is that document is handled file; Handle at document under the operating environment of file; If the displaying ratio of edit page (active area) does not take whole image, then have an invalid picture respectively on the both sides of edit page, just belong to the operating position that can't use.When invalid picture is dropped in user's sight line position, then can cursor not moved so far invalid picture, can avoid unnecessary cursor to move.
In addition, conflict, can close backup system when mouse moves detecting, cursor is controlled to weigh give back mouse for fear of utilizing eye sight line position operation cursor and mouse action cursor to produce.
And, also can import the notion of historical record, at the time point of carrying out a predeterminable event, the track that the record cursor moves also can quicken moving and control of cursor.For example, under the situation of start, the user can move to the lower right corner with cursor usually and close or begin resident program.Perhaps, when carrying out document processing file, can cursor be moved to the upper left corner and click the new file of unlatching or open ancient deed.In such mode of operation, can quicken moving and control of cursor.
In sum, in the foregoing description, utilize the vision of image capture unit auxiliary, learn user's sight line position and the corresponding relation of display device picture, move to the position in the picture that the user watches and control cursor.In view of the above, not only shorten the time of searching cursor, also shorten the time of moving cursor, improve user's operational light target convenience.And, also can under the collocation vision is auxiliary, derive relevant application.
Though the present invention discloses as above with embodiment; Right its is not in order to limit the present invention; Has common knowledge the knowledgeable in the technical field under any; Do not breaking away from the spirit and scope of the present invention, when doing a little change and retouching, so protection scope of the present invention is as the criterion when looking appended the claim scope person of defining.
Claims (9)
1. a method of controlling cursor is suitable for controlling the cursor on the display device, and this method comprises:
The picture of this display device is divided into a plurality of blocks;
Utilize image capture unit acquisition user's image;
According to the pupil position of this image, infer the sight line position;
In this picture, find out and the corresponding block of watching in this sight line position, wherein this watch block be these a plurality of blocks one of them; And
This cursor is moved to this watch block.
2. the method for control cursor according to claim 1 wherein comprises the step that the picture of this display device is divided into these a plurality of blocks:
Judge the distance of viewing and admiring between this user and this display device; And
View and admire the number that distance determines these a plurality of blocks that picture institute desire of this display device is divided according to this.
3. the method for control cursor according to claim 1 wherein comprises the step that the picture of this display device is divided into these a plurality of blocks:
In the expansion display recognition data of this display device, read the size of this display device; And
Determine the number of these a plurality of blocks that the picture institute desire of this display device is divided according to the size of this display device.
4. the method for control cursor according to claim 1 wherein moves to this with this cursor and watches the step of block to comprise:
This cursor is moved to this watch the predeterminated position in the block.
5. the method for control cursor according to claim 1 also comprises:
When the display reminding window, judge that corresponding this in this sight line position watch whether block is the centerablock of this picture; And
If this watches block is the centerablock of this picture, this cursor is moved to this watches the step of block to comprise:
This cursor is moved to the predeterminated position of this prompt window.
6. the method for control cursor according to claim 1 also comprises:
Time point carrying out predeterminable event writes down the track that this cursor moves.
7. the method for control cursor according to claim 6, wherein this predeterminable event is start.
8. the method for control cursor according to claim 1 also comprises:
When file is preset in unlatching, this picture is divided into active area and invalid picture; And
This active area is divided into this a plurality of blocks.
9. the method for control cursor according to claim 8 should preset file be that document is handled file wherein.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010102356296A CN102339124A (en) | 2010-07-22 | 2010-07-22 | Method for controlling cursor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010102356296A CN102339124A (en) | 2010-07-22 | 2010-07-22 | Method for controlling cursor |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102339124A true CN102339124A (en) | 2012-02-01 |
Family
ID=45514892
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010102356296A Pending CN102339124A (en) | 2010-07-22 | 2010-07-22 | Method for controlling cursor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102339124A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103605621A (en) * | 2013-11-26 | 2014-02-26 | 三星电子（中国）研发中心 | Method and device for controlling more than two terminal devices by single input device |
| CN103870097A (en) * | 2012-12-12 | 2014-06-18 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN105765513A (en) * | 2013-11-01 | 2016-07-13 | 索尼公司 | Information processing device, information processing method, and program |
| CN105929954A (en) * | 2016-04-19 | 2016-09-07 | 京东方科技集团股份有限公司 | Cursor control method and apparatus as well as display device |
| CN106325505A (en) * | 2016-08-17 | 2017-01-11 | 传线网络科技（上海）有限公司 | Control method and device based on viewpoint tracking |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1357862A (en) * | 2000-12-06 | 2002-07-10 | 英业达股份有限公司 | Cursor clicking and selecting method and device in windows |
| US20020126090A1 (en) * | 2001-01-18 | 2002-09-12 | International Business Machines Corporation | Navigating and selecting a portion of a screen by utilizing a state of an object as viewed by a camera |
| CN1797299A (en) * | 2004-12-22 | 2006-07-05 | 联想(北京)有限公司 | System and method for controlling cursor of display device |
| CN1889016A (en) * | 2006-07-25 | 2007-01-03 | 周辰 | Eye-to-computer cursor automatic positioning controlling method and system |
| US20080181452A1 (en) * | 2006-07-25 | 2008-07-31 | Yong-Moo Kwon | System and method for Three-dimensional interaction based on gaze and system and method for tracking Three-dimensional gaze |
-
2010
- 2010-07-22 CN CN2010102356296A patent/CN102339124A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1357862A (en) * | 2000-12-06 | 2002-07-10 | 英业达股份有限公司 | Cursor clicking and selecting method and device in windows |
| US20020126090A1 (en) * | 2001-01-18 | 2002-09-12 | International Business Machines Corporation | Navigating and selecting a portion of a screen by utilizing a state of an object as viewed by a camera |
| CN1797299A (en) * | 2004-12-22 | 2006-07-05 | 联想(北京)有限公司 | System and method for controlling cursor of display device |
| CN1889016A (en) * | 2006-07-25 | 2007-01-03 | 周辰 | Eye-to-computer cursor automatic positioning controlling method and system |
| US20080181452A1 (en) * | 2006-07-25 | 2008-07-31 | Yong-Moo Kwon | System and method for Three-dimensional interaction based on gaze and system and method for tracking Three-dimensional gaze |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103870097A (en) * | 2012-12-12 | 2014-06-18 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN105765513A (en) * | 2013-11-01 | 2016-07-13 | 索尼公司 | Information processing device, information processing method, and program |
| CN105765513B (en) * | 2013-11-01 | 2020-02-21 | 索尼公司 | Information processing apparatus, information processing method, and program |
| CN103605621A (en) * | 2013-11-26 | 2014-02-26 | 三星电子（中国）研发中心 | Method and device for controlling more than two terminal devices by single input device |
| CN105929954A (en) * | 2016-04-19 | 2016-09-07 | 京东方科技集团股份有限公司 | Cursor control method and apparatus as well as display device |
| CN105929954B (en) * | 2016-04-19 | 2019-10-18 | 京东方科技集团股份有限公司 | It is a kind of control cursor method and device, display equipment |
| CN106325505A (en) * | 2016-08-17 | 2017-01-11 | 传线网络科技（上海）有限公司 | Control method and device based on viewpoint tracking |
| CN106325505B (en) * | 2016-08-17 | 2019-11-05 | 传线网络科技（上海）有限公司 | Control method and device based on viewpoint tracking |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9411416B2 (en) | Computer device operable with user's eye movement and method for operating the computer device | |
| US20120280898A1 (en) | Method, apparatus and computer program product for controlling information detail in a multi-device environment | |
| US20060256083A1 (en) | Gaze-responsive interface to enhance on-screen user reading tasks | |
| CN102339124A (en) | Method for controlling cursor | |
| KR101455200B1 (en) | Learning monitering device and method for monitering of learning | |
| KR20150096319A (en) | Gesture recognition device and method of controlling gesture recognition device | |
| CN107408100A (en) | Sight is used for automatic page turning | |
| CN103294194A (en) | Translation method and system based on eyeball tracking | |
| CN106873774A (en) | interaction control method, device and intelligent terminal based on eye tracking | |
| CN103577094A (en) | Electronic apparatus and method | |
| CN111656313A (en) | Screen display switching method, display device and movable platform | |
| CN103631365A (en) | Terminal input control method and device | |
| CN105824491A (en) | Split screen processing method and device in mobile equipment | |
| Bovo et al. | Detecting errors in pick and place procedures: detecting errors in multi-stage and sequence-constrained manual retrieve-assembly procedures | |
| CN113254096A (en) | Timing control using method and device and electronic equipment | |
| CN206584114U (en) | A kind of low power-consumption intelligent AR devices and intelligence AR glasses | |
| CN114092985A (en) | Terminal control method, device, terminal and storage medium | |
| KR102175853B1 (en) | Method for controlling operation and an electronic device thereof | |
| KR102181897B1 (en) | Device and method for mobile tooltip using eye detecting | |
| CN102063256B (en) | Method and device for moving/stretching window | |
| CN114821753B (en) | Eye movement interaction system based on visual image information | |
| Raja et al. | Voice Assistant and Gesture Controlled Virtual Mouse using Deep Learning Technique | |
| KR101413558B1 (en) | Analysis device of user multi-intent, method of analysis user multi-intent | |
| CN110007748B (en) | Terminal control method, processing device, storage medium and terminal | |
| CN107168517A (en) | A kind of control method and device of virtual reality device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20120201 |
|
| WD01 | Invention patent application deemed withdrawn after publication |