CN102289328A

CN102289328A - Human-computer interaction method and device based on handheld camera

Info

Publication number: CN102289328A
Application number: CN2011102204279A
Authority: CN
Inventors: 王珂
Original assignee: Individual
Current assignee: Individual
Priority date: 2011-08-03
Filing date: 2011-08-03
Publication date: 2011-12-21

Abstract

The invention discloses a human-computer interaction method and a human-computer interaction device based on a handheld camera. Different from the conventional human-computer interaction method for arranging a camera at an equipment end (for example, a computer display or a television screen), the human-computer interaction method comprises the following steps that: a user uses the handheld camera as point caching equipment to replace a mouse and as a way of human-computer interaction; and the handheld camera acquires a video image by facing the television screen (or the computer display), acquires the accurate position of a screen quadrangle in the video image in real time by an image analysis method, calculates accurate coordinates of an intersection point of an optical axis of the camera and a screen plane relative to the screen, and transmits the coordinates to the equipment end in a wireless mode, wherein the coordinates can move along with the movement of the camera in one hand of the user. The human-computer interaction device can serve as an intelligent remote controller to replace the conventional television remote controller and can serve as human-computer interaction equipment which is applicable to electronic games.

Description

A kind of man-machine interaction method and device based on hand-held camera

Technical field

The present invention relates to the man-machine interaction method field, specifically, relate to the field that the image analysis method that utilizes machine vision carries out man-machine interaction.

Background technology

Man-machine interaction mode commonly used comprises some taking equipments such as the mouse touch screen of computing machine, and electronic game operating rod etc. all are to utilize laser infrared or mechanical method realizes the location.When the user need choose target on display device such as the screen, then need to move corresponding some taking equipment and produce relative physical displacement, thereby cursor is moved on screen.On the other hand, use infrared mode and televisor to carry out alternately in TV remote controller, the user by simple point get on the telepilot such as waiting button to realize choosing of on-screen menu up and down, the televisor of function complexity needs more button corresponding with it, and this makes that the size of telepilot is increasing.Corresponding electronic game on the televisor even use equipment such as game paddle, also can't realize accurate location to screen content as computer mouse, just can't utilize such as the aiming action of shooting game to have game paddle now and finish.

Summary of the invention

The objective of the invention is to utilize the principle of machine vision that a kind of method and device of accurate positioning screen coordinate are provided for televisor and computer user.By being installed in the real time video collection equipment on the telepilot, for increasing, televisor gets function as the point of computer mouse, promptly not only can choose on-screen menu by the directionkeys on the telepilot, and can realize the screen coordinate location, increase needs supports (as the aiming action of shooting game) of the accurately operation of location image contents to those, and this traditional remote controller was short of just.The present invention not only is applicable to televisor, also can be the computer user a kind of interactive operation mode that substitutes existing electronic game bar is provided.

The present invention comprises one and the telepilot and the calculating television screen position of video capture device (camera) has been installed and has calculated the computing method of cursor in the exact position.Algorithm is installed in the telepilot with embedded program or special digital chip form, and telepilot is realized communicating by letter transmission of coordinate information by infrared mode and main equipment (televisor) as the slave of televisor.When carrying out the electronic game operation, telepilot can be designed to other as rifle (shooting game) bearing circle shapes such as (car race games).Be used in ordinary individual's computing machine as this equipment, algorithm can be realized with the computing machine form of software.

Method flow is seen block diagram (Fig. 1), and corresponding concrete technology contents is as follows:

Step 1. camera is gathered video image towards TV screen (or computer screen), and the video image frame per second is a per second 15-30 frame, and resolution is 320 * 240 pixels;

Step 2. detects in every two field picture whether have television screen, if having, changes step 3 over to; Otherwise, withdraw from the detection of this two field picture.

Step 3. computed image central point drops on the position in television screen zone.Calculate earlier camera with the relative position of TV screen (or computer screen), calculate the intersection point on cam lens optical axis and TV screen (or computer screen) plane then, export as cursor position the left back exact position of this intersection point on TV screen (or computer screen) that obtain.

Wherein, step 1 realizes the real-time video collection, uses cheap computing machine to get final product with the CMOS camera.The visual field of cam lens need cover the four edges of television screen as far as possible, in order to realize this point, guarantee that camera and television screen are in suitable distance, perhaps use wide-angle lens.For the picture distortion that wide-angle lens produces, can correct by image processing method.

Step 2 and step 3 relate to the concrete enforcement of machine vision and image analysis method, will be described at next joint.

Specific implementation method

The committed step of this invention is to utilize the algorithm of graphical analysis, obtains the particular location of TV screen (or computer screen) on picture.Here two kinds of methods have been used.

Method one: the four edges E1, E2, E3 and the E4 that detect TV screen.Basic idea is to utilize Hough change detection straight line.At first use Hough conversion, image transitions in the Hough space, is analyzed the Hough space again, obtain most likely 4 straight lines of TV screen (or computer screen) based on image gradient.

Method two: based on the method such as the active shape method (ASM) of shape Statistics.This is a kind of method based on off-line learning, obtains statistical information about screen shape and colouring information according to the sampled images of a large amount of current model television screens, detects the quadrilateral that meets statistical model in the image most and exports as TV screen.

The quadrilateral of the TV screen that image acquisition arrives (or computer screen) is the mapping of rectangle template on the video camera imaging plane of the TV screen (or computer screen) of standard, and this mapping is a kind of transmission conversion.The relative space position that can calculate TV screen (or computer screen) and video camera according to the quadrilateral image and the rectangle template of TV screen (or computer screen).

If detected quadrilateral is E ₁, E ₂, E ₃And E ₄, can obtain tetragonal 4 summits according to them is P ₁, P ₂, P ₃, and P ₄If 4 summits of screen rectangle template are: T ₁, T ₂, T ₃, and T ₄, coordinate is respectively (establishing true origin locates in template center):

T ₁=(Wt/2 ,-Ht/2), T ₂=(Wt/2 ,-Ht/2), T ₃=(Wt/2, Ht/2) and T ₄=(Wt/2, Ht/2)

Wherein Wt and Ht are respectively the wide and high of screen template, represent with pixel.T ₁, T ₂, T ₃, T ₄And P ₁, P ₂, P ₃, P ₄Mapping relations be expressed in matrix as:

(1)

Wherein H is that 3 * 3 list should (Homography) matrix:

(2)

Some Pi(xp on such two groups of planes, yp) and Ti (xt, yt) between the transmission conversion of available 8 coefficients connect:

(3)

8 unknowm coefficient h0 ~ h7 can be according to known point Pi and Ti(i=1 ~ 4) find the solution.

The cam lens photocentre Pc Intersecting point coordinate with TV screen (or computer screen) plane Tc , can be by calculating with co-relation: because

So, have

This is equivalent to the inverse operation of the derivation of equation (3). Pc Coordinate can be got picture center (as Fig. 2) usually, and coordinate figure Tc sees Fig. 3 in position on screen.

Description of drawings

Fig. 1. invention method therefor FB(flow block);

Fig. 2. the picture of the TV that hand-held camera photographs (or computer) screen, the picture centre mark is '+';

Fig. 3. the relative position of Fig. 2 picture center on objective plane (screen among Fig. 2) marks with '+'.

Claims

1. man-machine interaction method and device based on a hand-held camera is characterized in that, may further comprise the steps:

(1) cam device is in handheld device such as TV remote controller, towards TV screen or the real-time images acquired of computer screen; (2) screen is accurately located, and calculates the corresponding relation of screen template and screen position, i.e. matrix H in real time;

(3) utilize the inverse operation of H, ask the relative coordinate of central point on image template of images acquired.