Embodiment
For the purpose, technical scheme and the advantage that make the utility model embodiment is clearer, below in conjunction with embodiment and accompanying drawing, embodiment is described in further details to the utility model.At this, illustrative examples of the present utility model and explanation thereof are used to explain the utility model, but not as to qualification of the present utility model.
The composition frame chart of a kind of multiple goal treating apparatus that Fig. 1 provides for the utility model embodiment, please refer to Fig. 1, this device comprises: luminophor 11, at least two image acquisition units 12, cluster and filter unit 13, multiple goal separative elements 14, intersect judgement unit 15 and cursor offset control unit 16, wherein:
Luminophor 11 is used to launch light, so that the refractive body on the telepilot can reflect this light, so that the image acquisition units 12 of present embodiment can be obtained with the consecutive image information of the telepilot of thing as a token of of the refractive body on this telepilot.
In the present embodiment, luminophor 11 can be the infraluminescence body, with the emission infrared light; Can be the lasing fluorescence body, with emission laser; Can be visible light emitters also, with visible emitting, present embodiment not with this as restriction.
In the present embodiment, luminophor 11 can be arranged at the reflective position of refractive body that makes things convenient on the telepilot, for example be arranged at around the display screen of terminal of multiple goal treating apparatus of carrying present embodiment, perhaps be arranged at image acquisition units 12 around, present embodiment not with this as restriction.
Each image acquisition units 12 is used to obtain with the refractive body consecutive image information of the telepilot of thing as a token of.
In the present embodiment, this image acquisition units 11 can be a camera, to obtain the image of telepilot by the mode of taking, to obtain its consecutive image information.It can pass through existing any camera chip, and for example Mage-Sensor realizes.
In the present embodiment, telepilot can be one or more.This telepilot is provided with refractive body as moving target, so that camera is known the movement locus of telepilot according to the movement locus of the refractive body that has reflected light, and efficiency confirmed in view of the above target.Wherein, refractive body is made by reflectorized material, need not any power source, and can make different shape as required, satisfies the 3D pattern.
Wherein, consecutive image information is meant the set of diagrams picture that camera photographs in the given time, for example takes 30 two field pictures in 1 second.
In the present embodiment, because a more than camera is set, and each camera all can be gathered the consecutive image information of telepilot, and therefore, corresponding each camera can obtain the consecutive image information of one group of telepilot.
Cluster and filter unit 13 are used for the described consecutive image information that gets access to according to each image acquisition units, confirm the effective target of every two field picture in the described consecutive image that each image acquisition units gets access to, and calculate the center of gravity of each effective target.
In the present embodiment, owing to may have a plurality of targets in the image that camera photographs, for example the target that the light that reflects of the refractive body on telepilot forms, also may have other static or moving targets, the target that forms of the light that sends of sunlight, the light that fixing shot-light sends, the light lamp that rocks or the like light source for example, this cluster and filter unit 13 are used for the consecutive image according to each camera shooting, judge the existence of each effective target.
In the present embodiment, the consecutive image information that corresponding each image acquisition units 12 collects, cluster and filter unit 13 all will be confirmed the effective target of every two field picture in this consecutive image, and calculate the center of gravity of each effective target, therefore, after handling through cluster and filter unit 13, obtain the center of gravity of the effective target of every two field picture in one group of consecutive image of corresponding each image acquisition units 12.
According to an embodiment of present embodiment, this cluster and filter unit 13 comprise: clustering processing module 131, motion detection block 132, noise elimination module 133 and computing module 134, wherein:
Clustering processing module 131 is used for the described consecutive image information that gets access to according to each image acquisition units 12, and the target with same characteristic features is merged, and obtains a plurality of targets of every two field picture in the described consecutive image.
Wherein, present embodiment carries out clustering processing in carrying out the target deterministic process, is in order to reduce the number of image object, on the one hand, the processing of too many target is taken a part of resource; On the other hand, having the different target of same characteristic features in the image may light source feedback just just itself.The consecutive image that present embodiment photographs camera by the clustering processing module carries out clustering algorithm to be handled, and the pixel fusion of identical characteristics is integral, and reduces the number of target in the image.Wherein, Jing Dian clustering algorithm has based on watershed divide and fuzzy C-means clustering, based on K-Means cluster or the like.
In the present embodiment, by clustering processing, the spot noise that will much loose is fused to together, the indirect purpose that has played de-noising, and therefore, this clustering processing module 131 can also be removed large tracts of land interference noise or static background.
Motion detection block 132 is used for finding out moving target from described a plurality of targets.
In the present embodiment, after the target of having determined to exist, can also judge that each target is static or motion, therefrom finds out moving target, so that calculate the center of gravity of each moving target according to motion detection.
Wherein, motion detection can adopt classic algorithm, if promptly each pixel difference of consecutive frame is greater than a threshold value, and this pixel of present frame is in the smooth background after the de-noising, thinks that then this pixel is the motion pixel.In the present embodiment, owing to pass through clustering algorithm, be clustered into a plurality of targets in every two field picture, then motion detection is based on the consecutive frame target, rather than based on pixel.
Noise elimination module 133 is used for described moving target is carried out Filtering Processing, to confirm the effective target of every two field picture in the described consecutive image that each image acquisition units gets access to.
In the present embodiment, if image acquisition units 12 is to be used to gather with the refractive body image information of a plurality of telepilots of thing as a token of, then in the present embodiment, this noise elimination module 133 can be confirmed a plurality of effective targets.Wherein, corresponding telepilot of effective target.
In the present embodiment, when the camera collection image,, circuit, electromagnetism etc. can introduce various noises owing to disturbing, white Gaussian noise for example, be characterized in randomness and low light level epistasis, noise brings interference can for the processing multiple goal, so present embodiment carries out filtering by noise elimination module 133 according to the continuous multiple frames image information, select for use different Filtering Model to filter out corresponding noise.Wherein, noise elimination module 133 can adopt classical frame filter model, for example average value filtering, Gauss's low-pass filtering or the like, for example, in N successive frame, the barycentric coordinates of same target are p (i), the absolute value of adjacent barycentric coordinates difference is A (i), A (in)=| p (in)-p (in-1) |, if the mean square deviation of the A (i) of N continuous frame is then thought Gaussian noise less than a threshold value.
Computing module 134 is used to calculate the center of gravity of described effective target.
Multiple goal separative element 14 is used for the barycentric coordinates of the effective target of the every two field picture of described consecutive image that gets access to according to each image acquisition units, confirms the motion track of described effective target.
The barycentric coordinates of the continuous multiple frames image motion target that each image acquisition units 12 that multiple goal separative element 14 utilizations of present embodiment are write down collects are judged the movement locus that a plurality of moving targets of current frame image are affiliated.Illustrate: two refractive body a, each autokinesis of b, moving target in the continuous multiple frames image that collects for any one image acquisition units 12, every frame all can write down the barycentric coordinates A (n) of two moving targets, B (n) then has two track TA (n) certainly, TB (n), every frame all can be judged A (n) and belong to TA (n), and B (n) belongs to TB (n).
In the present embodiment, this multiple goal separative element 14 is to be used for coming automatic judgement should belong to which moving target according to track each moving object, also be, for each moving object, multiple goal separative element 14 can according to its separately track judge automatically it belongs to which known target.
In the present embodiment, because its movement locus of different target has nothing in common with each other, the multiple goal separative element 14 of present embodiment judges automatically according to the movement locus of each moving object which target is the barycentric coordinates of the effective target of present frame should belong to, for example, the i bar track T (n of present frame, i), its in present frame coordinates of motion predicted value Pre (n, i)=2*T (n-1, i)-T (n-2, i), if Pre (n, i) and the absolute value of the difference of the barycentric coordinates C (n) of certain target of present frame less than a threshold value, think that then this target belongs to i bar track, by that analogy, which track is all targets that can calculate present frame belong to.Wherein, the detection of movement locus can utilize speed, acceleration, perhaps utilize respectively the target center of gravity (X, motion vector Y) or the like it should be noted that, when target many more, the correctness that its movement locus detects is low more, because there is similar interference between each target trajectory, is head it off, can adopt the model of probability verification to detect correctness and carry out corresponding coupling, not repeat them here.
In the present embodiment, can also according to wide, the height of each target and area check once more above-mentioned judge whether correct.For example, at first utilizing optical flow method and Kalman filter model, is that estimation criterion is checked with Minimum Mean Square Error, comes verification with wide height and area then, because under the fixed-size situation of luminophor, it becomes the tall and big little threshold value of can setting for of image width to be limited at camera.It should be noted that, if the user moves forward and backward telepilot, its wide high surface that is reflected to the moving target on the image amasss and also can change, for addressing this problem, can adopt a plurality of cameras, calculate the distance of each target range screen and angle with the 3D method and wait and distinguish, do not repeat them here.
The judgement unit 15 that intersects is used for the barycentric coordinates of the effective target of the every two field picture of described consecutive image that gets access to according to each image acquisition units, the actual center gravity coordinate of target that efficiency confirmed.
In the present embodiment, owing to be provided with at least two image acquisition units 12, the consecutive image information that collects for each image acquisition units, the processing of capital process cluster and filter unit 13, multiple goal separative element 14, through after the above-mentioned processing, obtain the barycentric coordinates of the effective target of corresponding each image acquisition units 12, the intersection judgement unit 15 of present embodiment is about to the barycentric coordinates according to the effective target of the consecutive image information acquisition of each image acquisition units 12 collection, the true coordinate of target that efficiency confirmed.
According to an embodiment of present embodiment, the judgement unit 15 that intersects is judged current moving object at a receiving end in the relation of the centre coordinate of a plurality of cameras (being generally 2), also is the true coordinate of effective target.If P (i, x) and P (i y) is the centre coordinate of i the camera collection moving object of arriving, wherein, i=1,2 ... N.P (i, x
k) and P (i, y
k) the k frame data of centre coordinate for this reason, P (i, x
Pred) and P (i, y
Pred) predicted value of centre coordinate correspondence, then P (i, x for this reason
Pred)=(3.0*P (1, x)-3.5*P (2, x)+2*P (3, x)-0.5*P (4, x)), in like manner, P (i, y
Pred)=(3.0*P (1, y)-3.5*P (2, y)+2*P (3, y)-0.5*P (4, y)).Get | and P (i, x)-P (i, x
Pred) | the barycentric coordinates of i the camera correspondence that value is minimum are the X coordinate of present frame, | and P (i, y)-P (i, y
Pred) | the barycentric coordinates of i the camera correspondence that value is minimum are the Y coordinate of present frame, wherein, i=1,2 ... N.Like this, can obtain the true coordinate of this moving object.
Cursor offset control unit 16 is used for according to the actual center gravity coordinate of described effective target and the motion track of described effective target, calculate the screen display coordinate of described effective target, the motion track of the corresponding screen display coordinate of motion track of acquisition and described effective target.
In the present embodiment, this cursor offset control unit 16 be used to calculate each moving object at the screen display coordinate of present frame outputing to the screen of terminal, thereby control moving of a plurality of cursors on the screen respectively.For example, the true barycentric coordinates of supposing each moving target of adjacent two frames are T (n), T (n-1), then the frame difference of present frame coordinate is d (n)=T (n)-T (n-1), receiving end can be noted the coordinate M (n) of previous frame screen mouse arrow, and then the displaing coordinate of present frame screen mouse arrow is M (n)+d (n).
According to an embodiment of present embodiment, this multiple goal treating apparatus can also comprise:
Output unit 17 is used for the screen display coordinate of described effective target is outputed to terminal, is shown by described terminal.
In the present embodiment, this output unit 17 can arrive end host with above-mentioned coordinates transmission by wired or wireless mode, thereby a plurality of cursors on the control terminal screen is mobile.
The multiple goal treating apparatus of present embodiment can be handled a plurality of moving targets, realizes the application that the multi-user operates simultaneously, and locatees quickly and accurately.
The composition frame chart of a kind of terminal that Fig. 2 provides for the utility model embodiment please refer to Fig. 2, and this terminal also comprises aforesaid multiple goal treating apparatus 22 and telepilot 23 except comprising terminal body 21, wherein:
Terminal body 21 is the composition of existing terminal, and the terminal here can be personal computer, Digital Television or simulated television, if for example terminal is a personal computer, then terminal body 21 is existing main frame and display; If terminal is a Digital Television, then terminal body 21 is existing digital television; If terminal is a simulated television, then terminal body 21 is existing set-top box and simulated television.
Multiple goal treating apparatus 22 has been done detailed description in the aforementioned embodiment, does not repeat them here.
Telepilot 23 is used for by refractive body to multiple goal treating apparatus 22 reflection rays, with by 22 pairs of multiple goal treating apparatus with the judgement and the processing of the motion track of the telepilot of thing as a token of of this refractive body, the moving of the cursor of control terminal body 21.
Wherein, telepilot 23 can be one or more.
According to an embodiment of present embodiment, this telepilot 23 mainly comprises:
Refractive body is used to reflect the light that the luminophor that is arranged at terminal body 21 sides or multiple goal treating apparatus 22 sides emits.
In the present embodiment, this refractive body is made by reflectorized material, need not any power source, and can make different shape as required, satisfies the 3D pattern.
According to the another one embodiment of present embodiment, this telepilot 23 can also comprise:
Basal keystroke portion is used to send the key set code stream information of basal keystroke;
Wherein, the basal keystroke of this basal keystroke portion has comprised numerical key, function key, shortcut etc., and is identical with the function of the basal keystroke of existing telepilot in function, do not repeat them here.
Mousebutton portion is used to send the key set code stream information of mousebutton;
Wherein, the mousebutton of this mousebutton portion has comprised left mouse button, right mouse button, Mouse Scroll adjusting key etc., and its function also function with each button of existing mouse is identical, does not repeat them here.
At least one is wireless or infrared emission unit, is used for the key set code stream information of described basal keystroke and/or the key set code stream information of described mousebutton are sent to described multiple goal treating apparatus.
Wherein, the wireless or infrared emission unit here is to be used for the key set code stream information of basal keystroke portion and/or the key set code stream information of mousebutton portion are sent to multiple goal treating apparatus 22, therefore, an embodiment according to present embodiment, two wireless or infrared emission units can be set, with the key set code stream information that sends basal keystroke portion respectively and the key set code stream information of mousebutton portion; According to the another one embodiment of present embodiment, a wireless or infrared emission unit can only be set, with the key set code stream information of transmission basal keystroke portion and the key set code stream information of mousebutton portion.
The terminal of present embodiment can be personal computer, Digital Television or simulated television, comprise aforesaid terminal body 21, multiple goal treating apparatus 22 and telepilot 23, cooperation by telepilot 23 and multiple goal treating apparatus 22, broken through the restriction of traditional remote controller and conventional mouse, more convenient and quicker growing terminal and the business that provides thereof are controlled and used.
Fig. 3 please refer to Fig. 3 for a kind of terminal applies synoptic diagram that the utility model embodiment provides, and in should using, this terminal comprises: terminal body 31; The telepilot 33 of multiple goal treating apparatus 32 and two band refractive bodies, wherein:
Terminal body 31 represents that by a display screen and a processing host in the present embodiment, the display screen of this terminal body 31 can be the display of computing machine, the display screen of Digital Television, the perhaps display screen of simulated television; The processing host of this terminal body 31 can be the main frame of counter, the process chip of Digital Television, the perhaps process chip of simulated television.
Multiple goal treating apparatus 32 can be contained in the processing host of computing machine, the processing section of Digital Television, the perhaps set-top box of simulated television.
The telepilot 33 of two band refractive bodies can be the mouse type telepilot, has the function of mouse and telepilot concurrently.
In should using, the luminophor that is arranged at display screen emits beam, the telepilot that has refractive body reflects described light to display screen, multiple goal treating apparatus 32 is caught the image of this telepilot 33 by two cameras, judge the existence of two moving targets, and after calculating separately coordinate respectively, be transferred to terminal body 31 by wired or wireless mode, thereby two cursors moves on the screen of control terminal body 31.
Present embodiment is to be that example is illustrated with two telepilots, but present embodiment not with this as restriction, for example the scheme of a telepilot or a plurality of telepilots can be applied to the utility model.
Present embodiment combines remote-controller function and mouse function are better, handle by multiple goal, promptly, utilize adjacent multiple image data to judge the center of gravity of moving target, judge movement locus under each moving target of present frame according to movement locus, and utilize probability model to test, make the air mouse scheme can judge and handle a plurality of moving targets, the application that practicable multi-user operates simultaneously, and locate quickly and accurately, and have following advantage:
1. not only can be used for seeing TV, video recording, VCD, DVD etc., can also be used for playing games, surf the Net, send e-mails, shopping online, online program request etc.
2. solved the limitation that user's operation can only be arranged when former mouse remote control device is worked, made and on PC or TV, can divide a plurality of zones that allow the multi-user operate respectively, for example picture-in-picture can have two users to change platform respectively on the TV.
3. having solved former mouse remote control device can only have a player's limitation in field of play, makes more actualization of recreation (especially shooting game).
The method of describing in conjunction with embodiment disclosed herein or the step of algorithm can directly use the software module of hardware, processor execution, and perhaps the combination of the two is implemented.Software module can place the storage medium of any other form known in random access memory (RAM), internal memory, ROM (read-only memory) (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or the technical field.
Above-described specific embodiment; the purpose of this utility model, technical scheme and beneficial effect are further described; institute is understood that; the above only is a specific embodiment of the utility model; and be not used in and limit protection domain of the present utility model; all within spirit of the present utility model and principle, any modification of being made, be equal to replacement, improvement etc., all should be included within the protection domain of the present utility model.