CN104090660A - Motion collecting and feedback method and system based on stereoscopic vision - Google Patents
Motion collecting and feedback method and system based on stereoscopic vision Download PDFInfo
- Publication number
- CN104090660A CN104090660A CN201410329067.XA CN201410329067A CN104090660A CN 104090660 A CN104090660 A CN 104090660A CN 201410329067 A CN201410329067 A CN 201410329067A CN 104090660 A CN104090660 A CN 104090660A
- Authority
- CN
- China
- Prior art keywords
- accessories apparatus
- infrared light
- stereoscopic vision
- computing machine
- light supply
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Position Input By Displaying (AREA)
Abstract
The invention discloses a motion collecting and feedback method and system based on stereoscopic vision. The method comprises the following steps that the space position of an infrared light source in an attachment position in a front space is collected, the space position is mapped into a virtual space, and the position and direction of a virtual object are controlled; when the virtual object interacts with a virtual environment, a wireless control attachment device vibrates and simulates the feedback feeling of force. The motion collecting and feedback method and system based on stereoscopic vision have the advantages that positioning is rapid and accurate, and operation is stable; the hardware is quite low in cost, simple in structure and quite small in size; force feedback is achieved and can largely improve the sense of reality of interaction behaviors, and user experience is improved; the reality function is improved, and the man-machine interaction modes are increased.
Description
Technical field
The present invention relates to a kind of action collection and feedback method and system based on stereoscopic vision, belong to virtual reality and augmented reality field.
Background technology
Gather hand motion and set it as input signal for computer control, thereby can in virtual environment, reduce this hand motion in order to man-machine interaction.This type of man-machine interaction mode has the following approach that realizes conventionally:
1, laser scanning: the scope in laser scanning front, sensor gathers the reflected light of each direction, calculates distance, obtains the depth information of each position, front, composition one width depth map.If hand is put to sensor front, the profile of hand can show on depth map; And if adopt laser continuous sweep front scope, the locus of hand, movement locus and attitude all can be by computing machine continuous acquisition so.After the data input computing machine gathering, formation control data, and then carry out man-machine interaction.The advantage of the method is that the space environment information obtaining is accurate, if shortcoming is to want that the action Quick Catch of realizing low delay needs more expensive equipment, and outfit of equipment volume is larger;
2, infrared light spot scanning: use infrared light supply forwards to penetrate randomly shaped hot spot, when this hot spot irradiates on the different object of front profile, light spot shape can change accordingly; Camera contrasts the shape of the light spot shape collecting and storage, and resolves by particular algorithm, thereby obtains the depth information of each position, equipment front, composition depth map.If hand is placed on before equipment, can realize equally the continuous acquisition of hand motion.The method is applicable to the ripe amusement equipment such as KINECT, and technology maturation is reliable, and price is medium, and outfit of equipment is long-pending medium, but its algorithm complexity, the system resource taking is more, and overall time delay is larger, thereby has reduced the level of comfort using;
3, electromagnetic induction: equipment is emitting electromagnetic wave forwards, in the time that the objects such as hand enter electromagnetic field, can form disturbance; Equipment is reduced to hand motion by particular algorithm by disturbance, realizes action collection.The method time delay is little, and cost is low, and the precision and stability that still reduces is poor, and can only realize action collection, cannot realize image acquisition for augmented reality purposes;
4, visual identity: gather image by single camera or dual camera, and gather hand motion by graphics algorithm, carry out man-machine interaction.But due to hand profile complexity, feature is not obvious, thereby the operand of the method is larger, can produce certain delay, affects user and experience.
To sum up, all problems of workout cost, time delay, precise and stable property simultaneously of above existing solution.The most important thing is does not have feedback mechanism, when staff and virtual world object occur when mutual, only to be moved by visually-perceptible, cannot produce the feedback of the power such as sense of touch, and authenticity is had a greatly reduced quality.
Summary of the invention
The object of the invention is to, a kind of action collection and feedback method and system based on stereoscopic vision is provided, it can effectively solve problems of the prior art, especially prior art simultaneously workout cost, time delay, precise and stable property problem and cannot produce the problem of force feedback.
For solving the problems of the technologies described above, the present invention adopts following technical scheme: a kind of action collection and feedback method based on stereoscopic vision, comprise the following steps: the locus that gathers infrared light supply on the interior accessories apparatus of front space, and this locus is mapped in Virtual Space, control position and the direction of dummy object; In the time of dummy object and virtual environment generation interbehavior, controlled in wireless accessories apparatus shakes, the feedback sense of analog force.
Preferably, in described collection front space, on accessories apparatus, the locus of infrared light supply specifically comprises: be fixed on the separately image in left and right collection front of Double-camera device on head mounted display, and left and right two-way view data is imported in computing machine, computing machine is processed two-way view data, obtains the locus of infrared light supply on accessories apparatus.
Preferred, described Double-camera device is the image in left and right collection front separately, and left and right two-way view data is imported in computing machine and specifically comprised: Double-camera device is by the CMOS of the left and right sides or the view data in ccd sensor collection front, this view data becomes digital signal and enters dsp processor after A/D converter, exposes, gain, white balance processing; Digital signal after treatment enters carries out Signal coding in coding chip; After coding, two-path video signal enters the synthetic road of data bus and exports computing machine to by usb data line, thereby can be stablized, digital picture clearly.
Described usb data line also can be the data line that Double-camera device carries.
Preferred, described computing machine is processed two-way view data, obtaining the locus of infrared light supply on accessories apparatus specifically comprises: computing machine is decoded to two-path video signal, and gathers X, the Y coordinate figure of two-path video signal mid-infrared light source in image; The coordinate figure of the infrared light supply of each camera lens collection is converted into the position angle of infrared light supply with respect to camera lens; In the time of the position angle of known relative two camera lenses and the spacing of two camera lenses, calculate and obtain X, Y, the Z coordinate figure of infrared light supply; The X of infrared light supply, Y, Z coordinate figure, after lens distortion mathematic(al) experience model is adjusted, are obtained to the real space coordinate of infrared light supply with respect to camera.The method can obtain with degree of precision and efficiency the locus of infrared light source point, catches thereby the high speed of realize target object is stable.
The method of the invention also comprises: by the model of storing in the relative position of multiple infrared light supplies on the accessories apparatus real space coordinate of camera (with respect to) and the computing machine contrast that is coupled, judge kind and the particular location of infrared light supply on this accessories apparatus of the accessories apparatus that described infrared light supply is corresponding, thereby judge locus and the attitude of these accessories apparatus, and continuous coordinate collection also can calculate motion track and the speed of these devices, and control device using this as man-machine interaction.
The method of the invention also comprises: computing machine to two-path video signal decoding after, a point right and left eyes shows, forms the stereoscopic vision of real space around; In this stereoscopic vision, point right and left eyes stack dummy object or scene; Simultaneous computer gathers X, the Y coordinate figure of two-path video signal mid-infrared light source, obtain the control command (being position and action command) of accessories apparatus in the visual field, realize the Interactive Experience under augmented reality environment, such as the shape accessories apparatus that puts on one's gloves, can directly click and control the virtual soldier of micro in the little battlefield of desktop, and these are clicked, and touching behaviors are the same triggers aforesaid FORCE FEEDBACK event, and the control that greatly promotes user is experienced.
Action collection and the feedback system based on the stereoscopic vision that realize preceding method, comprising: Double-camera device, pedestal, accessories apparatus and computing machine, and described pedestal is provided with A bluetooth module and usb data line, and usb data line is connected with computing machine; Described accessories apparatus is provided with B bluetooth module, B bluetooth module and the wireless connections of A bluetooth module; On described accessories apparatus, be also provided with infrared LED or infrared LED, vibrations module and drive box, driving box to be connected with A bluetooth module and vibrations module respectively.
Preferably, the camera lens of described Double-camera device adopts 120 degree and above wide-angle lens, gathers thereby can realize larger area action.
Preferably, described system also comprises: head mounted display, Double-camera device separately left and right is fixed on head mounted display.
In the present invention, described accessories apparatus is gloves-shaped accessories apparatus, finger ring shape accessories apparatus, rifle shape accessories apparatus or handle attachment device, on described gloves-shaped accessories apparatus, finger ring shape accessories apparatus, rifle shape accessories apparatus or handle attachment device, be respectively equipped with infrared LED, on gloves-shaped accessories apparatus, be also provided with vibrations module and drive box, on rifle shape accessories apparatus, handle attachment device, be also provided with trigger and button, trigger and button are connected with B bluetooth module.
In aforesaid action collection and feedback system based on stereoscopic vision, described Double-camera device comprises: CMOS or ccd sensor, A/D converter, dsp processor and coding chip, CMOS or ccd sensor, A/D converter, dsp processor and coding chip are connected in turn, and coding chip is connected with computing machine by data bus and usb data line.
In the present invention, on described pedestal, also comprise: charging wire, charging wire is connected with accessories apparatus.
Compared with prior art, the invention has the advantages that:
1. location is rapid, accurate, stable: use above-mentioned binocular parallax principle to carry out space orientation to infrared light source point by stereoscopic vision, because the feature of infrared light source point in space environment is very clear, therefore locate computing simple, and precision is higher, add the highest support of Double-camera device 120Hz frame per second, the in the situation that of object high-speed motion, also can keep stable and follow the trail of, therefore overall time delay is very low, better user experience;
2. hardware cost is extremely cheap, simple in structure and volume is very little: the present invention is by adopting Double-camera device to gather the image in front, and then the locus of infrared light supply on accessories apparatus in calculating front space, wherein, Double-camera device adopts micromodule CMOS, ccd image sensor to build, and not only price is extremely low, module height is integrated, and volume and weight is very little, use even if hang on head mounted display, can not form burden yet;
3. can feed back by realizable force: force feedback can be strengthened the sense of reality of interbehavior greatly, promote user and experience;
4. action acquisition range is wide: because camera lens adopts 120 degree and above wide-angle lens, can realize larger area action and gather;
5. can synchronously realize augmented reality function: in the present invention, adopt Double-camera device as sensor, the stereoscopic vision that equipment gathers is except for space orientation, can also export head mounted display to and reappear true vision, and in true vision also stackable information or dummy object; The present invention can also, by hand motion and these information or object interaction, realize augmented reality function in addition.
Brief description of the drawings
Fig. 1 is the structural representation of finger ring shape accessories apparatus;
Fig. 2 is the structural representation of rifle shape accessories apparatus;
Fig. 3 is the structural representation of handle attachment device;
Fig. 4 is the structural representation of pedestal;
Fig. 5 is the structural representation of gloves-shaped accessories apparatus;
Fig. 6 is the connected mode schematic diagram of Double-camera device and head mounted display;
Fig. 7 is the method flow diagram of a kind of embodiment of the present invention;
Fig. 8 is the structural representation of embodiment 2;
Fig. 9 is the structural representation of embodiment 3;
Figure 10 is the structural representation of embodiment 4;
Figure 11 is the structural representation of embodiment 5;
Figure 12 is the structural representation of embodiment 6.
Reference numeral: 1-Double-camera device, 2-head mounted display, 3-gloves-shaped accessories apparatus, 4-shakes module, 5-drives box, 6-finger ring shape accessories apparatus, 7-rifle shape accessories apparatus, 8-infrared LED, 9-trigger and button, 10-handle attachment device, 11-pedestal, 12-charging wire, 13-USB data line, 14-accessories apparatus, 15-computing machine, 16-A bluetooth module, 17-B bluetooth module, 18-CMOS or ccd sensor, 19-A/D converter, 20-DSP processor, 21-coding chip, 22-data bus.
Below in conjunction with the drawings and specific embodiments, the present invention is further illustrated.
Embodiment
Embodiments of the invention 1: a kind of action collection and feedback method based on stereoscopic vision, as shown in Figure 7, comprise the following steps: gather in front space the locus of infrared light supply on accessories apparatus 14: the CMOS of the Double-camera device 1 that is fixed on head mounted display 2 fronts by the left and right sides or ccd sensor 18 gather the view data in front, this view data becomes digital signal and enters dsp processor 20 after A/D converter 19, exposes, gain, white balance processing; Digital signal after treatment enters carries out Signal coding in coding chip 21; After coding, two-path video signal enters the synthetic road of data bus 22 and the usb data line that carries by Double-camera device 1 exports computing machine 15 to; Computing machine 15 is decoded to two-path video signal, and gathers X, the Y coordinate figure of two-path video signal mid-infrared light source in image; The coordinate figure of the infrared light supply of each camera lens collection is converted into the position angle of infrared light supply with respect to camera lens; In the time of the position angle of known relative two camera lenses and the spacing of two camera lenses, calculate and obtain X, Y, the Z coordinate figure of infrared light supply; The X of infrared light supply, Y, Z coordinate figure, after lens distortion mathematic(al) experience model is adjusted, are obtained to the real space coordinate of infrared light supply with respect to camera; By that analogy, can obtain the coordinate of multiple infrared light supplies on accessories apparatus 14 by above method, by the model contrast that is coupled of storage in the relative position of these coordinates and computing machine 15, thereby can judge kind (gloves-shaped annex or finger ring shape annex etc.) and the particular location of infrared light supply on this accessories apparatus 14 of the accessories apparatus 14 that described infrared light supply is corresponding, and then judge locus and the attitude of these accessories apparatus 14; Continuous motion track and speed of carrying out coordinate collection and can calculate these accessories apparatus 14; Described accessories apparatus locus, motion track and Rate mapping are entered in Virtual Space, control position and the direction of dummy object; In the time of dummy object and virtual environment generation interbehavior, controlled in wireless accessories apparatus 14 shakes, the feedback sense of analog force.Computing machine 15 to two-path video signal decoding after, a point right and left eyes shows, forms the stereoscopic vision of real space around; In this stereoscopic vision, point right and left eyes stack dummy object or scene; Simultaneous computer 15 gathers X, the Y coordinate figure of two-path video signal mid-infrared light source, obtains position and the action command of accessories apparatus 14 in the visual field, realizes the interaction under augmented reality environment.
Embodiment 2: action collection and the feedback system based on the stereoscopic vision that realize method described in embodiment 1, as shown in Figure 8, comprise: Double-camera device 1, pedestal 11, accessories apparatus 14 and computing machine 15, described pedestal 11 (as shown in Figure 4) is provided with A bluetooth module 16 and usb data line 13, and usb data line 13 is connected with computing machine 15; Described accessories apparatus 14 is provided with B bluetooth module 17, B bluetooth module 17 and 16 wireless connections of A bluetooth module; Double-camera device 1 is connected with computing machine 15; Described accessories apparatus 14 is gloves-shaped accessories apparatus 3 (as shown in Figure 5), is also provided with infrared LED 8, vibrations module 4 and drives box 5 on gloves-shaped accessories apparatus 3, drives box 5 to be connected with A bluetooth module 16 and vibrations module 4 respectively; Described Double-camera device 1 comprises: CMOS or ccd sensor 18, A/D converter 19, dsp processor 20 and coding chip 21, CMOS or ccd sensor 18, A/D converter 19, dsp processor 20 and coding chip 21 are connected in turn, and coding chip 21 is connected with computing machine 15 by data bus 22 and usb data line 13; Also comprise: head mounted display 2, Double-camera device 1 separately left and right is fixed on head mounted display 2 fronts (as shown in Figure 6).
Embodiment 3: action collection and the feedback system based on the stereoscopic vision that realize method described in embodiment 1, as shown in Figure 9, comprise: Double-camera device 1, pedestal 11, accessories apparatus 14 and computing machine 15, described pedestal 11 (as shown in Figure 4) is provided with A bluetooth module 16 and usb data line 13, and usb data line 13 is connected with computing machine 15; Described accessories apparatus 14 is provided with B bluetooth module 17, B bluetooth module 17 and 16 wireless connections of A bluetooth module; Double-camera device 1 is connected with computing machine 15; Described accessories apparatus 14 is finger ring shape accessories apparatus 6 (as shown in Figure 1), is also provided with infrared LED 8 on finger ring shape accessories apparatus 6; Described Double-camera device 1 comprises: CMOS or ccd sensor 18, A/D converter 19, dsp processor 20 and coding chip 21, CMOS or ccd sensor 18, A/D converter 19, dsp processor 20 and coding chip 21 are connected in turn, and coding chip 21 is connected with computing machine 15 by data bus 22 and usb data line 13; Also comprise: head mounted display 2, Double-camera device 1 separately left and right is fixed on head mounted display 2 fronts (as shown in Figure 6).
Embodiment 4: action collection and the feedback system based on the stereoscopic vision that realize method described in embodiment 1, as shown in figure 10, comprise: Double-camera device 1, pedestal 11, accessories apparatus 14 and computing machine 15, described pedestal 11 (as shown in Figure 4) is provided with A bluetooth module 16 and usb data line 13, and usb data line 13 is connected with computing machine 15; Described accessories apparatus 14 is provided with B bluetooth module 17, B bluetooth module 17 and 16 wireless connections of A bluetooth module; Double-camera device 1 is connected with computing machine 15; Described accessories apparatus 14 is rifle shape accessories apparatus 7 (as shown in Figure 2), and rifle shape accessories apparatus 7 is provided with infrared LED 8, trigger and button 9, and trigger and button 9 are connected with B bluetooth module 17; Described Double-camera device 1 comprises: CMOS or ccd sensor 18, A/D converter 19, dsp processor 20 and coding chip 21, CMOS or ccd sensor 18, A/D converter 19, dsp processor 20 and coding chip 21 are connected in turn, and coding chip 21 is connected with computing machine 15 by data bus 22 and usb data line 13; Also comprise: head mounted display 2, Double-camera device 1 separately left and right is fixed on head mounted display 2 fronts (as shown in Figure 6).
Embodiment 5: action collection and the feedback system based on the stereoscopic vision that realize method described in embodiment 1, as shown in figure 11, comprise: Double-camera device 1, pedestal 11, accessories apparatus 14 and computing machine 15, described pedestal 11 (as shown in Figure 4) is provided with A bluetooth module 16 and usb data line 13, and usb data line 13 is connected with computing machine 15; Described accessories apparatus 14 is provided with B bluetooth module 17, B bluetooth module 17 and 16 wireless connections of A bluetooth module; Double-camera device 1 is connected with computing machine 15; Described accessories apparatus 14 is handle attachment device 10 (as shown in Figure 3), and handle attachment device 10 is provided with infrared LED 8, trigger and button 9, and trigger and button 9 are connected with B bluetooth module 17; Described Double-camera device 1 comprises: CMOS or ccd sensor 18, A/D converter 19, dsp processor 20 and coding chip 21, CMOS or ccd sensor 18, A/D converter 19, dsp processor 20 and coding chip 21 are connected in turn, and coding chip 21 is connected with computing machine 15 by data bus 22 and usb data line 13; Also comprise: head mounted display 2, Double-camera device 1 separately left and right is fixed on head mounted display 2 fronts (as shown in Figure 6).
Embodiment 6: a kind of action collection and feedback system based on stereoscopic vision, as shown in figure 12, comprise: Double-camera device 1, pedestal 11, accessories apparatus 14 and computing machine 15, described pedestal 11 is provided with A bluetooth module 16 and usb data line 13, and usb data line 13 is connected with computing machine 15; Described accessories apparatus 14 is provided with B bluetooth module 17, B bluetooth module 17 and 16 wireless connections of A bluetooth module; On described accessories apparatus 14, be also provided with infrared LED 8.
Principle of work:
1, gloves-shaped accessories apparatus 3: be provided with infrared LED 8 and vibrations module 4 (infrared LED 8 and shake module 4 together with being) at key nodes such as glove fingertip, the built-in miniature eccentric motor in gloves finger belly position or vibration of ultrasonic wave generator one class vibrations module 4, there is the box 5 of driving gloves the back of the hand position, drives in box 5 and comprises battery, B bluetooth module 17, shock module driving circuit and miniature charging inlet etc.Battery is connected with charging wire 12 after charging by miniature charging inlet, is device power supply; A bluetooth module 16 wireless connections on B bluetooth module 17 and pedestal 11, computing machine 15 transmits signal by the A bluetooth module 16 on pedestal 11 to B bluetooth module 17, by B bluetooth module 17, control signal is sent to shock module driving circuit, finally drives vibrations module 4 to produce vibrations by shock module driving circuit;
2, finger ring shape accessories apparatus 6: only comprise battery, miniature charging inlet, microswitch and infrared LED 8, battery is connected the backward infrared LED 8 of charging and powers with charging wire 12 through miniature charging inlet;
3, rifle shape accessories apparatus 7: rifle ridge portion is arranged with two or more infrared LEDs 8, the body of a gun is provided with trigger and button 9, button can be realized the multiple operations such as walking control in software, trigger can be realized the simulation operation such as open fire, in the body of a gun, comprise battery, B bluetooth module 17, shock module driving circuit and miniature charging inlet etc., battery is connected with charging wire 12 after charging by miniature charging inlet, is device power supply; A bluetooth module 16 wireless connections on B bluetooth module 17 and pedestal 11, the one, transmit signal by the A bluetooth module 16 on pedestal 11 to B bluetooth module 17 by computing machine 15, by B bluetooth module 17, control signal is sent to shock module driving circuit, finally drive vibrations module 4 to produce vibrations by shock module driving circuit, the 2nd, during by lower gun body button or trigger, B bluetooth module 17 is passed push button signalling back computing machine 15 by pedestal 11;
4, handle attachment device 10: in the time that DASE needs space orientation and complicated button operation simultaneously, can adopt handle attachment device 10 or other shaped apertures, on device, being both provided with infrared light supply demarcates for locus, because assortment of keys realizes operating function, simultaneously built-in FORCE FEEDBACK vibrations module 4, device is connected with computing machine 15 with pedestal 11 by B bluetooth module 17, realizes principle identical with other a few class devices;
5, pedestal 11: pedestal 11 is connected with computing machine 15 by usb data line 13, is connected to accessories apparatus charging by miniature charging inlet and charging wire 12, and is carried out radio communication, exchange signal or implemented to control by A bluetooth module 16 and accessories apparatus;
6, the locus positioning principle of infrared light source point: the CMOS of Double-camera device 1 by the left and right sides or ccd sensor 18 gather the view data in front, this view data becomes digital signal and enters dsp processor 20 after A/D converter 19, exposes, gain, white balance processing; Digital signal after treatment enters carries out Signal coding in coding chip 21; After coding, two-path video signal enters the synthetic road of data bus 22 and exports computing machine 15 to by usb data line 13; Obtain after the two-path video signal that Double-camera device 1 inputs at computing machine 15, to two-path video signal decoding, and gather X, the Y coordinate figure of two-path video mid-infrared light source point in image, the coordinate that each camera lens gathers infrared light supply can be scaled the relative camera lens of light source position angle, the in the situation that of known relative two camera lens position angles and two camera lens spacing, can be regarded as out X, Y, the Z coordinate figure of light source, this coordinate, after lens distortion mathematic(al) experience model is adjusted, obtains the real space coordinate of light source point with respect to camera.Can obtain the coordinate of multiple light source points by above method, the relative position of these coordinates can with the contrast that is coupled of the model of Computer Storage, with judge these light source points corresponding be gloves or other accessories apparatus 14, and then judge locus and the attitude of these devices.And continuous coordinate collection can calculate motion track and the speed of these devices, and control device using this as man-machine interaction;
7, force feedback principle: when user by Double-camera device 1 and accessories apparatus 14 to virtual environment in virtual portrait or dummy object while controlling, virtual portrait or virtual object are known from experience and virtual environment generation interbehavior, in the time of some default mutual generation, computing machine 15 can carry out to accessories apparatus pedestal 11 transmitted signals the feedback of power to control certain vibrations module 4.Such as virtual portrait finger touches a stone of virtual environment, software engine will send signal to the drive software of accessories apparatus pedestal 11, the vibrations module 4 of being pointed by A bluetooth module 16 controlled in wireless gloves-shaped accessories apparatus 3 by pedestal 11 produces vibrations, to simulate the virtual sense of touching;
8, augmented reality principle: when computing machine 15 obtains after the two-path video signal that Double-camera device 1 inputs, to two-path video signal decoding, and a point right and left eyes shows on head mounted display 2, form the stereoscopic vision of real space around, in this stereoscopic vision, can superpose virtual object or scene, such as demonstrate a small-sized battlefield at desktop; Simultaneous computer 15 gathers the infrared light supply X in two-path video, Y coordinate figure, draw position and the action command of accessories apparatus 14 in the visual field by aforesaid locus Computing Principle and accessories apparatus 14 operation principles, thereby realize the Interactive Experience under augmented reality environment, such as the shape accessories apparatus 3 that puts on one's gloves, can directly click and control the virtual soldier of micro in the little battlefield of desktop, and these are clicked, and touching behaviors are the same triggers aforesaid FORCE FEEDBACK event, and the control that greatly promotes user is experienced.
Claims (10)
1. action collection and the feedback method based on stereoscopic vision, it is characterized in that, comprise the following steps: gather the locus of the upper infrared light supply of accessories apparatus (14) in front space, and this locus is mapped in Virtual Space, control position and the direction of dummy object; In the time of dummy object and virtual environment generation interbehavior, controlled in wireless accessories apparatus (14) shakes, the feedback sense of analog force.
2. action collection and the feedback method based on stereoscopic vision according to claim 1, it is characterized in that, in described collection front space, the locus of the upper infrared light supply of accessories apparatus (14) specifically comprises: be fixed on the separately image in left and right collection front of Double-camera device (1) on head mounted display (2), and left and right two-way view data is imported in computing machine (15), computing machine (15) is processed two-way view data, obtains the locus of the upper infrared light supply of accessories apparatus (14).
3. action collection and the feedback method based on stereoscopic vision according to claim 2, it is characterized in that, described Double-camera device (1) is the image in left and right collection front separately, and left and right two-way view data is imported in computing machine (15) and specifically comprised: Double-camera device (1) is by the CMOS of the left and right sides or the view data in ccd sensor (18) collection front, this view data becomes digital signal and enters dsp processor (20) after A/D converter (19), exposes, gain, white balance processing; Digital signal after treatment enters in coding chip (21) carries out Signal coding; After coding, two-path video signal enters the synthetic road of data bus (22) and exports computing machine (15) to by usb data line (13).
4. action collection and the feedback method based on stereoscopic vision according to claim 3, it is characterized in that, described computing machine (15) is processed two-way view data, the locus that obtains the upper infrared light supply of accessories apparatus (14) specifically comprises: two-path video signal is decoded, and gather X, the Y coordinate figure of two-path video signal mid-infrared light source in image; The coordinate figure of the infrared light supply of each camera lens collection is converted into the position angle of infrared light supply with respect to camera lens; In the time of the position angle of known relative two camera lenses and the spacing of two camera lenses, calculate and obtain X, Y, the Z coordinate figure of infrared light supply; The X of infrared light supply, Y, Z coordinate figure, after lens distortion mathematic(al) experience model is adjusted, are obtained to the real space coordinate of infrared light supply with respect to camera.
5. action collection and the feedback method based on stereoscopic vision according to claim 4, it is characterized in that, described method also comprises: by the model of storing in the relative position of upper accessories apparatus (14) multiple infrared light supplies and computing machine (15) contrast that is coupled, judge the kind of the accessories apparatus that described infrared light supply is corresponding (14).
6. action collection and the feedback method based on stereoscopic vision according to claim 4, it is characterized in that, described method also comprises: computing machine (15) to two-path video signal decoding after, a point right and left eyes shows, forms the stereoscopic vision of real space around; In this stereoscopic vision, stack dummy object or scene; Simultaneous computer (15) gathers X, the Y coordinate figure of two-path video signal mid-infrared light source, obtains the control command of accessories apparatus (14) in the visual field, realizes the interaction under augmented reality environment.
7. realize action collection and the feedback system based on stereoscopic vision of method described in claim 1～6, it is characterized in that, comprise: Double-camera device (1), pedestal (11), accessories apparatus (14) and computing machine (15), described pedestal (11) is provided with A bluetooth module (16) and usb data line (13), and usb data line (13) is connected with computing machine (15); Described accessories apparatus (14) is provided with B bluetooth module (17), B bluetooth module (17) and A bluetooth module (16) wireless connections; On described accessories apparatus (14), be also provided with infrared LED (8) or infrared LED (8), vibrations modules (4) and drive box (5), driving box (5) to be connected with A bluetooth module (16) and vibrations modules (4) respectively.
8. action collection and the feedback system based on stereoscopic vision according to claim 7, it is characterized in that, also comprise: head mounted display (2), Double-camera device (1) separately left and right is fixed on head mounted display (2).
9. according to action collection and feedback system based on stereoscopic vision described in claim 7 or 8, it is characterized in that, described accessories apparatus (14) is gloves-shaped accessories apparatus (3), finger ring shape accessories apparatus (6), rifle shape accessories apparatus (7) or handle attachment device (10), described gloves-shaped accessories apparatus (3), finger ring shape accessories apparatus (6), on rifle shape accessories apparatus (7) or handle attachment device (10), be respectively equipped with infrared LED (8), on gloves-shaped accessories apparatus (3), be also provided with vibrations modules (4) and drive box (5), rifle shape accessories apparatus (7), on handle attachment device (10), be also provided with trigger and button (9), trigger and button (9) are connected with B bluetooth module (17).
10. action collection and the feedback system based on stereoscopic vision according to claim 9, it is characterized in that, described Double-camera device (1) comprising: CMOS or ccd sensor (18), A/D converter (19), dsp processor (20) and coding chip (21), CMOS or ccd sensor (18), A/D converter (19), dsp processor (20) and coding chip (21) are connected in turn, and coding chip (21) is connected with computing machine (15) by data bus (22) and usb data line (13).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410329067.XA CN104090660B (en) | 2014-05-30 | 2014-07-10 | A kind of action collection and feedback method and system based on stereoscopic vision |
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410240371 | 2014-05-30 | ||
| CN2014102403717 | 2014-05-30 | ||
| CN201410240371.7 | 2014-05-30 | ||
| CN201410329067.XA CN104090660B (en) | 2014-05-30 | 2014-07-10 | A kind of action collection and feedback method and system based on stereoscopic vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104090660A true CN104090660A (en) | 2014-10-08 |
| CN104090660B CN104090660B (en) | 2017-11-10 |
Family
ID=51638381
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201420382187.1U Active CN203941499U (en) | 2014-05-30 | 2014-07-10 | A kind of action collection and feedback system based on stereoscopic vision |
| CN201410329067.XA Active CN104090660B (en) | 2014-05-30 | 2014-07-10 | A kind of action collection and feedback method and system based on stereoscopic vision |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201420382187.1U Active CN203941499U (en) | 2014-05-30 | 2014-07-10 | A kind of action collection and feedback system based on stereoscopic vision |
Country Status (2)
| Country | Link |
|---|---|
| CN (2) | CN203941499U (en) |
| WO (1) | WO2015180497A1 (en) |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104539929A (en) * | 2015-01-20 | 2015-04-22 | 刘宛平 | Three-dimensional image coding method and coding device with motion prediction function |
| CN104536579A (en) * | 2015-01-20 | 2015-04-22 | 刘宛平 | Interactive three-dimensional scenery and digital image high-speed fusing processing system and method |
| CN104699247A (en) * | 2015-03-18 | 2015-06-10 | 北京七鑫易维信息技术有限公司 | Virtual reality interactive system and method based on machine vision |
| CN104991650A (en) * | 2015-07-24 | 2015-10-21 | 贺杰 | Gesture controller and virtual reality system |
| WO2015180497A1 (en) * | 2014-05-30 | 2015-12-03 | 贺杰 | Motion collection and feedback method and system based on stereoscopic vision |
| CN105445937A (en) * | 2015-12-27 | 2016-03-30 | 深圳游视虚拟现实技术有限公司 | Mark point-based multi-target real-time positioning and tracking device, method and system |
| CN106354253A (en) * | 2016-08-19 | 2017-01-25 | 上海理湃光晶技术有限公司 | Cursor control method and AR glasses and intelligent ring based on same |
| CN106547458A (en) * | 2016-11-29 | 2017-03-29 | 北京小鸟看看科技有限公司 | A kind of virtual reality system and its space positioning apparatus |
| CN106768361A (en) * | 2016-12-19 | 2017-05-31 | 北京小鸟看看科技有限公司 | The position tracking method and system of the handle supporting with VR helmets |
| CN106899599A (en) * | 2017-03-09 | 2017-06-27 | 华东师范大学 | A kind of industrial environment outdoor scene enhanced exchange method |
| CN107168520A (en) * | 2017-04-07 | 2017-09-15 | 北京小鸟看看科技有限公司 | Method for tracing based on monocular cam, VR equipment and VR helmets |
| CN107392961A (en) * | 2017-06-16 | 2017-11-24 | 华勤通讯技术有限公司 | Space-location method and device based on augmented reality |
| WO2018072593A1 (en) * | 2016-10-21 | 2018-04-26 | 北京小鸟看看科技有限公司 | Virtual reality system, spatial positioning device and positioning method thereof |
| CN108076339A (en) * | 2017-12-19 | 2018-05-25 | 歌尔股份有限公司 | The AR equipment and application method that a kind of visual field can continuously extend |
| CN108205373A (en) * | 2017-12-25 | 2018-06-26 | 北京致臻智造科技有限公司 | A kind of exchange method and system |
| CN108459808A (en) * | 2017-02-20 | 2018-08-28 | (株)U欢乐世界 | With the display relevant augmented reality interaction systems of image theme and its method for running |
| CN109240483A (en) * | 2017-05-12 | 2019-01-18 | 上海华博信息服务有限公司 | A kind of VR motion editing system |
| CN109388142A (en) * | 2015-04-30 | 2019-02-26 | 广东虚拟现实科技有限公司 | A kind of method and system carrying out virtual reality travelling control based on inertial sensor |
| CN110622219A (en) * | 2017-03-10 | 2019-12-27 | 杰创科增强现实有限公司 | Interactive augmented reality |
| CN113296605A (en) * | 2021-05-24 | 2021-08-24 | 中国科学院深圳先进技术研究院 | Force feedback method, force feedback device and electronic equipment |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105721857A (en) * | 2016-04-08 | 2016-06-29 | 刘海波 | Helmet with double cameras |
| CN107368187A (en) * | 2017-07-12 | 2017-11-21 | 深圳纬目信息技术有限公司 | A kind of head-mounted display apparatus of dual interactive controlling |
| CN112025735A (en) * | 2020-09-10 | 2020-12-04 | 河南工业职业技术学院 | Passive compliant robot polishing device based on visual perception |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101387908A (en) * | 2007-09-10 | 2009-03-18 | 佳能株式会社 | Information-processing apparatus and information-processing method |
| CN101808250A (en) * | 2009-02-13 | 2010-08-18 | 北京邮电大学 | Dual vision-based three-dimensional image synthesizing method and system |
| US20120113209A1 (en) * | 2006-02-15 | 2012-05-10 | Kenneth Ira Ritchey | Non-Interference Field-of-view Support Apparatus for a Panoramic Facial Sensor |
| CN202870727U (en) * | 2012-10-24 | 2013-04-10 | 上海威镜信息科技有限公司 | Display unit device with motion capture module |
| CN103064188A (en) * | 2011-11-30 | 2013-04-24 | 微软公司 | Head-mounted display based education and instruction |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN203941499U (en) * | 2014-05-30 | 2014-11-12 | 贺杰 | A kind of action collection and feedback system based on stereoscopic vision |
-
2014
- 2014-07-10 CN CN201420382187.1U patent/CN203941499U/en active Active
- 2014-07-10 CN CN201410329067.XA patent/CN104090660B/en active Active
-
2015
- 2015-01-13 WO PCT/CN2015/070605 patent/WO2015180497A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120113209A1 (en) * | 2006-02-15 | 2012-05-10 | Kenneth Ira Ritchey | Non-Interference Field-of-view Support Apparatus for a Panoramic Facial Sensor |
| CN101387908A (en) * | 2007-09-10 | 2009-03-18 | 佳能株式会社 | Information-processing apparatus and information-processing method |
| CN101808250A (en) * | 2009-02-13 | 2010-08-18 | 北京邮电大学 | Dual vision-based three-dimensional image synthesizing method and system |
| CN103064188A (en) * | 2011-11-30 | 2013-04-24 | 微软公司 | Head-mounted display based education and instruction |
| EP2600331A1 (en) * | 2011-11-30 | 2013-06-05 | Microsoft Corporation | Head-mounted display based education and instruction |
| CN202870727U (en) * | 2012-10-24 | 2013-04-10 | 上海威镜信息科技有限公司 | Display unit device with motion capture module |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015180497A1 (en) * | 2014-05-30 | 2015-12-03 | 贺杰 | Motion collection and feedback method and system based on stereoscopic vision |
| CN104536579A (en) * | 2015-01-20 | 2015-04-22 | 刘宛平 | Interactive three-dimensional scenery and digital image high-speed fusing processing system and method |
| CN104539929A (en) * | 2015-01-20 | 2015-04-22 | 刘宛平 | Three-dimensional image coding method and coding device with motion prediction function |
| CN104699247B (en) * | 2015-03-18 | 2017-12-12 | 北京七鑫易维信息技术有限公司 | A kind of virtual reality interactive system and method based on machine vision |
| CN104699247A (en) * | 2015-03-18 | 2015-06-10 | 北京七鑫易维信息技术有限公司 | Virtual reality interactive system and method based on machine vision |
| CN109388142A (en) * | 2015-04-30 | 2019-02-26 | 广东虚拟现实科技有限公司 | A kind of method and system carrying out virtual reality travelling control based on inertial sensor |
| CN104991650A (en) * | 2015-07-24 | 2015-10-21 | 贺杰 | Gesture controller and virtual reality system |
| CN104991650B (en) * | 2015-07-24 | 2018-08-03 | 广东虚拟现实科技有限公司 | A kind of gesture controller and a kind of virtual reality system |
| CN105445937A (en) * | 2015-12-27 | 2016-03-30 | 深圳游视虚拟现实技术有限公司 | Mark point-based multi-target real-time positioning and tracking device, method and system |
| CN105445937B (en) * | 2015-12-27 | 2018-08-21 | 深圳游视虚拟现实技术有限公司 | The real-time location tracking device of multiple target based on mark point, method and system |
| CN106354253A (en) * | 2016-08-19 | 2017-01-25 | 上海理湃光晶技术有限公司 | Cursor control method and AR glasses and intelligent ring based on same |
| WO2018072593A1 (en) * | 2016-10-21 | 2018-04-26 | 北京小鸟看看科技有限公司 | Virtual reality system, spatial positioning device and positioning method thereof |
| CN106547458A (en) * | 2016-11-29 | 2017-03-29 | 北京小鸟看看科技有限公司 | A kind of virtual reality system and its space positioning apparatus |
| CN106768361B (en) * | 2016-12-19 | 2019-10-22 | 北京小鸟看看科技有限公司 | The position tracking method and system of handle matched with VR helmet |
| CN106768361A (en) * | 2016-12-19 | 2017-05-31 | 北京小鸟看看科技有限公司 | The position tracking method and system of the handle supporting with VR helmets |
| CN108459808A (en) * | 2017-02-20 | 2018-08-28 | (株)U欢乐世界 | With the display relevant augmented reality interaction systems of image theme and its method for running |
| CN106899599A (en) * | 2017-03-09 | 2017-06-27 | 华东师范大学 | A kind of industrial environment outdoor scene enhanced exchange method |
| CN110622219A (en) * | 2017-03-10 | 2019-12-27 | 杰创科增强现实有限公司 | Interactive augmented reality |
| CN110622219B (en) * | 2017-03-10 | 2024-01-19 | 杰创科增强现实有限公司 | Interactive augmented reality |
| CN107168520A (en) * | 2017-04-07 | 2017-09-15 | 北京小鸟看看科技有限公司 | Method for tracing based on monocular cam, VR equipment and VR helmets |
| CN109240483A (en) * | 2017-05-12 | 2019-01-18 | 上海华博信息服务有限公司 | A kind of VR motion editing system |
| CN107392961A (en) * | 2017-06-16 | 2017-11-24 | 华勤通讯技术有限公司 | Space-location method and device based on augmented reality |
| WO2018227900A1 (en) * | 2017-06-16 | 2018-12-20 | 华勤通讯技术有限公司 | Augmented reality-based spatial positioning method and apparatus |
| CN107392961B (en) * | 2017-06-16 | 2019-12-06 | 华勤通讯技术有限公司 | space positioning method and device based on augmented reality |
| US10997745B2 (en) | 2017-06-16 | 2021-05-04 | Huaqin Technology Co., Ltd. | Augmented reality based method and apparatus for spatial positioning |
| CN108076339B (en) * | 2017-12-19 | 2019-07-05 | 歌尔股份有限公司 | A kind of AR equipment and application method that the visual field can continuously extend |
| CN108076339A (en) * | 2017-12-19 | 2018-05-25 | 歌尔股份有限公司 | The AR equipment and application method that a kind of visual field can continuously extend |
| CN108205373A (en) * | 2017-12-25 | 2018-06-26 | 北京致臻智造科技有限公司 | A kind of exchange method and system |
| CN113296605A (en) * | 2021-05-24 | 2021-08-24 | 中国科学院深圳先进技术研究院 | Force feedback method, force feedback device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2015180497A1 (en) | 2015-12-03 |
| CN104090660B (en) | 2017-11-10 |
| CN203941499U (en) | 2014-11-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN203941499U (en) | A kind of action collection and feedback system based on stereoscopic vision | |
| CN104699247B (en) | A kind of virtual reality interactive system and method based on machine vision | |
| JP7095073B2 (en) | Robot as a personal trainer | |
| CN112513711B (en) | Method and system for resolving hemispherical ambiguities using position vectors | |
| JP7095602B2 (en) | Information processing equipment, information processing method and recording medium | |
| CN105027030B (en) | The wireless wrist calculating connected for three-dimensional imaging, mapping, networking and interface and control device and method | |
| US20150070274A1 (en) | Methods and systems for determining 6dof location and orientation of head-mounted display and associated user movements | |
| CN202150897U (en) | Body feeling control game television set | |
| US20110118877A1 (en) | Robot system and method and computer-readable medium controlling the same | |
| CN108700939A (en) | System and method for augmented reality | |
| CN103793060A (en) | User interaction system and method | |
| WO2014156706A1 (en) | Image processing device and method, and program | |
| US20150170419A1 (en) | Video processing device, video processing method, and video processing system | |
| CN103345064A (en) | Cap integrated with 3D identifying and 3D identifying method of cap | |
| CN102769802A (en) | Man-machine interactive system and man-machine interactive method of smart television | |
| KR20120068253A (en) | Method and apparatus for providing response of user interface | |
| JP2011215920A (en) | Program, information storage medium and image generation system | |
| CN108983982A (en) | AR aobvious equipment and terminal device combined system | |
| CN104243962A (en) | Augmented reality head-mounted electronic device and method for generating augmented reality | |
| JP2012115414A (en) | Game device, method of providing game, game program, and game system | |
| CN108885487B (en) | Gesture control method of wearable system and wearable system | |
| CN108292490A (en) | Display control unit and display control method | |
| CN206584310U (en) | A kind of MR gloves being applied in virtual reality | |
| CN106873775A (en) | A kind of implementation method of virtual reality interaction, system and MR gloves | |
| US20140045593A1 (en) | Virtual joint orientation in virtual skeleton |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C41 | Transfer of patent application or patent right or utility model | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20160418 Address after: 510335 international procurement center, 8, Pazhou Avenue East, Guangzhou, Guangdong, Haizhuqu District 1401 Applicant after: XIMMERSE, LTD. Address before: 510000 No. 680 East Whampoa Road, Guangdong, Guangzhou Applicant before: He Jie Applicant before: Hong Jianjun |
|
| CB03 | Change of inventor or designer information |
Inventor after: Dai Jingwen Inventor after: He Jie Inventor before: He Jie Inventor before: Hong Jianjun |
|
| CB03 | Change of inventor or designer information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |