WO2013082387A1 - Plateforme de vision stéréoscopique tridimensionnelle (3d) à large champ de vision ayant une commande dynamique d'opération d'affichage immersif ou d'affichage tête haute - Google Patents
Plateforme de vision stéréoscopique tridimensionnelle (3d) à large champ de vision ayant une commande dynamique d'opération d'affichage immersif ou d'affichage tête haute Download PDFInfo
- Publication number
- WO2013082387A1 WO2013082387A1 PCT/US2012/067234 US2012067234W WO2013082387A1 WO 2013082387 A1 WO2013082387 A1 WO 2013082387A1 US 2012067234 W US2012067234 W US 2012067234W WO 2013082387 A1 WO2013082387 A1 WO 2013082387A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- mirror
- eye
- image
- series
- images
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
- H04N13/344—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/366—Image reproducers using viewer tracking
- H04N13/383—Image reproducers using viewer tracking for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/011—Head-up displays characterised by optical features comprising device for correcting geometrical aberrations, distortion
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0179—Display position adjusting means not related to the information to be displayed
- G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
Definitions
- This invention relates generally to 3D stereo vision goggles or other platforms, that can be used for enhanced vision systems for use with endoscopic surgery, robotic assist surgery, open surgery, and surgical microscopes; as visual aids for patients with medical conditions such as Macular Degeneration; and for business and entertainment applications for which a 3D stereo vision display would be desirable
- robotic assist surgery uses a four arm robotic system to hold endoscopic tools.
- the surgeon performs the surgery by sitting at a remote console where he controls the robotic arms that are holding the endoscopic tools and camera.
- the robotic assist system uses a 3D endoscopic camera where the camera images are displayed on two high-resolution, high-definition flat screen displays.
- the surgeon views the two displays through two wide angle lenses.
- the surgeon can see the operation with depth perception and reasonable resolution, but a fairly narrow field-of-view.
- the surgeon must keep his head at a specific location and remain motionless in order to keep the stereo image in full display.
- the endoscope allows the surgeon to operate on a patient by making small incisions and inserting long thin tools used to conduct the operation with one hand, then inserting a long thin tool with a miniature camera at the end and holding it with the other hand. The surgeon coordinates the movement of the tools by viewing the operation on flat panel display. Endoscopy requires the surgeon to generally look at a flat screen monitor that is 2D and typically not at the optimal position. For example, the monitor is placed to the side such that the surgeon performs the surgery with his head turned to the side. This is unnatural compared to looking down at your hands as is done during normal open surgery.
- the ergonomic parameter is related to how natural or unnatural your body position is when using the tool, and is generally a measure of comfort.
- Architectures for new devices that can extend one or more visual parameters while maintaining the remaining parameters at a level equivalent to the unaided eye has proven to be elusive.
- a new architecture is required for medical vision systems that optimize all visual parameters.
- the present invention is a new type of 3D stereo vision goggle or other platform.
- the preferred embodiment of goggles include total horizontal display with a field-of-view of 120 degrees for each eye, and a 60 degree binocular overlap between both left and right eyes, a partially mirrored ellipsoid section on the inside of each side of the goggle that places an image directly on the back of the retina of each eye, and vergence focus based on real time eye tracking and control of eye gaze, creating a high-definition 3-D image for the wearer.
- Embodiments of the invention also include additional components and methods such as automatic alignment of the vision system to the wearer's eyes, dynamic control of immersive or see-through operation, a three axes head position sensor, and the ability to programmatically adjust the display to accommodate the wearer's eye prescription, including, but not limited to, magnification and astigmatism.
- the invention could also be used with vision systems for business and entertainment applications. Correction of geometric aberrations and distortion uses technology from several engineering fields, including optics, control theory, electronics, mechanical engineering, and software engineering.
- Figure 1 is a block diagram of the image path for each eye within one embodiment of the vision platform.
- Figure 2 is a block diagram of the main hardware components of one embodiment of the entire vision system platform.
- Figure 3 is a block diagram of a processor that acts as a display controller.
- Figure 4 is a top view of one embodiment of the goggle.
- Figure 5 illustrates the general alignment of the goggle described herein and how an ellipsoid section is selected and used for the goggle design.
- Figure 6(a) is a diagram showing a side view of one embodiment of the goggle.
- Figure 6(b) shows how the goggle's outside lens fits with respect to the ellipsoid mirror.
- Figure 6(c) is a front view of one embodiment of the goggle.
- Figure 7 is a diagram showing how an image from a display buffer for one eye is pre-distorted using a mapping algorithm.
- Figure 8a illustrates one embodiment of a three dimensional ellipsoid with two foci shown as black dots, and a section of the ellipsoid used to design the goggle or other embodiment of the invention.
- Figure 8b illustrates two of the ellipsoid sections from Figure 8a placed side-by-side in the same matter used by the invention.
- Figure 9 illustrates light taken as a ray emitted from one foci of an internally mirrored ellipsoid through an optical subassembly and through another foci located in a wearer's eye.
- Figure 10 shows the internal construction, OLED, lenses, and mirrors of the optical subassembly shown in Figure 9
- Figure 1 1 is a diagram that shows the rear view of a large field-of-view image segmented into six smaller frames and a hexagon mirror is used to reflect the six frames to their assigned position.
- Figure 12 is a timing diagram that is used to synchronize the servo controlled hexagon mirror's images at a specific update rate.
- Figure 13 shows how the stylized image and its six segmented frames described in Figure 1 1 will appear when the image is projected off the ellipsoid mirror from the hexagon mirror shown in figure 1 1 .
- Figure 14 is a top view of goggle manufacturing equipment used to correct and update the lookup table.
- Figure 15 is a graph showing the human eye's visual acuity versus field of view.
- Figure 16 is a wide field of view, high-resolution camera attached to a two axes gimbal system.
- Figure 17a is a neutral density filter for a high- resolution narrow field of view camera.
- Figure 17b is a neutral density filter for a low-resolution wide field of view camera.
- Figure 18 is a front view of goggle with camera module.
- Figure 19a is a simulation of an output image from the high resolution wide field of view camera.
- Figure 19b is a magnification of the simulated image in figure 19a.
- the embodiment of the invention presented in this section consists of a vision system with external image processing and a display electronic package that enables the wearer to receive a 3D stereoscopic image directly on the retina of their eyes.
- the invention described herein can be applied to other display technologies, such as movie displays and displays used for business applications, and provide viewers with the benefits of the invention described herein.
- the invention applies methods from multiple engineering disciplines, such as system design, optical design, electrical design, mechanical design, control theory, and software in a goggle vision system with the primary features of high resolution, improved acuity, wide field-of-view, superior depth perception, and focusing based on vergence control.
- One embodiment of this invention looks like a pair of ski goggles.
- An organic light emitting diode (OLED) array is used to generate an image that is projected through a series of lenses, reflected off a spinning polygon mirror, and reflected off mirrors. The image then reflects off the final mirror that has the shape of a section of an ellipsoid. Images are generated to seem as if it emanated from one of the ellipsoid's two foci, then the ellipsoid sector is oriented such that the other focus point is at the goggle wearer's eye's center-of-rotation. The image reflecting off of the ellipsoid section places the image onto the retina of the wearer.
- FIG. 1 A block diagram of the image path within an embodiment of the complete vision system platform is shown in Figure 1 .
- Multiple camera types such as standard 2D camera 1504, medical endoscope 1505, 3D cameras 1503, and a new type of camera called a vergence controlled high resolution camera 1502, can be used.
- Computer generated images 1501 can also be used, such as if the system is used for a 3D computer simulation or for entertainment. The details of a vergence controlled camera are described below.
- Image data from the camera or other input flows initially into the display controller's buffers 1507.
- the memory size of each buffer and "Pre-Distort" buffer 1507b and 1507d is large enough to contain all of the pixels for an entire image plus metadata for each pixel.
- the buffers are segmented into equal parts which each segment has the memory size equal to the number of pixels in the OLED array 1315. The number of segments is equal to the number of facets on the polygon mirror 501 . From the buffers 1507a and 1507b the data is mapped to "Pre-Distort" buffers 1507b and 1507d. The mapping takes into consideration distortions caused by the optical path as well as the wearer's eye prescription, as described further in Figure 7.
- the display controller transmits the data from the "Pre-Distort" buffer in segments to the goggles. Additional input from sensors 1517 and eye tracking 1516 can modify how the data is mapped into the Pre-Distort buffer.
- the Pre-distort buffer is continuously updated to the goggle communication interface 1508. Some embodiments can have more than one goggle to update, such as solutions applied to surgery. During most surgeries there are multiple surgeons present, or there may be medical students present.
- the application specific module 1506 provides multiple attachment ports so several goggles could see the same image as the primary surgeon. [0041] Display data received by the goggle 1508 is moved using Direct Memory Access (DMA) to the Organic Light Emitting Diode (OLED) array 1509.
- DMA Direct Memory Access
- OLED Organic Light Emitting Diode
- the image formed by the OLED goes through a set of lenses in the lens package 1510.
- the output of the lens package reflects off of a folding mirror then reflects off a servo controlled polygon mirror 151 1 .
- the image coming off the polygon mirror reflects off of a specific sector of an ellipsoid mirror 1512 which has one of its focus points positioned at the center of rotation of the wearer's eyes.
- the image passes through the wearer's pupil and lens then onto the wearer's retina 1513.
- Using a polygon mirror is one way of displaying the image.
- the repetitive update of all of the segments using the same sequence is called raster scanning.
- the system could use vector scanning.
- Vector scanning allows any segment to be updated in any order. Implementation is typically done with two independent mirrors and two motors. Since each mirror is attached to its own motor, each mirror can be placed at any angle. This allows segments to be displayed in any sequence. While there are only two ways of updating the segmented image, which are raster scanning and vector scanning, there are multiple ways of creating the segments that constitute in aggregate the image.
- One embodiment of the invention uses an OLED as a light emitting device for generating a small piece of the image which is called a segment.
- OLED Liquid Crystal Display
- LED Light Emitting Diode
- laser scanning The latter can use both raster and/or vector scanning.
- Additional features include but are not limited to dynamic control of the lens magnification 1515 and velocity control of the motor driving the polygon 1514.
- the velocity must stay locked at the 60 hertz rate.
- the 60 hertz frequency is locked in frequency and phase using feedback control theory. When the vertical pulse frequency varies, the entire frame shifts up and down, causing disorientation to the wearer.
- Real time eye tracking 1516 over the entire eye movement envelope is an essential function to support vergence control and vergence focus.
- the eye makes large angular movements called saccadic motion. This motion can result in the eye reaching maximum velocities of 500 degrees/second and accelerations of 2000 degrees/sec A 2.
- Fast sample rate cameras with sample rates of 200 hertz or higher are required to adequately track the position of the eye.
- Some embodiments require special processing that is not available in the display controller, such as surgical eye loupes used to magnify the operating field seen by the surgeon. Sensor data input from the goggle's accelerometers and image processing from stereo 3D cameras are required for this embodiment. A separate application specific module 1506 is used to support the embodiment's requirements.
- FIG. 1 A block diagram of hardware components of one embodiment of the vision platform is shown in figure 2.
- the main components are the goggles 1203 and a processor that acts as a display controller 1206.
- a cable assembly 1204 connects the goggle and display controller.
- the cable assembly 1204 comprises two copper wires for power and ground, and four fiber optic cables.
- the optic link is used for sending images to the goggles which then present the images to the retinas of the left eye and the right eye.
- the left and the right camera images for the camera module 1201 are sent from the camera module to the display controller, along with other data such as diagnostics, code debug, and error codes.
- the goggle 1203 comprises a Left Eye Projection Module 1202 and Right Eye Projection Module 121 1 and a Camera Module 1201 , which may be mounted to the goggle or not mounted to the goggle.
- Each projection module comprises a self- contained set of optical, electrical and mechanical parts.
- Each projector module has the functionality to align to the wearer's eyes.
- the display controller 1206 can operate in multiple configurations. For example, for patients with retinal diseases, such as Macular Degeneration, the display controller is mobile and runs on a battery. The controller is small enough for the patient to wear it on their hip or could be attached to the goggles themselves. Another configuration is in support of surgeries.
- the display controller configuration for surgical applications consists of the image processing electronics, software and display electronics.
- a detailed block diagram of the display 1206 controller is shown in figure 3.
- the Display Controller has five external interfaces. a. Goggle electrical and optical connector 1401 .
- External electrical and optical connector 1409 This is for use by third party companies to integrate the vision platform into their products.
- Ethernet port 1408 which is used for software development and diagnostics.
- Power module interface 1407 which supports two types of power modules, one for AC and the other for battery power.
- the display buffers 1403 and 1405 receive data from external video inputs.
- Each display buffer 1403 and 1405 consists of two internal buffers, as shown further in 1507 of Figure 1 .
- the first internal buffer receives the incoming camera data and the second takes the data from the first buffer and pre-distorts the data.
- the pre-distortion corrects for Keystoning, distortion caused by curved mirrors, and wearer's eyeglass prescriptions.
- the data from the pre-distortion buffer is transmitted to the output buffer in the goggle's projector and camera module 1401 .
- the macular centered display buffers 1402 and 1404 receive camera data from the camera module located in the goggles.
- Each buffer 1402 and 1404 contains two sub-buffers, one for high resolution narrow field-of-view images and the second for wide field-of-view peripheral images.
- the images are mixed together with the narrow field-of-view high resolution image placed where the wearer's eyes are pointed. This is part of the concept called vergence control. Vergence control is covered in greater detail below in the discussion of the new camera developed as part of this invention.
- This invention aligns and adapts to the differences in a wearer's facial geometry; such as differences in the width of a wearer's eyes, the vertical height of one eye to the other, or if a wearer has a flat forehead or a slopped forehead.
- This invention can align to each eye. This is accomplished by making the printed circuit boards dedicated to an axis. For example there is an x axis and a y axis. The z axis moves in a linear direction and in a rotational pitch direction. This provides a total of four dimensions for each eye.
- the previous components and modules when form a vision platform can be customized for a specific application, be it patient use, surgical applications, or other applications.
- One embodiment of this invention has an appearance very similar to a standard ski goggle.
- An example of such a general goggle structure is shown generally in Figures 5a, 5b, and 5c, with a specific embodiment of the goggle structure shown in Figures 6(c), showing a front view, and Figure 4, showing a top view.
- the components on the right side of the goggle for projecting images onto the retina of the right eye in Figure 4 are replicated on the left side of the goggle for the left eye.
- the drawing shown in Figure 4 is the top view of the goggle.
- the sixth is a global coordinate system that establishes the reference point among the local coordinate systems. Mathematically, the reference point is defined using homogeneous transfer equations that are common in computer graphics and robotic control.
- PCBs printed circuit boards
- XR and ZR (YR is going into the page) are the local references for the x, y and z axes of the right projector.
- XL and ZL (YL is going into the page) are the local projector references for the x, y, and z axes for the left projector.
- X and Z are the global coordinate system.
- the x axis motor 1307 can move the PCB which is attached to bushings 1304 and shaft 1305.
- the x axis motor can move the PCB approximately plus or minus .250 inches from its nominal position.
- the y axis PCB 131 1 is attached to the x axis board 1305 through bushing and shafts 1318. Movement of the x axis board also moves the y axis board.
- the z axis board 1310 is attached rigidly to the y axis board 131 1 using spacers.
- the z axis motor 1309 is attached to the bracket 1312. The entire optical system is attached to the bracket 1312.
- the optical system consists of elements annotated by focus adjustment motor 1301 , ellipsoid mirror 1302, OLED driver board 1303, polygon mirror 1313, ball bearings 1314, and bearing pillow blocks 1317. There are two more elements not shown in Figure 4 because OLED 1315 is occluding the parts.
- the occluded elements are Lens Assembly 1 1 14, and Prism 1 1 13 as shown in Figure 6c.
- the goal of the moving x, y, and z, axes is to provide a means to automatically position the second ellipsoid focal point 304, at the center of rotation of the wearer's eye, as shown in Figure 9 and discussed below. All three boards are moved in a coordinated fashion in order to achieve this goal. This process is done independently for the both right and left eye projectors.
- the optical path for projecting an image on an eye's retina photonically starts at the Organic Light Emitting Diode (OLED) array 1315, which is part of an optical subassembly.
- OLED Organic Light Emitting Diode
- a generalized view of the optical subassembly 413 is shown in Figure 10.
- the subassembly comprises OLED 405, a lens assembly 406a, a prism 406b, a first stationary folding mirror 407, the polygon mirror 408, and a folding mirror 409.
- the lens assembly 1 1 14 in Figure 6c also can vary the magnification or diopters from -5 to + 3 using motor 1301 .
- a prism 1 1 13 in Figure 6c translates the image to the first of four mirrors. The image leaves the prism 1 1 13 collimated.
- the polygon mirror servo motor 1303 is illustrated in Figure 4.
- the polygon mirror 1313 is mounted between two solid and ridge structures called pillow blocks 1317.
- the polygon's shaft is attached to the pillow block through ball bearings 1314.
- the final element of the optical path is the ellipsoid mirror section 1302.
- a small electric motor 1301 shown in Figure 4, adjusts only the refractive elements in the optical subsystem 1315.
- the remaining optical components consist of reflective elements.
- the first is adjusting the focus of the wearer to the display and the second is focusing the camera platform to the gaze of the wearer. Adjusting the focus of the goggles is achieved by entering the wearer's prescription. The software in the display controller will then adjust the focus to control the goggle to match the wearer's prescription.
- the second involves using a camera module option that plugs into the goggles.
- the camera module consists of two high resolution camera assemblies with two rotational degrees of freedom for each camera. Eye tracking in the goggle is used to move the left and right eye cameras in coordination with the wearer's left and right eye, as described in further detail below.
- a curved plate is pressed against the wearer's forehead 1316. This plate maintains a consistent reference point after the alignment process is completed.
- FIG. 5 shows an illustration of how an ellipsoid sector is applied to the construction and function of the goggle 105.
- An ellipsoid 106 is mathematically defined by two foci as shown in 1 1 1 and 1 12.
- one of the focus points is chosen to be the image source 101 , 102, and 1 12. If the inside wall of an ellipsoid were mirrored, light emitted from one focus point would reflect off of the internal mirrored wall and go through the other focus point. This property is used by this invention to directly place an image on the back of the goggle's wearer's retina.
- a specific section of the ellipsoid was chosen because its location allowed the mirror to be mounted in the goggle 105.
- other considerations on choosing the location of the ellipsoid section are the field of view of the goggle and where to locate the image source.
- the image is formed on the retina by placing the second focus point at the center of rotation of the eye 1 1 1 .
- the following conditions must be met: a.
- the image source must seem to be emitted for the first focus point 101 , 102, and 1 12. This process is done when the device is manufactured. Once aligned, the first focus point should not move.
- the second focus point 1 1 1 must be placed at the center of rotation of the wearer's eye. This process must be done for every wearer.
- a process that consists of a set of special alignment targets is presented to the wearer in sequence. Initial alignment begins with finding the center of gaze, which is accomplished by placing targets virtually straight in front of the goggle. Both projectors are used with the distance calibrated to a known distance away. Then using a sequence of spiral of dots, each dot is presented individually to the wearer. The last test is showing several 3D photos with single objects presented in the fore field at different distances. For each picture the location of the wearer's left and right eyes are temporally stored. After the testing is completed a mapping of distance to eye triangulation is calculated. Once done for the wearer the settings can be stored.
- the ellipsoid section covers the field of view of the eye. This process is completed during the design and manufacturing phase.
- the goggle is held against the wearer's face by a wide elastic strap that is under tension 1 1 10.
- Power, communication, and the image displayed are sent to the goggle through a cable assembly comprising four fiber optic cables and two copper wires. This cable plugs into a connector that is mounted on the back side of the wearer's head 108, 109.
- lengths a, b, and c are called the semi-axes.
- the shape generated is called a spheroid if two of the semi-axes are the same. If c ⁇ a the shape is called an oblate spheroid and if c>a the shape is called a prolate spheroid. When all three axes are different the shape is called a triaxial ellipsoid. If all three axes are the same the shape is a sphere.
- One embodiment of the invention uses a spheroid where the two common axes have the same radius of 2 inches and the third axis has a radius of 3 inches.
- a computer generated ellipsoid 201 shown in Figure 8a illustrates a 2x2x3 ellipsoid.
- Spherical parametric equations, (2), (3), (4) derived from the common ellipsoid equation (1 ) are used to determine the best mirror location within a spheroid and relative to a wearer's eye.
- x a cos u sin t> (2)
- FIG. 8a An illustration of one of methods how a wide FOV image is placed onto the eye's retina is shown in Figure 8a.
- An ellipsoid mirror section 204 is positioned in front of the eye where the ellipsoid's focus point 203 is positioned to the center of rotation of the wearer's eye. Both refractive and reflective optics condition the image such that it seems to be originating from the ellipsoid's other focal point 202.
- FIG. 8b The design criterion for selecting the mirrored section is shown in Figure 8b where one ellipsoid mirror section is placed in front of each eye.
- the major axis of the ellipsoid is rotated from the horizontal to allow for sufficient clearance between the projector module and the wearer's forehead.
- the image source has to emanate or seem to emanate from the source foci 207 and 208.
- Two images emanating from 207 and 208 reflect off of the mirrored surfaces of the ellipsoids 205 and 206. Then the reflected rays go through foci 209 and 210, presenting an inverted image on the back of the wearer's retina.
- Figure 9 shows the basic application of the ellipsoid shape and mirror to the invention.
- the foci of the ellipsoid are f1 301 and f2 304.
- the ellipsoid main axis is shown at 302.
- the optical axis is moved "off-axis" such that the image source can be located above the wearer's head while reflecting off the ellipsoid mirror 305 and going through the focus point 304.
- Focus point 304 is positioned at the center of rotation of the wearer's eyes.
- An optical subassembly 306 is placed between the source focus point 301 and ellipsoid 305.
- the optical subassembly has the characteristics that the light rays emitted from 306 should trace backwards to focus point 301 . This will allow the light to reflect off the ellipsoid mirror and through the eye's center of rotation focusing on the wearer's retina.
- Figure 10 shows one embodiment of the optical subassembly.
- the ellipsoid's major axis 401 and the two foci 403 and 404 are shown.
- the light emitting segmented image source is an Organic Light Emitting Diode (OLED) 405.
- OLED Organic Light Emitting Diode
- the output from the OLED goes through multi lens system 406 that reduces the magnification of the image and provides a dynamic focus control of the light reflects off a relay mirror 407, a servo controlled polygon mirror 408, and another relay or folding mirror 409. Light emitted from 409 appear as if the entire image originated from focus point 403.
- the light rays reflect off the ellipsoid mirror, with the emitted rays 410a and 412a.
- the reflected rays are 410b and 412b.
- the light rays that reflect off the ellipsoid mirror 41 1 are focused through the ellipsoid's second focus point 403. This focus point must be aligned to go through the wearer's eye's center of rotation.
- Figure 1 1 shows a simplified segmented image.
- the individual segments or frames are labeled 1 through 6 as illustrated in 506.
- a polygon mirror 501 has one facet for each segment, each facet on the polygon is machined to an angle that will reflect an image emitted by the OLED to a specific segment as shown in Figure 13.
- Each facet also has its own magnification that is unique for that specific segment.
- the polygon mirror 501 is servo controlled to rotate at a fixed velocity of 60 HZ.
- Optical couplers are attached to the polygon and provide a method to synchronize the projected frames and the facets.
- the synchronization is implemented using several electronic signals which can be seen in Figure 12.Timing of all of the frames requires an initial vertical sync as shown in 901 .
- An opto coupler is used to detect a detent in the side of the polygon mirror.
- the pulse 905a is aligned to occur right before the 1 st mirror facet.
- Internal microprocessor timers are loaded with a value that when reached equals a delay in t1 shown in timing diagram between signals 901 and 902. At this same time the first facet is aligned with the first row of frames.
- Each positive pulse of signal 901 represents a new set of frames or the beginning of a new image.
- the next signal is the row sync as shown in signal 902.
- This signal is also generated using an internal timer in the microprocessor.
- the row sync timer creates a repeatable delay after the start of the vertical sync.
- the two pulses 906a and 906b in between the two vertical sync pulses 905a and 905b indicate the beginning of two rows.
- the last signal is the column sync signal 903.
- the start of the row sync pulse triggers the column counter to start counting. Once the delay is reached (signal 904a, delay t2) the column row pulse is generated.
- the fourth timer controls the delay and generation of the two remaining column sync pulses 904b and 904c.
- the three column sync pulses that occur between the row pulses determine when the projector emits an image.
- the numbers 1 , 2, and 3 above the first set of three pulses 904a-c and the second set of three pulses 4, 5, and 6 shown in figure 12 occur on signal 903 and are directly correlated with the image segments 506 shown in Figure 1 1 .
- facet 6's column pulse changes from low to high
- the display controller will have loaded segment 6 into the OLEDs array buffer.
- pulse 6 will trigger the OLED array to be pulsed on for 6 microseconds.
- the light from the OLED strikes the surface of the polygon's facet 6, it is at an angle to reflect the image to segment 6 on image 506.
- the OLED array projects an image
- the image is reflected off a rotating polygon mirror.
- the mirror rotates continuously the image being projected on each facet is turned off before it is smeared. That limit is half of the angular translation between two pixels. This results in the image for each image segment being projected onto the wearer's retina for approximately six microseconds.
- Delivering a clear and undistorted image requires applying several modes of correction to compensate for every distortion source.
- the distortion correction is spread across several sub systems. For example, geometric aberrations and magnification variation are compensated for in the polygon's facet design.
- the mirrors and other optical elements are used to correct distortion due to geometric aberration and magnification variation across the frame. Luminosity variation across each of the image's sub-frames is compensated at the image source by current control of the pixels across the Organic Light Emitting Diode array (OLED) using a lookup table.
- OLED Organic Light Emitting Diode array
- the lookup table is constructed as a separate RAM buffer that is associated with the pre-distort buffers.
- One method to implement a lookup table is a structure of data for every pixel in the pre-distort buffer. An example is shown in Table 1 .
- mapping The system corrects for the effects of Keystoning and distortion by empirically measuring the Keystoning and distortion during manufacturing and placing the data in the lookup table 1003 shown in figure 7. This method is called mapping and is dependent on the superposition principle, which assumes that Keystoning, distortion, wearer's prescription, and dynamic inputs can be treated individually with their correction terms summed together. This results in each pixel's location being offset by the addition of the terms shown as inputs to the mapping algorithm 1007. [0087] The mapping algorithm's goal is to pre-distort the image's pixels as they are moved to the display buffer 1008, such that when the image is pulsed onto the eye, the wearer sees a clear and undistorted image.
- the pre-distorted image mapping expands all image segmented frames to a pre-distorted image larger than the image source.
- the oversized pre-distorted image is scaled to "fit" with the image buffer space.
- each image pixel's color is applied to gap pixels created by the pre-distortion algorithm 1007.
- the algorithm executes a standard graphics transformation matrix that rotates, translates, and scales the image. This process is performed during the engineering development of the device. The effects of distortion due to the ellipsoid and the retina are loaded and stored in a lookup table. During normal operation the lookup table describes the offset required for each pixel (See Table 1 ). The size and the value within the lookup table are defined by the number of segments required to display an image.
- a simple, but modified Direct Memory Access (DMA) is executed on the buffer ram.
- a traditional DMA consists of a custom ASIC that has internal registers for data source, data destination, and size of data package. The modified DMA differs from the traditional DMA because of the additional process before and after the move is complete.
- the Display Controller Processor Before the DMA is turned on, the Display Controller Processor writes the number of bytes to transfer, source address and destination address into the DMA Controller's internal registers.
- DMA is the typical method to transfer large amounts of data without consuming the Display Controller's Processor code execution time. Alternative solutions could use the Display Controller's Processor directly to move the data.
- the segment number is stored as a variable in the lookup table.
- the segments are transmitted to the goggle in numerical sequenced DMA transmission to the goggles over fiber optic cables.
- the metadata defined in Table 1 is sent with the data as well.
- the luminosity value must be set and the Pixel-Pixel Radius sets adjacent pixels to the same color as the Post-Distortion Pixel.
- the mapping is shown moving the pixel from the input buffer 1001 to the input to the multi input mapping algorithm 1007, to the pre-distortion buffer 1005.
- An example of illustrating the effect of pre- distortion to expand the image size is shown in 1006.
- the mapping function 1007 maps the pixel from the input buffer 1001 to a different and expanded position in the display buffer 108.
- the display buffer 1008 has "gaps" between pixels that are adjacent in the input buffer. The same pixel color is used to fill-in the spaces between input pixels.
- the lookup table 1003 is tested for accuracy and modified during manufacturing. As part of the final manufacturing process the finished goggle is placed into a custom designed goggle tester. The default values in the lookup table were developed during the engineering development phase. The manufacturing test removes variations in part dimensions by testing then modifying specific pixels in the image.
- An example of the goggle test is shown in Figure 14a. The goggle is placed in a clamp down holder in which the goggles are positioned in a vertical reference 1601 . Two glass hollow hemispheres are mounted where the wearer's eyes would typically be located. The posterior side of the hemispheres1606, shown in Figure 14b, are lightly frosted .
- a biconvex lens is mounted in same location as the lens of a typical human eye 1605, which is 17mm in front of the hemispheres.
- the lens system is designed to be a reduced eye, which means that a single lens 1605 and aperture 1607 are being used to model all of the refractive surfaces of the human eye and the varying refractive indexes at their interfaces.
- the cameras and glass hemispheres are mounted on two pivoting arms. Each arm's rotation axis passes through the eye's center of rotation. Several test images are then displayed on both left eye and right eye projectors. The image is displayed on the back of the frosted hemispheres.
- the wearer typically wears glasses his prescription can be entered and stored in the goggle's display controller. Correction for Myopia and Hyperopia retinal focus is achievable for most wearer's eye prescription 1002.
- Each projector has its own motor 1 1 12, as shown in Figure 6c, that can change the diopters of the projector system left and projector system right in the positive and negative direction.
- any astigmatism can also be corrected. This data is stored in a different and separate part of the display controller 1002. Astigmatism is corrected by entering the x axis rotation and magnification and then entering the y axis rotation and magnification. The calculation for correcting astigmatism requires numerous computation cycles. The high computational resources are minimized by conducting calculations when the goggles are not being used. Once the calculations are completed the display controller will store the results in the lookup table.
- Embodiments of the goggle can be either immersive or see-through.
- the goggles shown in Figure 6(b) contain an ellipsoid section which is semi-mirrored on the inside surface 1 106 as well as coated with special Liquid Crystals 1 105 that work on curved surfaces. By changing the voltage across the Liquid Crystal's pixels, light can be blocked or allowed to be transmissive.
- a three axes accelerometer 1 108 in Figure 6c is mounted in the goggle assembly. By applying a single integration or double integration to the outputs of the accelerometer the velocity and position of the head are determined respectfully.
- Knowing the position, velocity, and acceleration of the head, along with eye tracking allows the location and orientation of the eyes to be known at any time.
- One example of how the head position, velocity, and acceleration are applied is in endoscopic surgery.
- a single robotic arm that has multiple degrees of freedom can be controlled with head movement.
- a surgeon using this feature could control a robotic arm holding an endoscopic camera by moving his head. This will allow for the surgeon to have both hands free while performing endoscopic surgery.
- the surgeon could control the arm using head motion, and then disable the control when needed. This will allow the surgeon to continue with the surgery without having to keep his head motionless.
- the surgeon could operate in total immersive mode.
- the display controller can superpose, for example, a tumor in a liver. The tumor location and dimensions would be obtained by processing MRI or CT scans.
- Using the accelerometer for inertial tracking of the head allows the perspective of the tumor to change as the surgeon moves his head.
- the goggles In addition to sensors like the accelerometer the goggles have a specialized eye tracking ability. Two cameras and infrared light emitting diodes (LEDs) are used for each eye. A camera and LED are packaged together as shown in 1 103, 1 107 and 1 109 in Figure 6a. The camera has a minimum sample rate of 200 Hertz. The high sample rate is required to track the eye over its entire range of motion and throughout its acceleration and velocity profiles.
- LEDs infrared light emitting diodes
- the invention's ability to project a wide field of view and high acuity display onto the goggle wearer's eyes requires a video input with equal capabilities.
- the camera must provide vergence control for a more natural real time focus and 3D reality.
- the human eye has varying resolution.
- the highest resolution occurs at the fovea 1701 which has a resolution of 20/20 and a field of view of approximately 3 degrees.
- the peripheral region of the eye has a general resolution of 20/200.
- the brain takes the high resolution and the low resolution portions of an image projected onto the retina and transmits them over the optical nerve to the portion of the brain called the visual cortex.
- the visual cortex is located at the posterior part of the brain. This region takes the high resolution narrow field of view image and maps it across the low resolution wide field of view image. The result is that the entire image seems to be high resolution.
- the camera shown in figure 16 mimics the way the retina processes an image.
- a wide field of view lens 1801 starts the optical path for the camera.
- the image is collimated through lens assembly 181 1 before going through a beam splitter 1810.
- the image is now split into two paths, one is high resolution and the other low resolution.
- the low resolution path image goes through a neutral density filter shown in Figure 17b.
- the filter has three regions, the first is transparent 1905, the second is gradient 1906 which varies from transparent to opaque, and the last region is opaque.
- the low resolution path image goes through another lens package 1807 that focuses the image into the charge-coupled device (CCD) array of a wide angle camera 1805.
- CCD charge-coupled device
- the high resolution path starts from beam splitter 1910 then goes through a neutral density filter shown in Figure 17a that also has three regions.
- the first region is opaque 1901
- the second is a gradient from opaque to transparent 1902
- the third is transparent 1903.
- the portion of the image that passes through the filter is only a small central region measuring 10 degrees of field of view at the beginning of the gradient.
- the narrow image then goes through a lens assembly 1803 that magnifies and focuses the image onto the CCD array of camera 1804.
- the image has now been optically divided into a high and low resolution image and converted to an electrical signal stored in the CCD array of two cameras.
- the camera is part of a camera module that attaches to the top of the goggles shown in Figure 18, at 2001 and 2002.
- the camera module contains two cameras.
- the camera output is transmitted to the display controller and stored in the macular centered buffers 1402 and 1404.
- the display controller combines the high resolution and low resolution images from the cameras. From the macular centered buffers the image is moved to the pre-distortion buffers 1507b and 1507d, which are located in display buffer 1403 and 1405. From the pre-distortion buffers the images are sent back to the goggles to be projected onto the wearer's retina as described above.
- FIG. 19a The resulting output of this high-resolution, low-resolution and wide- field- of-view camera is shown in a simulated photograph figure 19a.
- the central region of the photograph has a high resolution which gradually blends to low resolution for the rest of the photograph.
- a magnified part of the central and blending regions of the photograph is shown in figure 19b.
- Vergence control captures two movements of human eyes. The first occurs when a person stares in the distance; the eyes diverge and are parallel as shown in Figure 20. When a person's gaze is near, the eyes converge and the focus is at a point 2101 which is a specific distance away.
- Wide-angle eye tracking is required such that the eye's position is known across the eye's entire field of view.
- Two miniature cameras 1 107 and 1 1 10 in Figure 6c and 1 103 in Figure 6a, each with an infrared LED, are used for each eye.
- the cameras are mounted at the bottom edge of the ellipsoid mirrors.
- the position of the eye at any moment is critical.
- Typical cameras sample at 60 to 120 frames per second. Sample rates of 60 to 120 hertz are too slow for accurate eye tracking.
- This invention uses a sampling camera of at least about 250 hertz which can track the eye even at the eye's maximum velocity, which is 500 degrees per second.
- the eye tracking data is used as an input to the camera servo system.
- the servo system does not attempt to continuously track the eye's movement with the camera. Instead the eye movement is broken down into stages.
- the eye starts at rest then the person moves the eye usually in a saccadic motion towards whatever has captured his attention.
- the eye eventually decelerates and comes to a stop.
- the human brain does not process the images coming from the retina during saccadic motion.
- the brain waits until the eye stops before the images become part of the person's conscience.
- the time it takes from the moment the eye stops moving until the time the brain completes processing the image varies from person to person.
- the range for most of the population is 30ms to 200ms.
- the servo system is designed to complete a move from where the eye started a saccadic move to where the eye ends the saccadic move in less than 15ms.
- Each camera shown in Figure 7 uses two motors to respond to the eye's saccadic movements.
- One motor controls the pitch angle 1809 and the other controls the yaw angle 1806 and 1807.
- the goggle processor coordinates the motion of these two motors by sending a position trajectory to the motor's servo code.
- the servo code then calculates a current command that is sent to the motor driver.
- the current command is calculated by the servo code and its value is proportional to the error in position between the position trajectory input and the current position of the camera angle
- vergence control for the vision platform.
- the primary benefit of vergence control is that it provides the goggle wearer with 3D stereo images that are very close to reality.
- Vergence control is achieved by mimicking two systems in the human body, retinal/brain physiology, and human eye dynamics and producing a high-definition narrow-field-of-view and low definition wide-field-of-view that moves with the wearer's eyes.
- the invention could provide an increased quality of life for individuals suffering from Macular Degeneration, other retinal diseases, brain injuries, or other visual shortcomings.
- the invention could use a video source mounted directly to the goggle or another portion of the wearer's body, which would supply an image directly to the retina of the individual.
- This invention also provides an enhanced vision solution for four types of surgical categories: robotic assist, general, endoscopic, and surgical microscopes.
- robotic assist general, endoscopic, and surgical microscopes.
- the advantages of the current invention are identified by category.
- Robotic assist surgery create a 3D stereo image by using two high definition LCD monitors.
- One monitor presents an image to one eye and the other monitor presents a similar image but from a slight horizontal offset from the image source.
- the two monitors along with two wide angle lenses and packaging produce a large costly system that requires a hydraulic mechanism to move the vision system into position.
- the image is clear but resolution is limited along with a narrow field-of-view.
- the surgeon must keep his head affixed in the same position for the duration of the surgery.
- the current invention separates the surgeon from a fixed viewing point. The surgeon can move around to a comfortable position, adjusting his or her head as desired while continuing to receive a clear, wide field-of-view, high resolution image.
- the current invention is also significantly more cost efficient that the large LCD solution.
- 3D cameras used with surgical microscopes that present a passive 3D image.
- the 3D image is seen on flat screen monitors only when special polarized glasses are worn.
- the hardware then alternates between the left and right eye polarization screens mounted on the front of the monitor.
- the current invention allows the surgeon to move independently from the microscope, compared to existing systems that require the surgeon to keep his eyes fixed, looking into the microscope's oculars.
- This invention using the microscope as the video input, gives the surgeon the freedom to sit or stand in a comfortable position. His head can be positioned naturally, looking at his hands.
- the current invention can display existing High Definition protocols, such as MP4, but the resolution will not be near the capabilities of the invention. In order to take full advantage of the invention's resolution capabilities, the camera described in Figures 16-19 is required. [00116] With the modularity and functionality of the invention it could be used for many other purposes. For example, the oil and gas and entertainment industries would benefit significantly from the invention.
- ROVs Remotely Operated Vehicles
- 3D data analysis Two primary applications in the oil industry would be remote piloting of Remotely Operated Vehicles (ROVs) and 3D data analysis.
- ROVs Remotely Operated Vehicles
- the 3D Stereo Vision Platform with 120 degrees field of view for each eye can greatly simplify ROV control.
- the wide field of view (FOV) coupled with the 3 axes accelerometers in the goggles could be used to offer natural head motion camera control.
- the ROV is moving, the high resolution and wide FOV makes it easy to make a multiple camera interface given to the navigator.
- Oil and gas companies also have very large data warehouses where they store many years of seismic data.
- the wide FOV and high acuity of the invention make it ideal to use if for the display and review of the 3D data.
- This is a camera-less based application in which the seismic data is rendered then sent to the display. If the optical interface to the goggle was used to display the seismic data then that would be an easy way to use the full resolution of the goggle vision platform.
- the first product that would be a natural fit for this invention is the ubiquitous gaming console.
- Virtual reality goggles and glasses have tried before with mixed success to penetrate this market. All of the products have only provided a seemingly larger display to look at and not addressed the total immersion effect. This invention would provide the total immersion affect or the missing feature that has limited widespread adoption of virtual reality in the gaming industry.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
Abstract
Conformément à des modes de réalisation, l'invention concerne d'une manière générale des lunettes de vision stéréoscopique tridimensionnelle (3D) ou d'autres plateformes qui pourraient être utilisées pour des systèmes de vision améliorés pour des applications chirurgicales, pour des patients ayant une dégénérescence maculaire, ou pour des applications de divertissement ou commerciales. L'invention prend des images reçues à partir d'une source d'entrée vidéo, et segmente et projette ces images hors d'un miroir défini par une partie d'un ellipsoïde et directement sur la rétine de l'œil d'un utilisateur. L'invention permet à des utilisateurs d'apprécier une vision stéréoscopique 3D avec un champ de vision accru, une qualité d'image accrue, un confort accru, un coût réduit et d'autres avantages.
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201280068856.1A CN104094162A (zh) | 2011-12-02 | 2012-11-30 | 动态控制沉浸或抬头显示操作的宽视场3d立体视觉平台 |
CA2857691A CA2857691A1 (fr) | 2011-12-02 | 2012-11-30 | Plateforme de vision stereoscopique tridimensionnelle (3d) a large champ de vision ayant une commande dynamique d'operation d'affichage immersif ou d'affichage tete haute |
EP12854278.4A EP2786196A4 (fr) | 2011-12-02 | 2012-11-30 | Plateforme de vision stéréoscopique tridimensionnelle (3d) à large champ de vision ayant une commande dynamique d'opération d'affichage immersif ou d'affichage tête haute |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201161566110P | 2011-12-02 | 2011-12-02 | |
US61/566,110 | 2011-12-02 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2013082387A1 true WO2013082387A1 (fr) | 2013-06-06 |
Family
ID=48536088
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2012/067234 WO2013082387A1 (fr) | 2011-12-02 | 2012-11-30 | Plateforme de vision stéréoscopique tridimensionnelle (3d) à large champ de vision ayant une commande dynamique d'opération d'affichage immersif ou d'affichage tête haute |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP2786196A4 (fr) |
CN (1) | CN104094162A (fr) |
CA (1) | CA2857691A1 (fr) |
WO (1) | WO2013082387A1 (fr) |
Cited By (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2821839A1 (fr) * | 2013-07-03 | 2015-01-07 | Airbus Defence and Space GmbH | Dispositif HMD avec dispositif d'eye-tracking réglable |
US20160091720A1 (en) * | 2014-02-21 | 2016-03-31 | Sony Computer Entertainment Inc. | Realtime lens aberration correction from eye tracking |
WO2016137662A1 (fr) | 2015-02-27 | 2016-09-01 | Welch Allyn, Inc. | Capture d'image rétinienne à foyer traversant |
WO2016168913A1 (fr) * | 2015-04-22 | 2016-10-27 | Esight Corp. | Procédés et dispositifs de correction d'aberration optique |
JP2017156547A (ja) * | 2016-03-02 | 2017-09-07 | パイオニア株式会社 | 画像表示装置、制御方法、プログラム及び記憶媒体 |
WO2017196270A1 (fr) * | 2016-05-13 | 2017-11-16 | Cy Vision Inc. | Micro-affichage à fibre de balayage |
CN108111874A (zh) * | 2016-11-16 | 2018-06-01 | 腾讯科技(深圳)有限公司 | 一种文件处理方法、终端及服务器 |
GB2557569A (en) * | 2016-09-09 | 2018-06-27 | Displaylink Uk Ltd | Adjusting display data |
WO2018100238A3 (fr) * | 2016-12-01 | 2018-07-12 | Varjo Technologies Oy | Système d'imagerie et procédé de production d'images de contexte et de mise au point |
US10147235B2 (en) | 2015-12-10 | 2018-12-04 | Microsoft Technology Licensing, Llc | AR display with adjustable stereo overlap zone |
JP2019512750A (ja) * | 2016-04-01 | 2019-05-16 | リニア アルジェブラ テクノロジーズ リミテッド | 人間の視覚的なメカニズムに適合したヘッドマウント型ディスプレイのためのシステム及び方法 |
WO2019106472A1 (fr) * | 2017-11-30 | 2019-06-06 | Tec Med S.R.L. Tecnologie Mediche | Système d'affichage immersif pour thérapies oculaires |
US10335029B2 (en) | 2014-02-11 | 2019-07-02 | Welch Allyn, Inc. | Opthalmoscope device |
US10376141B2 (en) | 2014-02-11 | 2019-08-13 | Welch Allyn, Inc. | Fundus imaging system |
US10413179B2 (en) | 2016-01-07 | 2019-09-17 | Welch Allyn, Inc. | Infrared fundus imaging system |
US10524653B2 (en) | 2015-11-02 | 2020-01-07 | Welch Allyn, Inc. | Retinal image capturing |
US10602926B2 (en) | 2016-09-29 | 2020-03-31 | Welch Allyn, Inc. | Through focus retinal image capturing |
JP2020513580A (ja) * | 2016-10-26 | 2020-05-14 | バルブ コーポレーション | 瞳孔位置を使用した光学レンズの歪み補正 |
US10758119B2 (en) | 2015-07-24 | 2020-09-01 | Welch Allyn, Inc. | Automatic fundus image capture system |
US10799115B2 (en) | 2015-02-27 | 2020-10-13 | Welch Allyn, Inc. | Through focus retinal image capturing |
US10949947B2 (en) | 2017-12-29 | 2021-03-16 | Intel Corporation | Foveated image rendering for head-mounted display devices |
US11096574B2 (en) | 2018-05-24 | 2021-08-24 | Welch Allyn, Inc. | Retinal image capturing |
Families Citing this family (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10338677B2 (en) * | 2015-10-28 | 2019-07-02 | Microsoft Technology Licensing, Llc | Adjusting image frames based on tracking motion of eyes |
GB2555378B (en) * | 2016-10-18 | 2021-06-09 | Virtually Live Switzerland Gmbh | HMD delivery system and method |
CN107490864A (zh) * | 2017-08-29 | 2017-12-19 | 杭州联络互动信息科技股份有限公司 | 一种用于vr头显的视频显示的方法、装置及vr头显 |
CN107462994A (zh) * | 2017-09-04 | 2017-12-12 | 浙江大学 | 沉浸式虚拟现实头戴显示装置和沉浸式虚拟现实显示方法 |
US10466489B1 (en) | 2019-03-29 | 2019-11-05 | Razmik Ghazaryan | Methods and apparatus for a variable-resolution screen |
US10554940B1 (en) | 2019-03-29 | 2020-02-04 | Razmik Ghazaryan | Method and apparatus for a variable-resolution screen |
US11284053B2 (en) | 2019-03-29 | 2022-03-22 | Razmik Ghazaryan | Head-mounted display and projection screen |
WO2021150921A1 (fr) | 2020-01-22 | 2021-07-29 | Photonic Medical Inc | Loupe numérique étalonnée, multimodale, à vision ouverte avec détection de profondeur |
US11758119B2 (en) | 2022-01-19 | 2023-09-12 | Rockwell Collins, Inc. | Low cost HUD using stereo and head tracking |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100013910A1 (en) * | 2008-07-21 | 2010-01-21 | Vivid Medical | Stereo viewer |
US20100238161A1 (en) * | 2009-03-19 | 2010-09-23 | Kenneth Varga | Computer-aided system for 360º heads up display of safety/mission critical data |
US20110018903A1 (en) * | 2004-08-03 | 2011-01-27 | Silverbrook Research Pty Ltd | Augmented reality device for presenting virtual imagery registered to a viewed surface |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
ATE357680T1 (de) * | 2000-10-07 | 2007-04-15 | David Dickerson | Informationssystem und verfahren zur zurverfügungstellen von informationen unter verwendung eines holographischen elements |
US7432879B2 (en) * | 2003-02-10 | 2008-10-07 | Schonlau William J | Personal viewer |
CN100367074C (zh) * | 2003-10-27 | 2008-02-06 | 深圳国际技术创新研究院 | 眼镜式显示器 |
CN2669214Y (zh) * | 2003-12-04 | 2005-01-05 | 一品光学工业股份有限公司 | 激光扫瞄装置的改良结构 |
FR2866123B1 (fr) * | 2004-02-10 | 2007-10-12 | Zile Liu | Procede et dispositif pour la creation d'images retiniennes utilisant le stigmatisme des deux foyers d'un dioptre sensiblement elliptique |
-
2012
- 2012-11-30 CN CN201280068856.1A patent/CN104094162A/zh active Pending
- 2012-11-30 EP EP12854278.4A patent/EP2786196A4/fr not_active Withdrawn
- 2012-11-30 CA CA2857691A patent/CA2857691A1/fr not_active Abandoned
- 2012-11-30 WO PCT/US2012/067234 patent/WO2013082387A1/fr active Application Filing
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110018903A1 (en) * | 2004-08-03 | 2011-01-27 | Silverbrook Research Pty Ltd | Augmented reality device for presenting virtual imagery registered to a viewed surface |
US20100013910A1 (en) * | 2008-07-21 | 2010-01-21 | Vivid Medical | Stereo viewer |
US20100238161A1 (en) * | 2009-03-19 | 2010-09-23 | Kenneth Varga | Computer-aided system for 360º heads up display of safety/mission critical data |
Non-Patent Citations (1)
Title |
---|
See also references of EP2786196A4 * |
Cited By (43)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2821839A1 (fr) * | 2013-07-03 | 2015-01-07 | Airbus Defence and Space GmbH | Dispositif HMD avec dispositif d'eye-tracking réglable |
US9535251B2 (en) | 2013-07-03 | 2017-01-03 | Airbus Defence and Space GmbH | HMD apparatus with adjustable eye tracking device |
US10674907B2 (en) | 2014-02-11 | 2020-06-09 | Welch Allyn, Inc. | Opthalmoscope device |
US10376141B2 (en) | 2014-02-11 | 2019-08-13 | Welch Allyn, Inc. | Fundus imaging system |
US10335029B2 (en) | 2014-02-11 | 2019-07-02 | Welch Allyn, Inc. | Opthalmoscope device |
US20160091720A1 (en) * | 2014-02-21 | 2016-03-31 | Sony Computer Entertainment Inc. | Realtime lens aberration correction from eye tracking |
US10228562B2 (en) * | 2014-02-21 | 2019-03-12 | Sony Interactive Entertainment Inc. | Realtime lens aberration correction from eye tracking |
JP2018501498A (ja) * | 2014-09-30 | 2018-01-18 | 株式会社ソニー・インタラクティブエンタテインメント | 眼追跡によるリアルタイムレンズ収差補正 |
CN107076989B (zh) * | 2014-09-30 | 2021-01-01 | 索尼互动娱乐股份有限公司 | 根据眼睛追踪的实时透镜像差校正 |
WO2016053735A1 (fr) * | 2014-09-30 | 2016-04-07 | Sony Computer Entertainment Inc. | Correction d'aberration de lentille en temps réel du suivi oculaire |
CN107076989A (zh) * | 2014-09-30 | 2017-08-18 | 索尼互动娱乐股份有限公司 | 根据眼睛追踪的实时透镜像差校正 |
US11045088B2 (en) | 2015-02-27 | 2021-06-29 | Welch Allyn, Inc. | Through focus retinal image capturing |
WO2016137662A1 (fr) | 2015-02-27 | 2016-09-01 | Welch Allyn, Inc. | Capture d'image rétinienne à foyer traversant |
US10799115B2 (en) | 2015-02-27 | 2020-10-13 | Welch Allyn, Inc. | Through focus retinal image capturing |
EP3261518A4 (fr) * | 2015-02-27 | 2018-11-21 | Welch Allyn, Inc. | Capture d'image rétinienne à foyer traversant |
US9836828B2 (en) | 2015-04-22 | 2017-12-05 | Esight Corp. | Methods and devices for optical aberration correction |
WO2016168913A1 (fr) * | 2015-04-22 | 2016-10-27 | Esight Corp. | Procédés et dispositifs de correction d'aberration optique |
US10758119B2 (en) | 2015-07-24 | 2020-09-01 | Welch Allyn, Inc. | Automatic fundus image capture system |
US10772495B2 (en) | 2015-11-02 | 2020-09-15 | Welch Allyn, Inc. | Retinal image capturing |
US11819272B2 (en) | 2015-11-02 | 2023-11-21 | Welch Allyn, Inc. | Retinal image capturing |
US10524653B2 (en) | 2015-11-02 | 2020-01-07 | Welch Allyn, Inc. | Retinal image capturing |
US10147235B2 (en) | 2015-12-10 | 2018-12-04 | Microsoft Technology Licensing, Llc | AR display with adjustable stereo overlap zone |
US10413179B2 (en) | 2016-01-07 | 2019-09-17 | Welch Allyn, Inc. | Infrared fundus imaging system |
JP2017156547A (ja) * | 2016-03-02 | 2017-09-07 | パイオニア株式会社 | 画像表示装置、制御方法、プログラム及び記憶媒体 |
JP2019512750A (ja) * | 2016-04-01 | 2019-05-16 | リニア アルジェブラ テクノロジーズ リミテッド | 人間の視覚的なメカニズムに適合したヘッドマウント型ディスプレイのためのシステム及び方法 |
WO2017196270A1 (fr) * | 2016-05-13 | 2017-11-16 | Cy Vision Inc. | Micro-affichage à fibre de balayage |
GB2557569A (en) * | 2016-09-09 | 2018-06-27 | Displaylink Uk Ltd | Adjusting display data |
GB2557569B (en) * | 2016-09-09 | 2022-07-06 | Displaylink Uk Ltd | Adjusting display data |
US10602926B2 (en) | 2016-09-29 | 2020-03-31 | Welch Allyn, Inc. | Through focus retinal image capturing |
JP2020513580A (ja) * | 2016-10-26 | 2020-05-14 | バルブ コーポレーション | 瞳孔位置を使用した光学レンズの歪み補正 |
JP7006962B2 (ja) | 2016-10-26 | 2022-01-24 | バルブ コーポレーション | 瞳孔位置を使用した光学レンズの歪み補正 |
US10855969B2 (en) | 2016-11-16 | 2020-12-01 | Tencent Technology (Shenzhen) Company Limited | File processing method, terminal, server, and computer storage medium |
CN108111874B (zh) * | 2016-11-16 | 2020-01-31 | 腾讯科技(深圳)有限公司 | 一种文件处理方法、终端及服务器 |
CN108111874A (zh) * | 2016-11-16 | 2018-06-01 | 腾讯科技(深圳)有限公司 | 一种文件处理方法、终端及服务器 |
US10122990B2 (en) | 2016-12-01 | 2018-11-06 | Varjo Technologies Oy | Imaging system and method of producing context and focus images |
WO2018100238A3 (fr) * | 2016-12-01 | 2018-07-12 | Varjo Technologies Oy | Système d'imagerie et procédé de production d'images de contexte et de mise au point |
IL274752B1 (en) * | 2017-11-30 | 2024-07-01 | Tec Med S R L Tecnologie Mediche | Comprehensive display system for eye treatments |
WO2019106472A1 (fr) * | 2017-11-30 | 2019-06-06 | Tec Med S.R.L. Tecnologie Mediche | Système d'affichage immersif pour thérapies oculaires |
US11612516B2 (en) | 2017-11-30 | 2023-03-28 | Tec Med S.R.L. Tecnologie Mediche | Immersive display system for eye therapies |
US11682106B2 (en) | 2017-12-29 | 2023-06-20 | Intel Corporation | Foveated image rendering for head-mounted display devices |
US10949947B2 (en) | 2017-12-29 | 2021-03-16 | Intel Corporation | Foveated image rendering for head-mounted display devices |
US11779209B2 (en) | 2018-05-24 | 2023-10-10 | Welch Allyn, Inc. | Retinal image capturing |
US11096574B2 (en) | 2018-05-24 | 2021-08-24 | Welch Allyn, Inc. | Retinal image capturing |
Also Published As
Publication number | Publication date |
---|---|
EP2786196A1 (fr) | 2014-10-08 |
CN104094162A (zh) | 2014-10-08 |
CA2857691A1 (fr) | 2013-06-06 |
EP2786196A4 (fr) | 2015-11-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9077973B2 (en) | Wide field-of-view stereo vision platform with dynamic control of immersive or heads-up display operation | |
WO2013082387A1 (fr) | Plateforme de vision stéréoscopique tridimensionnelle (3d) à large champ de vision ayant une commande dynamique d'opération d'affichage immersif ou d'affichage tête haute | |
US10983354B2 (en) | Focus adjusting multiplanar head mounted display | |
JP7329105B2 (ja) | ディスプレイシステムのための深度ベース中心窩化レンダリング | |
JP7273940B2 (ja) | 深度平面間の低減された切り替えを伴う多深度平面ディスプレイシステム | |
US6529331B2 (en) | Head mounted display with full field of view and high resolution | |
TWI710797B (zh) | 眼睛投影系統 | |
Rolland et al. | Head-mounted display systems | |
US20120013988A1 (en) | Head mounted display having a panoramic field of view | |
US20190212565A1 (en) | Wearable display for stereoscopic viewing | |
US20060250322A1 (en) | Dynamic vergence and focus control for head-mounted displays | |
US11360313B1 (en) | System and method for automatic vision correction in near-to-eye displays | |
US9983412B1 (en) | Wide field of view augmented reality see through head mountable display with distance accommodation | |
JP2022535646A (ja) | 調整可能な瞳孔間距離機構を有する両眼タイプのヘッドマウントディスプレイシステム | |
JP7275124B2 (ja) | 映像投射システム、映像投射装置、映像表示光回折用光学素子、器具、及び映像投射方法 | |
JP2023528262A (ja) | 相関内部結合および外部結合光学領域を伴う拡張および仮想現実ディスプレイシステム | |
Hua et al. | Near-eye displays: state-of-the-art and emerging technologies | |
JP6832318B2 (ja) | 目用投影システム | |
US20240036318A1 (en) | System to superimpose information over a users field of view | |
US20240220012A1 (en) | Optical see through (ost) head mounted display (hmd) system and method for precise alignment of virtual objects with outwardly viewed objects | |
CN109963141B (zh) | 视觉显示***及方法,以及头戴显示装置 | |
Shevlin | A fixed-viewpoint volumetric stereoscopic 3d display using adaptive optics | |
Dunn | Deformable Beamsplitters: Enhancing Perception with Wide Field of View, Varifocal Augmented Reality Displays |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 12854278 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2857691 Country of ref document: CA |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
REEP | Request for entry into the european phase |
Ref document number: 2012854278 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2012854278 Country of ref document: EP |