WO2020229599A1 - Anzeigevorrichtung mit reduziertem energieverbrauch - Google Patents
Anzeigevorrichtung mit reduziertem energieverbrauch Download PDFInfo
- Publication number
- WO2020229599A1 WO2020229599A1 PCT/EP2020/063462 EP2020063462W WO2020229599A1 WO 2020229599 A1 WO2020229599 A1 WO 2020229599A1 EP 2020063462 W EP2020063462 W EP 2020063462W WO 2020229599 A1 WO2020229599 A1 WO 2020229599A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- light modulator
- spatial light
- field
- view
- image
- Prior art date
Links
- 230000003287 optical effect Effects 0.000 claims abstract description 66
- 230000004913 activation Effects 0.000 claims abstract description 10
- 238000005286 illumination Methods 0.000 claims abstract description 8
- 230000033001 locomotion Effects 0.000 claims description 78
- 238000000034 method Methods 0.000 claims description 34
- 230000008878 coupling Effects 0.000 claims description 31
- 238000010168 coupling process Methods 0.000 claims description 31
- 238000005859 coupling reaction Methods 0.000 claims description 31
- 238000003384 imaging method Methods 0.000 claims description 31
- 230000006870 function Effects 0.000 claims description 30
- 230000004075 alteration Effects 0.000 claims description 21
- 230000001360 synchronised effect Effects 0.000 claims description 14
- 238000001514 detection method Methods 0.000 claims description 11
- 230000003190 augmentative effect Effects 0.000 claims description 7
- 230000001902 propagating effect Effects 0.000 claims description 6
- 230000001419 dependent effect Effects 0.000 claims description 3
- 230000000644 propagated effect Effects 0.000 claims description 2
- 230000008569 process Effects 0.000 claims 1
- 238000001228 spectrum Methods 0.000 description 25
- 238000012937 correction Methods 0.000 description 15
- 239000011521 glass Substances 0.000 description 11
- 238000005265 energy consumption Methods 0.000 description 9
- 235000009508 confectionery Nutrition 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 239000004973 liquid crystal related substance Substances 0.000 description 5
- 230000010287 polarization Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 230000003068 static effect Effects 0.000 description 4
- 101710171187 30S ribosomal protein S10 Proteins 0.000 description 3
- 101000812304 Bacillus subtilis (strain 168) 30S ribosomal protein S16 Proteins 0.000 description 3
- 101000675258 Geobacillus stearothermophilus 30S ribosomal protein S19 Proteins 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 230000001427 coherent effect Effects 0.000 description 3
- 210000003128 head Anatomy 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 210000001525 retina Anatomy 0.000 description 2
- 101710171219 30S ribosomal protein S13 Proteins 0.000 description 1
- 101000811743 Mesostigma viride Uncharacterized protein ycf20 Proteins 0.000 description 1
- 230000003213 activating effect Effects 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 239000013065 commercial product Substances 0.000 description 1
- 230000001627 detrimental effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000001459 lithography Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 230000035484 reaction time Effects 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3141—Constructional details thereof
- H04N9/315—Modulator illumination systems
- H04N9/3152—Modulator illumination systems for shaping the light beam
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B26/00—Optical devices or arrangements for the control of light using movable or deformable optical elements
- G02B26/08—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
- G02B26/10—Scanning systems
- G02B26/101—Scanning systems with both horizontal and vertical deflecting means, e.g. raster or XY scanners
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B26/00—Optical devices or arrangements for the control of light using movable or deformable optical elements
- G02B26/08—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
- G02B26/10—Scanning systems
- G02B26/105—Scanning systems with one or more pivoting mirrors or galvano-mirrors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3141—Constructional details thereof
- H04N9/315—Modulator illumination systems
- H04N9/3155—Modulator illumination systems for controlling the light source
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3141—Constructional details thereof
- H04N9/317—Convergence or focusing systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3141—Constructional details thereof
- H04N9/3173—Constructional details thereof wherein the projection device is specially adapted for enhanced portability
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B2027/0178—Eyeglass type
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
Definitions
- the invention relates to a display device for displaying two-dimensional and / or three-dimensional information, such as objects or scenes, which provides a low energy consumption.
- the invention preferably relates to augmented reality (AR) display devices or displays. This includes, for example, head-mounted displays and head-up displays. It should be noted, however, that the invention is not intended to be restricted to such displays.
- the invention also relates to a method for displaying two-dimensional and / or three-dimensional information with which a reduced energy consumption is to be brought about in a display device.
- AR augmented reality
- spatial light modulators In display devices for displaying two-dimensional and / or three-dimensional scenes, so-called spatial light modulators are often used, with which the incident light can be modulated in accordance with the information required and to be displayed.
- Various types of spatial light modulators which are also referred to as SLM, are known here.
- a spatial light modulator commonly used for this purpose is, for example, an LCoS-SLM (Liquid Crystal on Silicon), which reflects the incident light instead of transmitting it.
- LCoS-SLM are usually very small in size, e.g. with diagonals of less than 20 mm, but can have a very large number of pixels, e.g. 4000 x 2000 pixels.
- LCoS-SLM exist as a commercial product for modulating amplitude as well as modulating phase of light.
- the advantage of an LCoS-SLM is clearly its reflectivity.
- An LCoS-SLM can have a frame rate of e.g. 60 Hz, in some cases a few hundred Hertz, e.g. 180 Hz or 240 Hz. However, a frame rate higher than 500 Hz cannot be achieved.
- Another type of SLM is a MEMS-SLM (Micro Electro-Mechanical Mirror Systems) whose main advantage over an LCoS-SLM is its higher speed.
- Another advantage lies in the modulation values, phase values or amplitude values, which can be set in a more stable manner and are less error-prone than those of an LCoS-SLM.
- the modulation values of neighboring pixels can be set better independently of one another, while with an LCoS-SLM the modulation values of neighboring pixels can undesirably influence one another.
- amplitude-modulating MEMS-SLMs are currently limited to binary types, phase-modulating MEMS-SLMs not currently being found on the display market, but rather being used, for example, in the field of lithography.
- MEMS SLMs have a relatively small number of pixels, for example fewer than VGA (640 x 480 pixels).
- the effort and cost of manufacturing a new spatial light modulator increases with the number of pixels.
- the energy consumption or power consumption in a display device plays an important role.
- the power consumption of the spatial light modulator must also be considered.
- a significant part of the power consumption of a spatial light modulator is due to the data transmission to the pixels of the spatial light modulator.
- the energy consumption also depends on the length of the data lines from the edge of the spatial light modulator to the individual pixels.
- a spatial light modulator that is small in size and with a lower number of pixels on average has shorter data lines and therefore usually less energy consumption. Therefore, such a spatial light modulator can be more energy efficient even when operated at high frame rates, i.e. if the same number of pixels is described per unit of time compared to a spatial light modulator with a high number of pixels but a lower frame rate.
- a mobile display device e.g. a head-mounted display that is attached to a head of a viewer or user and cannot be connected to a power grid via a cable
- energy consumption is a particularly important factor.
- a display device that enables the use of a spatial light modulator with a small number of pixels would be considered advantageous here.
- a large field of view (FoV) with a good resolution for a head-mounted display (HMD) a large number of pixels are required.
- a typical value is 60 pixels / degree field of view for the representation of a flat two-dimensional (2D) image, since this value (60 pixels / degree field of view) corresponds to the resolution of the human eye.
- a holographic representation of a three-dimensional (3D) scene however, a larger number of pixels per degree of field of view is required.
- a field of view of, for example, 60 degrees x 30 degrees then requires 3600 pixels x 1800 pixels to generate a flat two-dimensional image, but many more pixels, e.g. 15,000 pixels x 7500 pixels, are required to generate a holographic image.
- LCoS-SLMs with a resolution of approx. 4000 pixels x 2000 pixels already exist as commercial products. However, these still have significant disadvantages. A higher resolution is often achieved by using smaller pixels, for example 3 to 5 micrometer pixels, which in an LCoS-SLM increases the susceptibility of the modulation values to errors, for example the undesired influence of the modulation values by neighboring pixels.
- display devices in the form of head-mounted displays are known, each of which has a light guide, a coupling device and a coupling device as well as additional optical elements which are arranged in the light path before the light is coupled into the light guide.
- Head-mounted displays that do not have a light guide, but an arrangement of focusing means, such as lens elements and / or curved mirror elements, are known, for example, from US 2010/0097671 or US 2013/0222384.
- US 2013/0222384 describes a segmented multiple imaging of a spatial light modulator.
- a large field of view is generated in that different segments are generated sequentially by imaging the spatial light modulator, with an imaging of the spatial light modulator being carried out at a different position in the field of view.
- an arrangement of two mirrors that are rotated in the same direction is used to generate the segments.
- AR Augmented Reality
- AR Augmented Reality
- Such AR displays are also referred to as mixed reality displays and enable a person to see through a transparent or semitransparent system in order to view his or her real environment and also images of virtual objects, such as. Text, graphics, videos, etc. that are created to appear as part of the real environment by overlaying it.
- the insertion or display or overlay of additional information with a person's natural perception or environment is therefore referred to as augmented reality (AR).
- This additional information such as, more precisely specified than, speed displays, temperature displays, signs, warnings, or assistance functions, navigation system functions, radio functions or shopping displays, are displayed in a person's field of vision without the person being in their behavior or operating behavior is adversely affected.
- an AR display it is therefore important that a user can continue to observe his real environment in addition to the virtually created objects. Therefore, only a small part of the real environment may be covered by the content of the objects, which is shown to the user through the AR display.
- an AR display requires a higher brightness compared to applications of a display device as a television or VR-HMD, since the user should see the information or virtual objects in an environment with bright sunlight at approximately the same brightness level as in the real environment .
- an AR display for example an AR head-mounted display (AR-HMD), as a mobile device, there is also the need for low energy consumption.
- AR-HMD AR head-mounted display
- the object of the present invention to create a device and a method which allow a three-dimensional representation of information. Furthermore, the device should be compact and light in weight and also be energy efficient.
- the object is achieved by a device having the features of claim 1.
- a display device which is designed in particular as an AR (augmented reality display device or AR display).
- the AR display is preferably designed as an AR head-mounted display or as an AR head-up display has a lighting device, a spatial light modulator device, an optical system and a control device.
- the lighting device is provided for emitting light, for example essentially coherent light.
- the spatial light modulator device is provided for modulating light emitted by the lighting device and can at least
- the optical system in turn is provided as a segment for generating at least one image of the spatial light modulator device and, in addition to at least one imaging element, has a deflection device for directing the image of the spatial light modulator device to a defined one Position in the field of view of a user.
- the control device is coupled to the lighting device and the deflection device and is designed to control or switch the lighting device as a function of a control of the deflection device.
- An AR display is a display device in which virtual information or objects are superimposed on the real environment of a person or user using the AR display, so that the user can receive additional information that is useful to him when viewing his real environment could be.
- the user of the display device or device according to the invention could be shown information relating to a point of interest or navigation instructions during a city tour, which are then superimposed on the real environment in the user's field of vision.
- the optical system of the display device has a deflection device with which an image of the spatial light modulator device generated by the optical system is directed to a position or location defined in the user's field of vision is to superimpose the required information or the required object with the real environment and display it to the user.
- the control device of the display device according to the invention is coupled to the lighting device and the deflection device in order to be able to control the lighting device as well as the deflection device accordingly.
- the lighting device is activated or switched by the control device as a function of the activation of the deflection device.
- the deflection device of the optical system is activated and sets the position in the field of view at which the virtual information or the virtual object is to be displayed or represented. If the required position in the field of view is reached, the lighting device is activated by the control device so that the lighting device emits light that strikes the spatial light modulator device and an image of this is generated by the optical system.
- This image of the spatial light modulator device is thus directed as a segment to the defined position in the user's field of vision and is superimposed with the real environment in the field of vision, so that the user can view the information represented thereby.
- the display device according to the invention provides the user with the required information or information in an energy-efficient manner, since data for generating the virtual information to be displayed are only sent or transmitted to the spatial light modulator device or generated by the spatial light modulator device itself if the control device does Lighting device controls accordingly to emit light.
- a calculation of the virtual information to be displayed such as a Calculation and totaling of sub-holograms for an overall hologram to display a holographic three-dimensional information or object or a type of image processing such as blurring of objects that are outside the focus of a user for the display of a stereoscopic scene, and their display only for those areas in the Field of view takes place in which virtual information is also to be represented or displayed by means of the display device according to the invention.
- no data is calculated or, in another embodiment, no data is calculated and transmitted to the spatial light modulator device. In this way, the energy consumption for data transmission alone can be reduced considerably.
- the display device can thus advantageously be used as an augmented reality display for combining real surroundings and virtual information that is displayed or superimposed, such as e.g. two-dimensional and / or three-dimensional objects.
- the generated at least one image of the spatial light modulator device as a segment that has virtual information takes up only a fraction, for example 2% to 30% or even only 5% to 20%, of the field of view. This means that the field of view is only filled with little virtual information.
- the displayed image of the spatial light modulator device as a segment or the displayed images of the spatial light modulator device as segments do not completely fill the field of view or only form part of the field of view, so that between displayed virtual information, each complete virtual information form, there are gaps or areas in the field of view that are filled with the real information or in which a user can view his real environment.
- An image of the spatial light modulator device as segments or several images of the spatial light modulator device together as segments can thus form virtual information, which in turn is separated from other displayed virtual information in the field of vision by a gap in which the viewer perceives the real environment.
- the virtual information to be displayed can be generated holographically or stereoscopically. Furthermore, the virtual information can be displayed as a two-dimensional or three-dimensional representation. Combinations of two-dimensional and three-dimensional representations are also possible. According to the invention, the term “virtual information” should be understood to mean not only the completely generated virtual information, such as an object or a scene, but also only part of the virtual information, such as a part of an object or a part of a scene . Further advantageous refinements and developments of the invention emerge from the further subclaims.
- the optical system is provided for generating at least two images of the spatial light modulator device and, corresponding to the number of images of the spatial light modulator device, for generating virtual visibility areas, the at least two images of the spatial light modulator device as Segments are present in the field of view.
- the at least two images of the spatial light modulator device can advantageously be combined with one another as segments in the field of view and / or partially overlap one another or be spaced apart from one another via a gap.
- the images of the spatial light modulator device are preferably generated sequentially in the field of view.
- a segmented display of the virtual information is created in the field of view.
- a large field of view or a large viewing angle can be created by combining several images of the spatial light modulator device as segments. For example, a certain number of segments can be used, such as more than 10 segments, more than 30 segments or even more than 50 segments, which enables virtual information to be displayed in a user's field of vision.
- the number of images of the spatial light modulator device as segments can advantageously be varied in each frame between a minimum value, e.g. an image as a segment, and a maximum value, e.g. 10 to 50 images as segments, can be set differently and the position of the images of the spatial light modulator device as segments in the field of view can be set differently in each frame.
- the determination of the number and the position of the images of the spatial light modulator device as segments in the field of view is dependent on the real environment of a user. This means that the number and position of the images of the spatial light modulator device can be adjusted as a function of the real environment.
- the number and the position of the displayed images of the spatial light modulator device as segments is thus variable and can be adjusted depending on the virtual information required in the field of view. For this purpose, the user or viewer observes the displayed two-dimensional and / or three-dimensional information or the displayed two-dimensional and / or three-dimensional object through a virtual visibility area in a viewer plane.
- this situation is to be interpreted in such a way that when the virtual information is generated and displayed holographically in the coding direction of a hologram encoded on the spatial light modulator device, a virtual viewer window is present as a virtual visibility area and, when the virtual information is displayed stereoscopically in the field of view, an optimal viewing area, also known as the “sweet spot”, is available as a virtual visibility area.
- the virtual viewer window and the sweet spot thus each or together, depending on the way in which the virtual information is presented, form a virtual visibility area in a viewer plane in which a user, in particular one of the user's eyes, is to observe the information generated is located.
- one or more segments of three-dimensional information to be displayed which should lie in the viewing direction of the eye of a user and thus fall on the retina in the center of the fovea of the eye, could be generated and displayed holographically.
- one or more segments of the same or further three-dimensional information to be displayed which should not lie in the viewing direction of the user's eye and thus fall on the retina of the eye, but not in the center of the fovea, could be generated and displayed stereoscopically.
- the individual segments for displaying the virtual information in the field of view should be generated purely holographically, as this provides a more realistic depth display of the reconstructed information or object compared to a stereoscopic display the virtual information can be achieved.
- the at least one image of the spatial light modulator device is an image of the entire spatial light modulator device or an image of only a partial area of the spatial light modulator device.
- the image of the spatial light modulator device generally forms a segment which is superimposed on the real environment or the real field of view of the user and is therefore a segment of the field of view of the user.
- the segment in the field of view generated by the display device according to the invention which has the virtual information, can be created by mapping the entire spatial light modulator device, ie the total area of the spatial light modulator device, so that all pixels of the spatial light modulator device contribute to the generation of the segment.
- the generated segment can also be created by mapping only a part or partial area of the spatial light modulator device, ie not all pixels of the spatial light modulator device contribute to the generation of the segment.
- a single generated segment thus covers or covers only a small area of the field of view of the user of the device according to the invention.
- an individual segment can only cover an area of approx. 3 ° x 3 ° or approx. 5 ° x 3 ° or approx. 7 ° x 7 ° of the entire field of view, the invention not being restricted to these figures.
- the entire field of view can, for example, span an area of approx. 40 ° x 20 ° or approx. 60 ° x 30 ° or 60 ° x 60 °, whereby these figures should not have a restrictive effect here either.
- the number of segments that have the virtual information and are superimposed on the real environment is smaller than the number of segments that would be required if the entire field of view were built up by segments or around the entire field of view by means of Generate segments. For example, if the total field of view encompasses an area of approximately 60 ° x 60 ° and the size of a single segment is approximately 5 ° x 5 °, then theoretically 12 x 12 segments, i.e. 144 segments are required in order to be able to generate the entire field of view. However, if only about 15% of the field of view with virtual information, i. with e.g. two-dimensional and / or three-dimensional objects, then only about 25 to 30 segments, depending on the size of the segments, could be selected and sufficient to display the required information (s) in the field of view. Thus, greater time savings and greater energy efficiency can be achieved.
- the deflection device has at least one scanning mirror element which is movably mounted, or at least one grating element.
- the device according to the invention preferably has at least one scanning mirror element for deflecting and directing the light. Due to its movable mounting, the at least one scanning mirror element can move or rotate and the image generated by the spatial light modulator device can be moved as a segment to a defined position in the field of view of the user. In this way, several images of the spatial light modulator device can be generated and directed to defined positions in the field of view.
- a commercially available scanning mirror can be used as the scanning mirror element.
- the deflection device can have at least one grating element, for example a switchable grating element or a polarization-selective grating element, such as a polarization grating in combination with a polarization switch.
- the deflection device can, for example, have a stack of grating elements with different grating periods, so that different angles of deflection can be set by different combinations of N gratings 2 to the power N (2 N ).
- the deflection device can also have a combination of at least one scanning mirror element and at least one grating element, for example a volume grating.
- the at least one grating element or volume grating has an angle selectivity. If the scanning mirror element is set in such a way that light strikes it within the angular selectivity of the grating element or volume grating, the light is further deflected by the grating element or volume grating. The deflection angle of the scanning mirror element is thus increased by the grating element or volume grating.
- the scanning mirror element is set in such a way that light strikes it outside the angular selectivity of a grating element or volume grating, it is not deflected by this grating element or volume grating.
- the scanning mirror element can be used in order to select one of several grating elements or volume grids, whose angle selectivity and deflection angle are set differently, which then deflects the light further.
- the optical system has at least one combiner.
- the at least one combiner thus combines the information from the real environment of the user and the information generated with the device according to the invention in the user's field of vision, so that the user's eye can see and observe both information together in the field of vision.
- the at least one combiner can be a partially reflective mirror element that at least partially reflects light emanating in the beam path from the light modulator device in the direction of a user's eye and at least partially transmits ambient light.
- the at least one combiner can be, for example, a windshield of a means of transport, such as a vehicle.
- the at least one combiner can also be a light guide from which light emanating from the light modulator device is decoupled in the beam path in the direction of a user's eye and at least partially allows ambient light to pass through.
- the deflection device can be between the spatial and the spatial
- Light modulator device and the combiner or between the lighting device and the spatial light modulator device are configured to be any suitable light modulator device.
- the scanning mirror element can be arranged in a Fourier plane of the spatial light modulator device. This Fourier plane is then further mapped into the observer plane, in which an eye of the user is located, by means of the optical system, in which the virtual visibility region, i.e. a virtual viewer window or a sweet spot is generated through which the user must look in order to be able to view the virtual information displayed in the field of view.
- the virtual visibility region i.e. a virtual viewer window or a sweet spot is generated through which the user must look in order to be able to view the virtual information displayed in the field of view.
- the deflection device has two scanning mirror elements which can be rotated synchronized with one another.
- These scanning mirror elements can be rotated or moved synchronized with one another.
- This synchronized movement of the scanning mirror elements to one another can also move the image of the spatial light modulator device as a segment or the image plane of the spatial light modulator device to a defined position in the field of view without changing the position of the virtual visibility area in the observer plane.
- the arrangement of the two scanning mirror elements in the device according to the invention can then be such that, for example, one scanning mirror element is arranged in the light direction in front of the Fourier plane of the spatial light modulator device and the further scanning mirror element in the light direction after the Fourier plane.
- the deflection device has at least two grid elements, both of which are switched in a synchronized manner with one another.
- the synchronized switching of both grating elements can also move the image of the spatial light modulator device as a segment or the image plane of the spatial light modulator device to a defined position in the field of view without changing the position of the virtual visibility area in the observer plane.
- a grating element can be arranged in the light direction in front of the Fourier plane of the spatial light modulator device and the further grating element in the light direction after the Fourier plane.
- the at least one combiner can preferably have at least one focusing element or at least one focusing function.
- the at least one combiner can have at least one focusing element in order to direct or set the virtual information to be displayed in the field of view in the depth range to the required depth.
- the focusing element is preferably designed in such a way that it does not impair or influence the perception of the real environment in the field of view.
- the focusing element could be designed as a grating element with a limited acceptance angle, preferably as a volume grating with a limited acceptance angle.
- the acceptance angle is adapted to the angle of incidence of the light carrying the information, but not to the range of the angle of incidence of light in which the light from the real environment strikes the grating element. As a result, the light that strikes the grid element from the real environment is not influenced by it and passes through it unhindered.
- the combiner can, for example, be designed as a partially reflecting mirror or as a light guide, on the surface of which a focusing element, for example a grating element, is provided or applied.
- the at least one combiner itself can have a focusing function in that it is curved or at least partially curved.
- the curved design of the at least one combiner enables the light or the image of the spatial light modulator device to be focused as a segment at a z-position defined therewith (along the z-direction or along the optical axis of the optical system).
- the at least one combiner can be designed to be at least partially curved. If the at least one combiner is designed, for example, as a partially reflecting mirror element, in one embodiment of the invention the mirror surface can be curved or arched, for example in the form of a concave mirror, and thus have a focusing function. Also a combination a curved surface with an additional grid element on this surface of the at least one combiner is possible, for example.
- the at least one combiner could thus be designed as a type of spectacle lens or as a windshield. It can be designed flat or planar and have a focusing element.
- the at least one combiner could, however, also be designed to be at least partially curved and thus itself act as a focusing element or additionally be combined with focusing elements.
- a continuous movement of the at least one scanning mirror element or a step-by-step movement of the at least one scanning mirror element with a firmly defined step width is provided in the deflection device.
- the at least one scanning mirror element can thus be moved continuously or in steps in order to direct the image of the spatial light modulator device as a segment to a required position in the field of view.
- a step-by-step movement of the at least one scanning mirror element can take place, for example, in such a way that the scanning mirror element is moved by a defined angle and then stopped in order to control the control device of the device according to the invention at this defined fixed scanning mirror element position in such a way that the lighting device emits light, so that an image of the spatial light modulator device is generated as a segment and this segment is directed to the position in the field of view approached with the scanning mirror element.
- control device controls the lighting device only when the scanning mirror element is stopped, so that the light emitted by it is modulated with the required information by means of the spatial light modulator device and an image of the spatial light modulator device is generated by the optical system, which is then sent to the is directed by the scanning mirror element predetermined position in the field of view.
- the scanning mirror element is moved by a further defined angle by means of the control device, the movement of the scanning mirror element then being stopped again so that another image of the spatial light modulator device is generated and can be directed to another defined position in the field of view.
- Such a start-stop movement of the scanning mirror element is possible at a high speed.
- the lighting device of the device according to the invention should have at least one light source that can be operated in a pulsed manner.
- the lighting device or the at least one light source is then located only as long as it is switched on as long as the scanning mirror element is in the stop state.
- the scanning mirror element is in motion, the lighting device or the light source is in an OFF state.
- the at least one scanning mirror element can also provide a continuous movement.
- a continuous movement of the at least one scanning mirror element would, however, also cause a continuous displacement of the generated image of the spatial light modulator device. However, this is undesirable.
- the at least one scanning mirror element is combined with a compensating mirror element that provides a synchronized movement for the movement of the at least one scanning mirror element assumes that when both mirror elements move in the same direction, an image of the spatial light modulator device can be generated at a fixed, constant position and when both mirror elements move in opposite directions, an image of the spatial light modulator device can be shifted in the field of view.
- the image of the spatial light modulator device can be held at the same required position during its generation and thus directed to a defined position in the field of view.
- the compensation control element can also be activated by the control device. Both movements, ie the movement of the at least one scanning mirror element and the movement of the compensation mirror element synchronized therewith, are thus combined with one another in order to direct an image of the spatial light modulator device as a segment to a required position in the field of view.
- the at least one scanning mirror element could be combined with the compensation mirror element in such a way that, during the continuous movement of the scanning mirror element, the latter reaches a position at which an image of the spatial light modulator device is to be generated, but this image of the spatial light modulator device is not due to the continuous movement of the scanning mirror element can still be directed to the defined position as required.
- the compensation mirror element can be controlled in such a way that it executes a movement synchronized with the movement of the scanning mirror element. As a result, the generated image of the spatial light modulator device is shifted or moved as a segment in the opposite direction to the direction of movement of the scanning mirror element, so that it is shifted to the required defined position in the field of view by this compensation movement of the compensation mirror element and can be judged.
- a subsequently generated image of the spatial light modulator device is directed in the same way to its defined position in the field of view.
- the synchronized movement of the compensation mirror element is only maintained by activating the control device as long as the lighting device or the at least one light source is in an ON state.
- a movement of the scanning mirror element and the compensation mirror element in the same direction can be provided as long as the lighting device is in an ON state.
- the compensation mirror element can move into its initial state when the lighting device or the at least one light source is in an OFF state.
- a continuous movement of the at least one scanning mirror element at different predefined speeds or a step-by-step movement of the at least one scanning mirror element with different adjustable step widths is provided for generating at least two images of the spatial light modulator device as segments in the field of view within a frame.
- the segments can be directed to the required defined position in the field of view by means of the at least one scanning mirror element at different speeds with a continuous movement or different step widths with a step-by-step movement.
- the speed or the step width of the movement of the at least one scanning mirror element depends on the required position of the image to be generated by the spatial light modulator device as a segment in the user's field of view.
- the at least one scanning mirror element can be operated at a lower speed between the two or more representations of virtual information.
- the speed or the step width of the movement of the at least one scanning mirror element is adapted to the defined position of the respective image of the spatial light modulator device as a segment in the field of view.
- the size and shape of the segments generated could vary with position in the field of view.
- the speed or the step width of the movement of the at least one scanning mirror element could likewise be varied accordingly.
- the size and / or shape of the at least one image of the spatial light modulator device can be varied as a segment in successive frames or the size and / or shape of the at least two images of the spatial light modulator device as segments with the defined position in the field of view within a frame or in successive frames can be varied.
- a change in the size and / or shape of the at least one image of the spatial light modulator device as a segment in successive frames is particularly useful if the size and / or shape of the displayed scene or objects also changes in successive frames and is represented with a fixed number of segments shall be.
- the size of an object changes so that it can be displayed in one frame with a single segment, but is slightly larger than the segment in the next frame, it can be more advantageous to use the size and / or shape of the at least one image to adapt the spatial light modulator device as a segment than to represent this object by two segments of fixed size.
- a change in the size or shape of the at least one image of the spatial light modulator device as a segment with the defined position in the field of view within a frame can serve the same purpose, but can also be used if a higher resolution is required in certain areas of the field of view than in others Areas. For example, in the central area of the field of view, small segments with fine resolution and Edge area of the field of view large segments are generated with coarser resolution in their extension.
- a change in the size or shape of the at least one image of the spatial light modulator device as a segment with the defined position in the field of view within a frame can also serve, for example, to simplify the optical system. This accepts that when the at least one image of the spatial light modulator device is generated as a segment in a simple optical system, there may be a change in magnification with the position in the field of view or changes in the optical distortions which influence the shape of the image.
- the at least one combiner is designed as a partially reflective mirror element or as a light guide.
- the at least one combiner could be designed as a partially reflecting mirror element.
- This partially reflective mirror element could, for example, be a windshield in a means of locomotion or also a spectacle lens.
- the at least one combiner could also be designed as a light guide, the light propagating in the light guide due to total reflection.
- the light guide as a combiner is part of the optical system and also serves to generate the image of the spatial light modulator device as a segment.
- the light of the real environment of the user can penetrate unhindered through the light guide as a combiner, whereby the at least one image of the spatial light modulator device generated by the device according to the invention is superimposed as the segment carrying the virtual information with the real environment of the user in the field of view.
- the deflection device is designed as a switchable coupling element for coupling the light into the combiner designed as a light guide and / or as a decoupling element for decoupling the light from the combiner designed as a light guide.
- the deflection device for directing the image of the spatial light modulator device to a defined position in the field of view can be designed as a switchable coupling element and / or also as a switchable decoupling element.
- a switchable decoupling element by means of a switchable decoupling element, light can be decoupled from the light guide at different positions of the light guide and an image of the spatial light modulator device can be generated as a segment at different positions in the field of view.
- a light guide like the one in the WO 2018/146326 A1 is described, the disclosure content of which is intended to be included here in its entirety.
- the light propagates within the light guide via a reflection at the interfaces of the light guide, the coupling of the light from the light guide being provided after a predetermined number of reflections of the light at the interfaces of the light guide.
- the light outcoupling device can be designed to be controllable, the light outcoupling device being controllable in such a way that in one control state of the light outcoupling device light is decoupled after a predetermined number of reflections and in another control state of the light outcoupling device the light propagates further in the light guide.
- a display device is described in WO 2018/146326 A1, in particular a display device provided close to the eye, which has a lighting device that has at least one light source, at least one spatial light modulator device, an optical system and such a light guide device.
- a lighting device that has at least one light source, at least one spatial light modulator device, an optical system and such a light guide device.
- the decoupling of light coming from different pixels of the spatial light modulator device after entering the light guide device after an equal number of reflections at interfaces of the light guide for all pixels can be provided.
- the number of reflections of the light at the interfaces of the light guide for the production of one segment can differ from the number of reflections of the light at the interfaces of the light guide for the production of another segment.
- the number of reflections of the light at the boundary surfaces of the light guide can be the same, and the coupling point of the light into the light guide can be different for these segments.
- a light deflection device can be provided in front of the light guide device in the light direction.
- the light deflection device for shifting the light coupling point and / or the coupling-out element for coupling the light out of the light guide is only activated if an image of the spatial light modulator device is required as a segment at a position in the field of view associated with the coupling point.
- the deflection device can have the at least one scanning mirror element already mentioned.
- the optical system and the light guide are designed such that light beams emanating from the individual pixels of the spatial light modulator device impinge on the light guide at an average of different angles relative to the surface of the light guide and can be coupled in, whereby a coupling angle spectrum can be defined, the light beams propagating in the light guide being able to be coupled out of the light guide at different angles on average relative to a virtual visibility region, whereby a coupling angle spectrum can be defined.
- the light guide could be designed as in WO 2019/012028 A1, the disclosure content of which is intended to be included here in its entirety.
- a light guide is described which is designed in such a way that the coupling-out angle spectrum of the light is enlarged compared to the coupling-in angle spectrum of the light.
- the coupling-out angle spectrum and the coupling-in angle spectrum of the light can also be of the same size.
- the at least one scanning mirror element can then preferably be arranged as a combiner in the beam path between the spatial light modulator device and the light guide.
- the coupling angle of the light into the light guide can be changed, whereby the propagation angle of the light in the light guide also changes.
- the light guide as a combiner can have passive or active coupling-out elements. By means of these decoupling elements, the light can be decoupled from the light guide at different positions and / or at different decoupling angles and thus directed to defined positions in the field of view.
- a spatial light modulator device instead of a spatial light modulator device that has a relatively large number of pixels (e.g. greater than 1000 pixels in one dimension), a spatial light modulator device in conjunction with a light guide as a combiner and at least one scanning mirror element that has a relatively small number of pixels (e.g. smaller than 1000 pixels in one dimension).
- a spatial light modulator device with a large number of pixels would generate a specific coupling angle spectrum which would be coupled into the light guide, the field of view achieved with the device being proportional to the coupling angle spectrum.
- a spatial light modulator device with a small number of pixels, for example several hundred pixels in one direction, is preferably used.
- This spatial light modulator device generates a small coupling angle spectrum for each segment to be generated in the field of view.
- the at least one scanning mirror element is designed in such a way that the central angle of the light to be coupled into the light guide is different for each segment to be generated and containing virtual information.
- images of the spatial light modulator device as segments can be made with one and the same light guide at different positions in the field of view are generated in which virtual information is displayed to the user.
- each generated segment has a field of view which is proportional to the small coupling angle spectrum, the combination of several segments in turn generating a large angle spectrum viewed as a whole.
- the invention is not intended to be limited to any particular type of spatial light modulator device. Various types or combinations of more than one spatial light modulator device can also be used with the invention.
- the spatial light modulator device can preferably have an LCoS-SLM or a MEMS-SLM.
- An LCoS-SLM is a spatial light modulator device that has a relatively large number of pixels but a relatively low frame rate.
- a MEMS SLM on the other hand, only has a small number of pixels, but has a relatively high frame rate.
- the spatial light modulator device could be divided into virtual areas or partial areas.
- Each pixel of the spatial light modulator device or, in another embodiment, all pixels that contribute to the representation of virtual information are assigned to at least one virtual area or sub-area.
- the virtual areas or partial areas can also overlap on the spatial light modulator device. In the case of overlapping virtual areas or partial areas, a pixel of the spatial light modulator device can also be assigned to more than one virtual area or partial area.
- the spatial light modulator device could be an LCoS-SLM with a number of pixels of 4000 x 2000 pixels, and by dividing the LCoS-SLM into virtual areas or sub-areas, images of these areas of the LCoS-SLM could be created as segments by illuminating only these virtual areas or sub-areas with a size of 400 x 400 pixels.
- data is written into all pixels of the spatial light modulator device or just into those pixels that are intended to contribute to the representation of the virtual information.
- data can be written by scanning lines on the spatial light modulator device.
- the virtual areas or partial areas of the spatial light modulator device e.g. an LCoS, lit one after the other.
- the deflection device which has at least one scanning mirror element, can be arranged in the beam path between the lighting device and the spatial light modulator device and direct the light onto the respective virtual areas or sub-areas of the spatial light modulator device by means of the control device by means of a defined movement of the at least one scanning mirror element.
- the sequence for illuminating the virtual areas or partial areas of the spatial light modulator device can be adapted to the sequence of writing the data into the pixels of the spatial light modulator device.
- the areas or partial areas of the spatial light modulator device can have the same size or also a different size.
- the virtual regions or partial regions of the spatial light modulator device can also have a greater overlap.
- virtual areas or partial areas provided adjacently can have an overlap of only one pixel or even a few pixels.
- the lighting device is only in an ON state in the case of pixels or virtual areas that contribute to the display of virtual information in the field of view of a user. If pixels or virtual areas on the spatial light modulator device which do not contribute to the display of virtual information at this point in time are scanned, the lighting device is in an OFF state.
- a spatial light modulator device with only a relatively small number of pixels such as a MEMS SLM, could also be used.
- the number of pixels could be below typical resolutions for a display device or display, such as less than 640 x 480 pixels (VGA), e.g. 200 x 200 pixels or 300 x 200 pixels or also 400 x 400 pixels, the number of pixels of a spatial light modulator device for the device according to the invention not being limited to these disclosed numbers of pixels.
- the deflection device can preferably be arranged in the beam path between the spatial light modulator device and the at least one combiner.
- an image of the spatial light modulator device corresponds as a segment to an image of the entire area of the spatial light modulator device. Therefore, all segments generated are preferably of the same size.
- the images of the spatial light modulator device as segments are time-sequential in the user's field of vision shown or displayed. This is done in each case by writing information, e.g.
- the additional information in the field of view of the user of the device according to the invention is to be displayed stereoscopically.
- a display device according to the invention should be provided for generating an image for the left eye
- a display device according to the invention should be provided for generating an image for the right eye of the user.
- Both display devices according to the invention can, for example, be combined with one another in the manner of a type of glasses.
- a display device can also be provided for both eyes in such a way that, for example, a lighting device and a spatial light modulator device are used, an optical system being provided with which light is multiplexed in time or space to direct light to the left eye and light to the right eye of a user becomes.
- a separate combiner can be assigned to each eye of the user, for example a separate light guide for the left eye and a separate light guide for the right eye, with light from the spatial light modulator device being coupled into one or the other light guide sequentially with a switching element .
- a common combiner can also be provided for both eyes of a user, for example the combiner would then be the windshield of a means of transport or a vehicle.
- the optical system has a variable focus system with which the distance of the at least one image of the spatial light modulator device can be set as a segment in the field of view to the user.
- the device according to the invention can have a varifocal configuration.
- This means that the virtual information generated can be adjusted in depth in the field of view along the optical axis of the optical system of the device.
- the distance between the virtual information, ie the generated image of the spatial light modulator device as a segment, to the observer plane in which the user is with his eye is to be understood.
- the distance between the segment having the virtual information can be adjusted by means of a variable focus system of the optical system according to a required depth position in the field of view.
- the variable focus system can preferably have at least one grating element with a controllable grating period or a combination of active and passive imaging elements.
- the grid element can for example be a liquid crystal grid element.
- variable focus system can also have combinations of tunable grating elements and passive imaging elements, such as lens elements.
- the variable focus system can also have a switchable grating element such as, for example, a switchable polarization grating or a passive grating in combination with a polarization switch.
- variable focus system is preferably arranged in the vicinity of a Fourier plane or in a Fourier plane of the spatial light modulator device, other arrangements also being possible.
- the Fourier plane of the spatial light modulator device is formed in the beam path, for example, between the spatial light modulator device and the at least one combiner.
- variable focus system has at least one grating element with a controllable grating period
- the invention can provide that the at least one grating element with a controllable grating period has prism functions and / or phase functions for correcting aberrations caused by the optical system.
- the at least one grating element can also have other functions, such as prism functions or phase functions, for aberration correction. These functions can be written into the at least one grid element.
- a gaze tracking system for detecting a direction of gaze of the user.
- a gaze tracking system By providing a gaze tracking system, it can be determined where the user is looking at a certain point in time in the field of view or which part of the field of view and which part or object of the virtual information displayed or the real information in the field of view is currently of interest to the user and therefore is targeted by this. Then the depth position of that targeted object or objects is determined in the z-direction that the user is actively looking at or is aiming at. For real Information about their depth position, ie the distance to the user's eye, can be detected, for example, by means of an additional sensor.
- a superimposed virtual information in the environment of or with content-related reference to the targeted real information in the field of vision or other important virtual information, such as warning notices, should then be displayed, for example, in the same depth as the real information that the user is aiming at.
- the depth position of the image of the spatial light modulator device can be shifted as a segment to that depth position along the z-direction that the user is actively aiming at at that moment.
- the virtual information in the field of vision, which the user does not look at or aim at could be influenced by means of software systems in such a way that this information would, for example, be displayed slightly blurred or slightly blurred or distorted, or can optionally not be displayed at all.
- a detection device can also be provided for determining the area of the field of view in which virtual information is to be displayed.
- the detection device determines in the field of vision of a user in which area of the field of vision one or more virtual information items are to be generated and displayed or represented.
- the lighting device of the display device according to the invention can have at least one light source that can be controlled in pulsed form.
- the method according to the invention has the following features:
- a control device which is coupled to a lighting device for emitting light and a deflection device of an optical system of a display device, for operating the lighting device as a function of an activation of the deflection device in order to at least one image of a spatial light modulator device as a segment at a defined position in Direct a user's field of view,
- the method according to the invention enables information to be presented in an energy-efficient manner in a user's field of vision, since information is only presented and displayed for the user when it is required.
- the detection device can be used to determine which part or area of the field of view is to be filled or displayed with virtual information, for example with two-dimensional and / or three-dimensional objects or scenes and which part or area of the field of view should not have virtual information.
- At least two images of the spatial light modulator device and, corresponding to the number of images of the spatial light modulator device, virtual visibility areas can be generated, the at least two images of the spatial light modulator device being formed as segments in the user's field of vision, preferably combined with one another or overlapping or by one Gap spaced.
- At least one combiner of the optical system can superimpose real information in the field of view with virtual information which is additionally generated in the field of view by the representation of the at least one image of the spatial light modulator device as a segment.
- the at least one image of the light modulator device as a segment can be generated in accordance with a required position in the field of view.
- the at least one image of the spatial light modulator device can be generated dynamically for each frame in such a way that the generated image of the spatial light modulator device as a segment depends on the position of the virtual information to be displayed in the field of view for the respective frame.
- the images of the spatial light modulator device can be generated as segments in particular in such a way that the virtual information in the form of objects is within a minimum number of images of the spatial light modulator device as segments for an individual frame or for an individual image. That is, to display virtual information in the form of, for example, a navigation instruction z. B. only three images of the spatial light modulator device required as segments.
- the number of images to be generated by the spatial light modulator device for the display of virtual information in the form of an object also depends on the size of the object to be displayed.
- the image of the spatial light modulator device can be generated in such a way that the entire object consists of only one image of the spatial light modulator device is generated as a segment and displayed in the field of view.
- the center of the object can, for example, coincide as a segment with the center of the image of the spatial light modulator device.
- the field of view is divided into grid fields, with a check being carried out for each frame to determine in which grid field of the field of view a virtual information item is to be displayed, the spatial light modulator device and at least one scanning mirror element of the deflection device being controlled in such a way that An image of the spatial light modulator device is generated as a segment only for the grid field in which the virtual information is to be displayed per frame and is directed to the defined position in the field of view.
- the field of view can be designed as a type of grid arrangement which has several grid fields. This grid arrangement can be firmly defined and thus be the same for each frame. However, the number of segments to be generated as images of the spatial light modulator device for displaying virtual information is smaller than the total number of segments that would be required to generate the entire field of view.
- the time for generating an individual image of the spatial light modulator device as a segment is 1 / M of the total image time, if M is the number of segments which contain the virtual information.
- a segment in the left field of view area can be generated by a larger gap at a distance from a segment in the right field of view area.
- a scanning mirror element of the deflection device would have to be moved at different speeds in order to bridge the gaps of different sizes.
- the number of segments M which have virtual information can also vary from frame to frame.
- the field of view is divided into grid fields, with all grid fields being scanned one after the other by means of at least one scanning mirror element of the deflection device, with a check being made for each frame to determine in which grid field of the field of view virtual information is to be displayed and only for the respective grid field in which the virtual information is also to be displayed, an image of the spatial light modulator device containing virtual information is generated and assigned as a segment by means of the optical system.
- the field of view can be designed as a type of grid arrangement that has a plurality of grid fields.
- This grid arrangement can be firmly defined and thus be the same for each frame.
- the number of segments to be generated as images of the spatial light modulator device for displaying virtual information is also here smaller than the total number of segments that would be required to generate the entire field of view.
- all grid fields of the grid arrangement are scanned one after the other, but an image of the spatial light modulator device is only generated by controlling the lighting device, the spatial light modulator device and the deflection device if virtual information is to be displayed in the respective grid field to be scanned.
- the individual images of the spatial light modulator device as segments in which virtual information is to be displayed are generated sequentially.
- the time for generating an individual image of the spatial light modulator device as a segment is 1 / N of the total image time, if N is the total number of segments.
- the lighting device is therefore not activated, so that no image of the spatial light modulator device is generated. Thus, no data is transmitted to the spatial light modulator device either, whereby the data transfer is reduced. It would also be possible, should the lighting device still be switched on, that for all pixels that are assigned to a grid field in which no virtual information is to be displayed, a global reset to a level takes place in which the liquid crystal layer of the spatial light modulator device is activated in such a way that the liquid crystals move back into a kind of start state. Another possibility would be that these pixels are switched to an undefined state.
- the at least one scanning mirror element of the deflection device can be moved continuously or in steps with a defined step width as a segment to a defined position in the field of view in order to direct the at least one image of the spatial light modulator device.
- the lighting device is switched on when the at least one scanning mirror element is in a hold state or in a stop state after a defined step width and the spatial light modulator device for generating an image of the spatial Light modulator device is illuminated, whereby the generated image of the spatial light modulator device is directed as a segment to a defined position in the field of view.
- the lighting device is switched off when the at least one scanning mirror element is in a state of motion.
- the lighting device is thus activated in connection with the activation of the deflection device, in particular the at least one scanning mirror element, and brought into an ON state and an OFF state, depending on whether the at least one scanning mirror element is in a stop state or in motion.
- the lighting device and the deflecting device are activated by the control device.
- a compensation mirror element can be combined with the at least one scanning mirror element, the compensation mirror element performing a synchronized, preferably in the same direction, movement to the at least one scanning mirror element when the lighting device is in an ON state.
- the synchronized movement of the compensating mirror element for moving the at least one scanning mirror element is only carried out when the lighting device is in an ON state or is switched on.
- the lighting device is coupled to the deflecting device, the deflecting device also having the compensation mirror element in addition to the at least one scanning mirror element.
- the compensation mirror element is not part of the deflection device. If this is the case, then the lighting device is not only coupled to the deflecting device, but also to the compensating mirror element.
- the control device controls the lighting device and the deflection device and, if necessary, the compensation mirror element, if this is not part of the deflection device.
- At least one scanning mirror element and at least one compensation mirror element can also be designed in such a way that they are movable in two dimensions, preferably horizontally and vertically.
- a generated image of the spatial light modulator device is directed as a segment to a defined horizontal and vertical position in the field of view.
- the position of the image of the spatial light modulator device is shifted as a segment in one direction, for example vertically, but in a direction perpendicular to it, for example horizontally, maintained.
- the scanning mirror element can be continuously moved back and forth between a minimum setting and a maximum setting, for example a continuous movement from left to right and then a continuous movement back from right to left with a slow movement from top to bottom. After the end of a frame, the scanning mirror element can be moved back to its starting position.
- a two-dimensional continuous movement of the at least one scanning mirror element can also take place, for example, in the form of Lissajous figures, so that the initial state of the scanning mirror element is reached again after a frame.
- a one-time calibration can be carried out from which it is determined which settings of the scanning mirror element and, if applicable, of the compensation mirror element correspond to which position of the image of the spatial light modulator device as a segment in the field of view. If, for example, when the scanning mirror element moves step by step, it is provided with a stepping motor and controlled, a specific number of steps of the stepping motor can be assigned to a position in the field of view.
- a calibration can be used to assign the speed of the movement and time interval to a position in the field of view.
- the calibration data can, for example, be stored in a look-up table and this look-up table can be used by the control device for controlling the scanning mirror element.
- the at least one combiner can be designed as a light guide, the spatial light modulator device being illuminated by the lighting device and the light modulated by the spatial light modulator device being directed onto the deflection device, which directs the light onto the combiner designed as a light guide, the light entering the combiner is coupled in and propagated in it, the light propagating in the combiner being coupled out according to the required defined position in the field of view and the at least one image of the spatial light modulator device being directed as a segment to this defined position.
- the at least one image of the spatial light modulator device is shifted as a segment by means of a variable focus system in the z-direction along an optical axis of the optical system to a depth position in the field of view to which a user accommodates.
- the shifting of the image of the spatial light modulator device as a segment to a depth position that a user of the device according to the invention is aiming at or at which the user looks, by means of the variable focus system can be done with a high degree of accuracy, especially for those segments that are in or in the direction of view Near the line of sight of the user are provided.
- Those segments that are shown and displayed in the field of view further away from the viewing direction of the user can be arranged either at any fixed depth or at the same depth as the segments which are located in the viewing direction of the user.
- these segments are provided with a lower accuracy in the firmly defined depth or the same depth as the segments in the direction of view of the user, ie with some tolerances.
- the depth position of the image of the spatial light modulator device may vary as segments for different images of the spatial light modulator device due to aberrations, such as the curvature of the image field, of the optical system in the imaging of the spatial light modulator device.
- Spatial light modulator device By setting the correct depth position for the images of the spatial light modulator device as segments according to the respective determined viewing depth of a user by means of a gaze tracking device in the viewing direction and allowing some tolerances in the depth position of the images Spatial light modulator device as segments further away from the direction of view of the user can advantageously be used a variable focus system with a low image frequency compared to the required image frequency of the deflection device.
- the device according to the invention can be designed as a stereoscopic display device or as a varifocal stereoscopic display device in which an amplitude-modulating spatial light modulator device is used, into which two-dimensional amplitude data are written.
- the spatial light modulator device can be designed as a complex-valued spatial light modulator device.
- This can be, for example, a phase-modulating spatial light modulator device in combination with a beam combiner.
- the two-dimensional information to be displayed is also written into the spatial light modulator device by means of amplitude data.
- the ability of the spatial light modulator device to phase modulate the light can then be used, for example, to write in phase functions for an aberration correction.
- At least one grating element with a relatively low frequency, e.g. 50 Hz - 200 Hz, and with a controllable, tunable grating period can be used in combination with a spatial light modulator device operated at a relatively high frequency, which can be designed as a MEMS SLM, and with static optical elements for aberration correction.
- the static optical elements can undertake an aberration correction for the entire display device.
- the at least one grating element with a controllable, tunable grating period can have the same lens function for all images of the spatial light modulator device as segments for shifting the depth position of all segments, but can also provide an aberration correction for this defined depth position of the segments.
- the aberration correction is the same for all segments.
- the fast MEMS-SLM can also carry out an individual aberration correction for the individual images of the spatial light modulator device as segments, since both the two-dimensional image information and the phase function are updated for an aberration correction in each segment.
- a scattering device or a diffuser can be provided in a stereoscopic display device.
- the scattering device can, for example, in the vicinity of the spatial light modulator device or in an intermediate image plane of the spatial Be arranged light modulator device. With the scattering device, the area of the sweet spot can be widened so that a large virtual visibility area can be created in the observer plane.
- the virtual information is generated and displayed holographically in the field of view of a user.
- the spatial light modulator device can be designed as an amplitude-modulating, phase-modulating or complex-valued (amplitude and phase) spatial light modulator device into which holographic data are written or a hologram is encoded.
- the spatial light modulator means is complex valued, e.g. as a phase-modulating spatial light modulator device in combination with a beam combiner.
- it can be designed as a phase-modulating spatial light modulator device into which holograms calculated iteratively, for example using a Gerchberg-Saxton method, are written.
- a holographic display device normally does not require a variable focus system, since the three-dimensional information to be displayed can already be generated with the complete depth information by means of the hologram encoded in the spatial light modulator device.
- At least one grating element with a controllable, tunable grating period can also be provided, which does not change the position of the image of the spatial light modulator device as a segment but is provided for aberration correction.
- a holographic display device Similar to a varifocal system, it is again possible in a holographic display device to combine a correction of aberrations with a static optical element or to carry out an aberration correction by means of at least one grating element with a controllable, tunable grating period, whereby the grating period can be different for each frame and from the information to be displayed in the respective frame depends.
- An aberration correction could also be carried out directly in the spatial light modulator device, so that the correction of aberrations is already taken into account when calculating a hologram and is included.
- the hologram encoded in the spatial light modulator device can be different for each image to be generated by the spatial light modulator device as the segment carrying the virtual information.
- a different aberration correction could also be carried out by the grating element for each image of the spatial light modulator device as a segment.
- a static aberration correction could also be carried out by tilting the spatial light modulator device relative to the optical system of the display device according to the invention when imaging the spatial light modulator device .
- Fig. 1 in a basic representation of an AR display device, shown in
- Fig. 2 in a schematic representation of an inventive
- 3 in a basic representation, a further embodiment of a display device according to the invention in plan view; 4: a basic illustration of a subdivision of the field of view of a
- Fig. 5 a basic representation of images of a spatial
- Light modulator device as segments in the field of view, each of which has virtual information for a user
- Light modulator device as segments in the field of view, each of which has virtual information for a user
- Fig. 7 a basic representation of an in an inventive
- Display device provided deflection device according to the invention in different control states
- FIG. 8 shows a basic representation of a light coupling into a light guide
- FIG. 9 shows a basic representation of a display device according to the invention, which has a combiner designed as a light guide and is provided for generating at least two images of a spatial light modulator device as segments in the field of view, when using a spatial light modulator device with a relatively small number of pixels.
- a display device which is designed here as an augmented reality display (AR).
- the AR display device is designed here in the form of glasses, so that the display device is designed as an AR head-mounted display, which shows what a user B of the AR glasses can view through them in his field of view S.
- user B is shown here only by two arms with two hands, for example, holding onto a bicycle handlebar.
- the display device in the form of AR glasses is attached to the head of user B. The user B thus looks through the AR glasses and can use them to his natural or real Consider environment R. 1 thus shows only the field of view S of user B. In his field of view in FIG.
- the virtual information C1, C2 and C3 shown therefore only fill a small part of the field of view S. This means that only a small percentage of the field of view S is formed by virtual information.
- Most of the user's field of view S is formed by the content of the real environment R.
- a possible embodiment of the display device is shown. This refinement could be used both for an AR head-mounted display and for a head-up display.
- the display device should be designed as an AR head-mounted display in order to establish a connection with FIG. 1.
- the display device has an illumination device 10 which can have at least one light source, three light sources according to the basic colors RGB (red-green-blue) being able to be provided for a colored representation of the virtual information.
- the SLM 11 is designed here as an SLM with a relatively small number of pixels, for example fewer than 1000 pixels in one direction.
- a deflection device 12 and a combiner 13 which are both components of an optical system of the display device.
- the deflection device 12 here has a scanning mirror element 12-1 which is movably arranged and can move or rotate about its axis of rotation.
- the deflection device 12 with the scanning mirror element 12-1 is in the beam path between the SLM 11 and the combiner arranged.
- the scanning mirror element 12-1 can carry out a continuous movement or also a step-by-step movement with a firmly defined step width, with which the incident light can be directed in a specific direction.
- the combiner 13, which is designed as a spectacle lens according to FIG. 1, is provided for superimposing virtual information generated by the display device with information in the real environment in the user's field of vision.
- the combiner 13 is designed in such a way that light from the real environment can pass through the combiner unhindered, ie is not influenced by the combiner.
- the combiner 13 can be flat or planar or also curved.
- the optical system can have further imaging elements, such as, for example, an imaging element 14 which is designed here as a passive lens element.
- the display device has a control device 15 which is coupled to the lighting device 10 and the deflection device 12.
- the lighting device 10 can be controlled and switched accordingly as a function of an activation of the deflection device 12, here in particular the scanning mirror element 12-1, i.e. can be brought into an ON state and an OFF state.
- the control device 15 could also be coupled to the SLM 11. However, it is also possible for the SLM 11 to be operated by its own control device for writing data.
- the following describes the general procedure for generating virtual information, such as, for example, the virtual information C1 according to FIG. 1.
- the virtual information should be generated holographically, with stereoscopic generation of course also being possible.
- the lighting device 10 If the lighting device 10 controlled by the control device 15 has been switched to a corresponding ON state, the lighting device 10 emits light that is essentially sufficiently coherent and impinges on the SLM 11, with data of the virtual information being transmitted to the SLM 11 or transmitted.
- the light emitted by the lighting device 10 and incident on the SLM 11 is shown here by an arrow.
- the light now modulated by the SLM 11 with the virtual information to be displayed passes through the imaging element 14, whereby an image of the SLM 11 is generated on the scanning mirror element 12 - 1 of the deflection device 12.
- the deflection device 12 is arranged here in a Fourier plane of the SLM 11.
- the scanning mirror element 12-1 was already activated by the control device 15 prior to the activation of the lighting device 10 in such a way that it has moved into a position that is necessary for the display of this virtual information at a defined position in the field of view S of a user B, who is passing through here the eye should be represented is required.
- a detection device 16 is used to determine for each frame before the generation of the virtual information which area of the field of view S with virtual information, for example a two-dimensional or three-dimensional object or scene, is to be filled and which area in the field of view S should not have any virtual information but only information about the real environment of the user B.
- the image of the SLM 11 is now directed as segment S1 by means of the scanning mirror element 12-1 in the direction of the combiner 13, which superimposes the image of the SLM 11 as segment S1 with the real environment.
- the image of the SLM 11 is mapped as segment S1 by the combiner 13 into an observer plane 17 in order to generate a virtual visibility area 18 there.
- the virtual visibility area 18 can be a virtual viewer window in the case of a holographic display device or a sweet spot in the case of a stereoscopic display device. In this way, the virtual information is represented and displayed at the defined position in the field of view S.
- images of the SLM 11 can be generated as segments S2 and S3, directed by means of the scanning mirror element 12-1 to the required and defined positions in the field of view S and by means of the combiner 13 these segments S2 and S3 are superimposed on the real environment and the user B in the field of view S are shown and displayed.
- the images of the SLM 11 as segments S1, S2 and S3 are generated sequentially and shown and displayed in the field of view S. However, this takes place at such a high frequency that the eye of user B cannot recognize this successive generation of segments S1, S2 and S3 with the naked eye and thereby perceives them to be simultaneous.
- the combiner 13 can also have a fixed (non-variable) focusing element, for example a grating element.
- the display device in FIG. 2 also has a variable focus system 19.
- the variable focus system 19 makes it possible to vary the distance of the image of the SLM 11 as a segment in the field of view towards the user B, ie to set the depth of the image of the SLM 11 as a segment in the field of view S.
- variable focus system 19 is preferably arranged in the area of the Fourier plane of the SLM 11, ie in the Fourier plane of the SLM 11 or at least in the vicinity of the Fourier plane of the SLM 11, and can for example have at least one grating element have an adjustable, tunable grating period in which a lens function is written.
- the setting of the depth of the image of the SLM 11 as a segment in connection with the detection of a viewing direction of the user B can preferably take place.
- a gaze tracking device 20 determines the direction of view of user B and also the depth position in the field of view into which user B focuses or the depth that he is aiming at.
- the image generated by the SLM 11 as a segment can then be shifted by means of the variable focus system 19 to that depth position towards user B that user B is aiming at or looking at at that moment.
- variable focus system is not absolutely necessary in a holographic display device, since the virtual information can already be displayed holographically with its required depth. However, it could nevertheless be useful to use a variable focus system, for example in order to correct aberrations caused by the optical system by shifting the image of the SLM 11 as a segment in the depth or along the z-direction.
- the variable focus system 19 can have at least one grating element with an adjustable, tunable grating period which, for example, has prism functions or phase functions.
- variable focus system 19 In stereoscopic display devices, it makes sense to use such a variable focus system 19 in order to shift the image of the SLM as a segment in its depth and / or to correct aberrations of the optical system.
- a further display device is shown, which can also be designed as an AR display device or AR display and can be used both as an AR head-mounted display and as an AR head-up display.
- the display device has an illumination device 30, an SLM 31, a deflection device 32, a combiner 33 and imaging elements, only one imaging element 34 being shown here.
- the deflection device 32, the combiner 33 and the imaging element 34 are components of an optical system of the display device.
- the lighting device 30 is already followed by the deflection device 32, which here also has a scanning mirror element 32-1.
- the imaging element 34, the SLM 31 and the combiner 33 are arranged. This means that the deflection device is provided here between the lighting device 30 and the SLM 31.
- the SLM 31 is here as an SLM with a relatively large number of pixels, such as greater than 1000 pixels in one direction.
- the scanning mirror element 32-1 of the deflection device 32 is movably arranged, as is to be made evident by the dashed lines, and can thus move or rotate about its axis of rotation.
- the scanning mirror element 32-1 can perform a continuous or step-by-step movement with a firmly defined step width, with which the incident light can be directed in a specific direction.
- the combiner 33 which can also be designed as a spectacle lens in this exemplary embodiment, but should not be restricted thereto, is provided for superimposing virtual information generated by the display device with information in the real environment in the field of view S of user B.
- the combiner 33 is also designed here in such a way that light from the real environment can pass through the combiner 33 unhindered, ie is not influenced by the combiner 33.
- the combiner 33 can be flat or even or also curved.
- the display device has a control device 35 which is coupled to the lighting device 30 and the deflection device 32.
- the lighting device 30 can also be controlled and switched accordingly in this exemplary embodiment as a function of an activation of the deflection device 32, here in particular the scanning mirror element 32-1. can be brought into an ON state and an OFF state.
- the control device 35 could also be coupled to the SLM 31.
- the display device can also have a gaze tracking device 39 which determines the viewing direction of the user B and also the depth position in the field of view into which the user B is focusing or the depth that he is aiming at. The generated image of the SLM 31 as a segment can then, if necessary, be shifted to that depth position towards user B by means of a variable focus system that user B is aiming at or looking at at that moment.
- the following describes the general procedure for generating virtual information, such as the virtual information C1 according to FIG. 1, in connection with the display device shown in FIG. 3.
- virtual information should be generated holographically, with stereoscopic generation of course also being possible.
- the lighting device 30 controlled by the control device 35 If the lighting device 30 controlled by the control device 35 has been switched to a corresponding ON state, the lighting device 30 emits light which is essentially sufficiently coherent and which strikes the deflection device 32, in particular the scanning mirror element 32-1.
- the deflection device 32 is now arranged in front of the SLM 31 in the direction of light.
- the scanning mirror element 32-1 has already been before the control of the lighting device 30 is controlled by the control device 35 in such a way that it has moved into a position necessary for the display of the required virtual information at a defined position in the field of view S of a user B, who is to be shown here by the eye, is required.
- a detection device 36 determines which area of the field of view S should be filled with virtual information, for example a two-dimensional or three-dimensional object or scene, and which area in the field of view S should not have any virtual information but only information about the real environment of user B.
- the light emitted by the lighting device 30 and impinging on the scanning mirror element 32-1 is also shown here by an arrow.
- the light L1 reflected and directed by the scanning mirror element 32-1 of the deflection device 32 in accordance with a defined position in the field of view S then strikes the imaging element 34, which collimates the light L1.
- This collimated light L1 now strikes the SLM 31, only a partial area of the SLM 31 being illuminated in this case.
- FIG. 3 only a left sub-area of the SLM 31 is illuminated by means of the light L1, this sub-area being the entire left-hand area of the SLM 31 or only a sub-area in the left-hand area of the SLM 31.
- the illustration is only shown as an example .
- data of the virtual information are transmitted or transmitted to the corresponding sub-area of the SLM 31, here the left-hand sub-area.
- the information for the virtual information to be displayed with the light L1 is therefore only located in the field of view S in this partial area that is illuminated on the SLM 31.
- the light incident on this partial area of the SLM 31 is modulated with the information to be displayed and then hits as Segment S1 on the combiner 33.
- the combiner 33 now serves as an imaging element for generating an image of the SLM 31, here an image of the partial area of the SLM 31, and also superimposes this image of the SLM 31 as segment S1 with the real environment of the user B.
- the image of the SLM 31 as a segment S1 is mapped into a viewer plane 37, as a result of which a virtual visibility area 38 is formed.
- the virtual visibility area 38 can be a virtual viewer window in the case of a holographic display device or a sweet spot in the case of a stereoscopic display device. In this way, the virtual information is represented and displayed at the defined position in the field of view S.
- the user B In order to be able to observe the virtual information in the field of view S, the user B must arrange his eye in the observer plane 37 and look through the virtual visibility area 38.
- the same procedure as described can be used.
- Different positions of the scanning mirror element 32-1 of the deflecting device 32 create different directions of light bundles L2, L3 corresponding to required positions of the virtual information in the field of view S, which then impinge on different partial areas of the SLM 31.
- Images of the SLM 31 are thus generated as segments S2 and S3, directed to the required and defined positions in the field of view S and shown and displayed to the user B in the field of view S.
- the images of the SLM 11 as segments S1, S2 and S3 are generated sequentially and shown and displayed in the field of view S. However, this takes place at such a high frequency that the eye of user B cannot recognize this successive generation of segments S1, S2 and S3 with the naked eye and thereby perceives them to be simultaneous.
- the combiner 33 can also have a fixed focusing element, for example a grating element.
- the display device according to FIG. 3 can also have a variable focus system.
- the variable focus system can be designed in accordance with the variable focus system 19 according to FIG. 2, so that the same should apply to the display device according to FIG. 3.
- the display devices according to FIGS. 2 and 3 can be used for the following embodiments and configurations according to FIGS. 4 to 7 and 9, in which special procedures of a method for generating virtual information are described.
- FIG. 4 shows the AR glasses according to FIG. 1, which a user B wears on his head in order to additionally receive virtual information that can be faded in and displayed in his real environment in the field of vision.
- the field of view S of user B is divided into individual grid fields RF, which are arranged as a type of grid or form a grid arrangement.
- the grid fields RF here all have the same shape and size. They are square in this embodiment.
- the grid fields RF can also have a different shape and size.
- the size and shape of the grid fields can vary across the field of view S.
- the field of view S is scanned or rasterized with the grid fields RF and determined by the detection device where in the field of view the virtual information C1, C2 and C3 useful for the user B are to be displayed or are to be displayed.
- each raster field RF of the field of view S is now approached one after the other line by line by moving the scanning mirror element of the deflection device step by step with a defined step width and only the raster field is assigned an image of the SLM as a segment in which virtual information is also to be displayed.
- the moving scanning mirror element is put into a stop state by means of the control device, so that the lighting device is also activated by means of the control device and brought into an ON state, whereupon in connection with the SLM and the combiner and the at least one imaging element of the optical system, an image of the SLM is generated as a segment and assigned to the raster field RF1. Part of the virtual information C1 is thereby displayed.
- the control device now controls the scanning mirror element and the lighting device again, so that the scanning mirror element is put into an ON state and the lighting device is put into an OFF state.
- the scanning mirror element now moves a defined step width, so that a grid field RF2 is approached, from which it was also determined that this grid field RF2 contributes to the display of the virtual information C1.
- the control device now controls the scanning mirror element and the lighting device again accordingly, so that the scanning mirror element is put into the stop state and the lighting device is put into the ON state.
- an image of the SLM corresponding to the virtual information part to be displayed is generated as a segment by means of the SLM, the combiner and the at least one imaging element of the optical system and assigned to the grid field RF2 so that the corresponding virtual information is displayed there.
- the two subsequent grid fields RF3 and RF4 are approached with the scanning mirror element in accordance with what has been disclosed above and an image of the SLM is generated as a segment that carries the virtual information. These two images of the SLM as segments are then assigned to the two grid fields RF3 and RF4, as can be seen in FIG.
- the scanning mirror element is then moved further step by step with a defined step width along this upper line of the raster grid and in each case moved into a stop state and an ON state during the scanning of the further raster fields.
- the control device Since it was determined for the other grid fields RF5 to RF15 of this line that no virtual information is to be provided and displayed for these grid fields, the control device will not control the lighting device, so that the lighting device remains in an OFF state for these grid fields RF5 to RF15 and no images of the SLM are generated as segments.
- the second line of the raster grid is then approached by means of the scanning mirror element, with no image of the SLM being generated as a segment for the first raster field RF16 since no virtual information is to be displayed in this raster field.
- each grid field of the grid is approached one after the other by means of the scanning mirror element and an image of the SLM is generated as a segment for the other grid fields RF35 and RF48 to RF51 and RF63, RF64 and assigned to the associated grid field and displayed.
- the grid fields RF1, RF2, RF3, RF4, RF17, RF18 and RF19 thus contribute to the representation of the virtual information C1.
- the grid fields RF20 to RF26 and RF35 contribute to the display of the virtual information C2 and the grid fields RF48 to RF51 and RF63, RF64 to the display of the virtual information C3.
- the procedure described above is used for each frame.
- the individual images of the SLM are generated and displayed as segments in a time-sequential manner.
- the method has been described with a step-by-step movement of the scanning mirror element. However, it is also possible for the scanning mirror element to move continuously. 7 describes this in detail later.
- the procedure can also be slightly modified.
- the field of view is first divided into grid fields RF in the manner of a grid, which are then scanned or scanned and by means of the detection device it is determined where in the field of view the virtual information C1, C2 and C3 useful for the user B should or should be displayed. This means that it is checked and determined in which grid fields RF of the field of view S the virtual information C1, C2 and C3 are to be displayed. Because only these grid fields RF of the field of view S have to be filled with corresponding virtual information that is superimposed on the real information present there.
- the scanning or rastering of the field of view can also take place in this procedure line by line or column by column grid field per grid field.
- FIG. 6 shows a further exemplary embodiment for the procedure for generating and displaying virtual information.
- the positioning of the images of the SLM as segments for displaying the virtual information does not take place on a grid or a fixed grid, as explained in the exemplary embodiments according to FIGS. 4 and 5, but can be freely selected in the field of view of user B of the display device. In this way, it is possible to reduce the number of images of the SLM as segments for displaying the same virtual information C1, C2 and C3 or the same content as in FIGS. 4 and 5.
- the virtual information C1, C2 and C3 can be generated in FIG. 6 with a smaller number of images of the SLM as segments, namely with only 17 images of the SLM as segments instead of 21 images of the SLM as segments according to FIG.
- the images of the SLM as segments for the respective virtual information C1, C2 or C3 are thus only generated for each frame at those positions in the field of view at which the information is necessary.
- the images of the SLM are generated as segments in such a way that the virtual information as an object takes up the entire image of the SLM as far as possible if the object is larger than the image of the SLM, or the virtual information as an object is completely in the image of the SLM is provided when the size of the object is smaller than the image of the SLM.
- the center of the object can coincide as a segment with the center of the image of the SLM. This means that the required virtual information can be displayed as segments with only a minimal number of images from the SLM.
- the virtual information C1 is generated and displayed in the field of view as follows.
- the detection device is used to determine at which position in the field of view the virtual information C1 is to be represented and displayed. It is then determined with what number of images of the SLM as segments the information C1 to be displayed must be generated and can be displayed with as few images as possible or even with only one image of the SLM as a segment. If a suitable number and also the required position of the respective image of the SLM as a segment in the field of view S is determined, the scanning mirror element of the deflection device is moved by the control device to the relevant position for an image of the SLM as a segment in the field of view S and then into a stop Condition kept.
- data of the virtual information to be displayed in this segment are transmitted to the SLM or generated by the SLM itself and encoded on it.
- the control device controls the lighting device so that it is switched to the ON state, whereupon data of the virtual to be displayed Information in this segment is displayed in such a way that the virtual information is located completely in the image of the SLM as total information or as partial information, so that only a few images of the SLM are required as segments to display the virtual information in the field of view S.
- the lighting device now illuminates the SLM in order to modulate the light according to the virtual information and, in conjunction with the combiner and the optical system, to generate an image of the SLM as segment BS1 and to direct this to the determined position in the field of view S by means of the scanning mirror element. Thereafter, to display the virtual information C1, another image of the SLM is generated and displayed as segment BS2 in the same way. This procedure continues until the virtual information C1 is completely displayed in the field of view S for the user.
- the generation and display of the images of the SLM as segments is therefore also time-sequential in this exemplary embodiment.
- the individual images of the SLM can also overlap as segments in order to display the virtual information C1 according to FIG. 1.
- the generation and presentation of the virtual information C2 and C3 takes place in the same way as the generation and presentation of the virtual information C1.
- the scanning mirror element can move at a higher speed can be moved from the position of the image of the SLM as segment BS6 to, for example, a position for the still to be generated image of the SLM as segment BS7 for the virtual information C2 in order to pass the existing gap between these two segments BS6 and BS7 more quickly.
- the speed of the movement of the scanning mirror element should be higher than when approaching the respective position for the image of the SLM as a segment for the virtual information C1.
- the respective positions of the SLM images to be generated and displayed can be approached as segments for the virtual information C2 in the field of view S with a lower speed of movement of the scanning mirror element, similar to the speed as when displaying the virtual information C1.
- the generation and display of the respective images of the SLM as segments for the virtual information C2 is carried out in the same way as for the generation and display of the virtual information C1.
- the same procedure can be used to display the virtual information C3.
- the transition from the last generated image of the SLM as segment BS 13 for the virtual information C2 to the still-to-be generated image of the SLM as segment BS 14 of the virtual information C3 can take place at a higher speed of movement of the scanning mirror element around the large gap to run faster in time.
- This sequence of approaching positions and generating images of the SLM as segments in the field of view S is only intended to be exemplary. A different order is of course also possible. For example, after the last image of the SLM as segment BS13, the image of the SLM that is still to be generated could also be generated and displayed as segment BS17, since the gap between the two segments BS13 and BS17 is not as large as between the segments BS13 and BS14, and thus this one Position of segment BS17 can be approached faster.
- the images of the SLM are displayed as segments freely in the field of view, can overlap and can also have different shapes and / or sizes.
- FIGS. 4 to 6 the generation and display of virtual information was described with the aid of a step-by-step movement of the scanning mirror element with a defined step width.
- a deflector 50 includes a scanning mirror element 51 and a Compensation mirror element 52.
- the scanning mirror element 51 is the element of the deflector 50 that performs continuous movement.
- the compensation mirror element 52 is also movably mounted.
- the scanning mirror element 51 and the compensation mirror element 52 are arranged at an angle to one another, as can be seen in FIG. 7. In illustration (a) in FIG. 7, this angle is approximately 90 degrees.
- control device 53 which also controls a lighting device (not shown) and possibly also an SLM.
- segment BS1 the generation of a first image of the SLM is shown as segment BS1.
- both mirror elements 51 and 51 form at the moment the image of the SLM is generated as segment BS1 at a previously defined position in the user's field of vision
- the compensation mirror element 52 a defined angle to one another, the compensation mirror element 52 remaining fixed, i.e. is not moving.
- the scanning mirror element 51 continues to move.
- the light L impinging on the deflection device 50 thus strikes the scanning mirror element 51 first and is reflected by the latter in the direction of the compensation mirror element 52 in accordance with its alignment.
- the light impinging on the compensation mirror element 52 is also reflected by the latter in accordance with its orientation, hits the combiner and is then directed to a corresponding position in the user's field of vision.
- This embodiment of the deflection device 50 could occur, for example, if a first image of the SLM is generated as a segment at a defined position in the field of view, so that at the moment when the scanning mirror element 51 is first activated by the control device 53 for continuous movement, the Lighting device is controlled by means of control device 53 and emits light in order to generate and display an image of the SLM as a segment for virtual information precisely at this first position of the scanning mirror element in the field of view.
- the compensation mirror element 52 is rotated in the same direction by the same amount as the scanning mirror element 51, so that at the point in time at which the lighting device is controlled by the control device 53 of the display device and placed in an ON state, the SLM is illuminated and the Image of the SLM is generated and displayed as a segment at the same position in the field of view as in illustration a).
- the image of the SLM is thus at the same position in the field of view displayed as segment BS1.
- the compensation mirror element 52 thus compensates for the movement of the scanning mirror element 51.
- the dashed lines and arrows are intended to show the incident light beam on the two mirror elements 51 and 52 according to illustration a), the solid lines and arrows being intended to illustrate the incident, offset light beam.
- the advantage of such an arrangement of scanning mirror element and compensation mirror element in the deflection device is that continuously scanning or moving mirror elements are often faster in their speed of movement than mirror elements which move step by step from point to point and are then stopped.
- the image of the SLM as Segment BS1 are displayed.
- the compensation mirror element 52 is moved to its initial state, no image of the SLM is generated as a segment until the required new position of the compensation mirror element 52 is reached.
- the lighting device When the compensation mirror element 52 moves into its initial state, the lighting device is switched off or is in the OFF state. Only when the compensation mirror element 52 has reached its new required position is the lighting device switched back to the ON state by means of the control device 53. Thereafter, both mirror elements 51 and 52 or only one of the two could move on and another or further image of the SLM could be generated as a segment and displayed to the user in the field of view.
- FIG. 1 A general generation of information in connection with a coupling of light into a light guide according to the prior art is shown in FIG.
- An SLM 60 is used here, which has a relatively large number of pixels and thus an HD (high definition) TV resolution or higher.
- an imaging element 62 for example a lens element, arranged in the beam path between the SLM 60 and a light guide 61, a coupling angle spectrum 63 of the light modulated and emitted by the SLM 60 is generated in such a way that the light rays emanating from the individual pixels of the SLM 60 under im Means of different angles relative to the surface of the light guide 61 impinge on the light guide 61.
- This coupling angle spectrum 63 hits the light guide 61 and is coupled into the light guide 61 by means of a mirror surface 64.
- the mirror surface 64 is fixedly arranged within the light guide 61 at a defined angle.
- the light rays that strike the mirror surface 64 are reflected by the latter and propagate with total reflection in the light guide 61.
- a decoupling device 65 which has corresponding decoupling elements, for example decoupling grating elements, the light can be emitted from the light guide 61 in the direction of a user's eye B, whereby a decoupling angle spectrum 66 can be defined.
- An image of the SLM 60 is mapped into an observer plane 67 in order to generate a virtual visibility area 68 there.
- WO 2019/012028 A1 the disclosure of WO 2019/012028 A1 to be included here in its entirety.
- a light guide is described which is designed such that the coupling-out takes place after a fixed number of reflections in the light guide and that the coupling-out angle spectrum is enlarged compared to the coupling-in angle spectrum.
- the decoupled light propagates further to a visibility area and the decoupling angle spectrum corresponds to the field of view.
- the propagation and decoupling of light should not be limited to these possibilities.
- FIG. 9 shows the generation of images of the SLM as segments by means of a display device according to the invention.
- This display device of FIG. 9 can in particular be designed as an AR display device, such as, for example, an AR head-mounted display or also an AR head-up display.
- this display device can generate the virtual information holographically or stereoscopically and present it in the field of view of a user.
- Representation a) shows the display device when a first image of an SLM is generated as a segment and representation b) in FIG. 9 shows generation of a second image of an SLM as a segment.
- the display device of FIG. 9 has an illumination device 70, an SLM 71, a combiner 72, a deflection device 73 and at least one imaging element 74 of an optical system.
- the SLM 71 now has a relatively small number of pixels, e.g. less than 1000 pixels in one direction, on.
- the deflection device 73 here has a scanning mirror element 73-1 for deflecting the incident light, which is movably and thus rotatably mounted.
- the scanning mirror element 73-1 is arranged in the vicinity of a light coupling surface of the combiner 72 so that coupling can take place with high accuracy.
- the optical system of the display device should be represented here by the imaging element 74, the combiner 72 and the deflection device 73, it being understood that several imaging elements or other optical elements can also be provided.
- the imaging element 74 is provided between the SLM 71 and the deflection device 73 in the beam path.
- the combiner 72 is designed as a light guide which can be designed to be planar or flat or also curved.
- a control device 75 is provided which is coupled to the illumination device 70 and the deflection device 73, in particular the scanning mirror element 73-1. It can also be additionally coupled to the SLM 71, whereby the SLM 71 itself can also be operated via its own control device.
- the lighting device 70 If the lighting device 70 is now in an ON state due to activation by means of the control device 75, it sends light to the SLM 71, modulates this light from the SLM 71 and hits the deflection device 73 via the imaging element 74.
- the imaging element 74 generates a coupling angle spectrum of the light which is smaller in its extent than the coupling angle spectrum of the light according to FIG.
- This coupling angle spectrum is then coupled into the combiner 72 designed as a light guide, for example by means of a coupling device which e.g. may have a mirror element or at least one grating element, for example a volume grating.
- the light can be decoupled from the combiner 72 via a decoupling device 77 which has at least one decoupling element, e.g. a decoupling grating element, such as a volume grating, so that the decoupled light is directed towards a user B in a viewer plane 78 and there forms a virtual visibility area 79 through which the user B can then view the virtual information (s) generated in the field of view .
- a decoupling device 77 which has at least one decoupling element, e.g. a decoupling grating element, such as a volume grating, so that the decoupled light is directed towards a user B in a viewer plane 78 and there forms a virtual visibility area 79 through which the user B can then view the virtual information (s) generated in the field of view .
- a first image of the SLM 71 is generated and represented as a segment which has virtual information in the field of view.
- a detection device is used to determine where in the field of view virtual information is to be displayed. If this is known, the control device 75 controls the scanning mirror element 73-1 in order to move it into a defined position that is required to display at least part of the virtual information. In addition, the control device 75 puts the lighting device 70 in an ON state, so that the SLM 71 illuminates with light is transmitted to the SLM 71 data to represent the virtual information.
- the light incident on the SLM 71 is modulated by the latter in accordance with the virtual information and is incident on the imaging element 74, whereby an image of the SLM 71 is generated on the scanning mirror element 73-1, which is arranged in the Fourier plane of the SLM 71.
- An angular spectrum of the light 76 generated with the imaging of the SLM 71 by means of the imaging element 74 is coupled by means of the scanning mirror element 73-1 into the combiner 72 in the form of a light guide at a first central angle ⁇ and propagates further in the combiner 72 with total reflection.
- the decoupling device 77 If the light propagating in the combiner 72 hits the decoupling device 77 at a defined angle, it is decoupled from the combiner 72 and the image from the SLM 71 is directed as a segment into the observer plane 78, in which a virtual visibility area 79 is formed. In this way, a first image of the SLM 71 is generated and displayed as a segment in the form of virtual information at a defined position in the field of view.
- a second image of the SLM 71 is generated and displayed as a segment at a second defined position in the field of view of user B that is different from the first defined position.
- the scanning mirror element 73-1 has been moved or rotated to a different position by the control device 75, i. from the position shown in dotted lines to the position with the solid line along the arrow.
- the central angle of the light bundle that is coupled into the combiner 72 is changed.
- the angular spectrum of the light 76 generated with the image of the SLM 71 is then at a different central angle by means of the scanning mirror element 73-1, i.e.
- the light now propagating in the combiner 72 at different angles compared to illustration a) can, however, be coupled out in the same way as described for illustration a).
- a second image of the SLM 71 is generated and displayed as a segment at a position in the field of view that differs from the first image of the SLM 71 as a segment for displaying further virtual information.
- the scanning mirror element 73-1 is rotated by further defined step widths by means of the control device 75 in order to display the images of the SLM 71 as segments in the field of view at the respective defined position.
- a step-by-step movement of the scanning mirror element 73-1 was assumed.
- the deflection device 73 of FIG. 9 can also be designed according to FIG. 7, whereby a continuous movement of the scanning mirror element is provided.
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
Abstract
Description
Claims
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020217040763A KR20220008315A (ko) | 2019-05-16 | 2020-05-14 | 에너지 소비가 감소된 디스플레이 디바이스 |
CN202080051725.7A CN114174893A (zh) | 2019-05-16 | 2020-05-14 | 具有降低功耗的显示装置 |
US17/595,402 US20220247982A1 (en) | 2019-05-16 | 2020-05-14 | Display apparatus with a reduced power consumption |
DE112020002420.2T DE112020002420A5 (de) | 2019-05-16 | 2020-05-14 | Anzeigevorrichtung mit reduziertem Energieverbrauch |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP19174874 | 2019-05-16 | ||
EP19174874.8 | 2019-05-16 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020229599A1 true WO2020229599A1 (de) | 2020-11-19 |
Family
ID=67437290
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2020/063462 WO2020229599A1 (de) | 2019-05-16 | 2020-05-14 | Anzeigevorrichtung mit reduziertem energieverbrauch |
Country Status (5)
Country | Link |
---|---|
US (1) | US20220247982A1 (de) |
KR (1) | KR20220008315A (de) |
CN (1) | CN114174893A (de) |
DE (1) | DE112020002420A5 (de) |
WO (1) | WO2020229599A1 (de) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11705089B2 (en) * | 2020-04-07 | 2023-07-18 | Texas Instruments Incorporated | Display spatial brightness control |
US20210389152A1 (en) * | 2020-06-10 | 2021-12-16 | Here Global B.V. | Method, apparatus, and system for projecting augmented reality navigation cues on user-selected surfaces |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100097671A1 (en) | 2006-12-12 | 2010-04-22 | See Real Technologies S.A. | Head-Mounted Display Device for Generating Reconstructions of Three-Dimensional Representations |
US20130222384A1 (en) | 2010-11-08 | 2013-08-29 | Seereal Technologies S.A. | Display device, in particular a head-mounted display, based on temporal and spatial multiplexing of hologram tiles |
US20140139404A1 (en) * | 2012-11-20 | 2014-05-22 | Seiko Epson Corporation | Virtual image display apparatus |
WO2017158073A1 (en) * | 2016-03-16 | 2017-09-21 | Seereal Technologies S.A. | A display for two-dimensional and/or three-dimensional images |
US20180120559A1 (en) * | 2016-10-28 | 2018-05-03 | Magic Leap, Inc. | Method and system for large field of view display with scanning reflector |
WO2018146326A2 (de) | 2017-02-13 | 2018-08-16 | Seereal Technologies S.A. | Lichtleitvorrichtung und anzeigevorrichtung zur darstellung von szenen |
WO2018211074A1 (de) | 2017-05-19 | 2018-11-22 | Seereal Technologies S.A. | Anzeigevorrichtung mit einem lichtleiter |
WO2019012028A1 (de) | 2017-07-13 | 2019-01-17 | Seereal Technologies S.A. | Anzeigevorrichtung zur vergrösserung des sichtfelds |
WO2019076963A1 (de) | 2017-10-18 | 2019-04-25 | Seereal Technologies S.A. | Anzeigevorrichtung und verfahren zur erzeugung eines grossen sichtfeldes |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
SE0200547D0 (sv) * | 2002-02-25 | 2002-02-25 | Micronic Laser Systems Ab | An image forming method and apparatus |
CN108107579B (zh) * | 2017-12-18 | 2021-02-19 | 杭州光粒科技有限公司 | 一种基于空间光调制器的全息光场大视域大出瞳的近眼显示*** |
-
2020
- 2020-05-14 CN CN202080051725.7A patent/CN114174893A/zh active Pending
- 2020-05-14 US US17/595,402 patent/US20220247982A1/en active Pending
- 2020-05-14 KR KR1020217040763A patent/KR20220008315A/ko unknown
- 2020-05-14 DE DE112020002420.2T patent/DE112020002420A5/de active Pending
- 2020-05-14 WO PCT/EP2020/063462 patent/WO2020229599A1/de active Application Filing
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100097671A1 (en) | 2006-12-12 | 2010-04-22 | See Real Technologies S.A. | Head-Mounted Display Device for Generating Reconstructions of Three-Dimensional Representations |
US20130222384A1 (en) | 2010-11-08 | 2013-08-29 | Seereal Technologies S.A. | Display device, in particular a head-mounted display, based on temporal and spatial multiplexing of hologram tiles |
US20140139404A1 (en) * | 2012-11-20 | 2014-05-22 | Seiko Epson Corporation | Virtual image display apparatus |
WO2017158073A1 (en) * | 2016-03-16 | 2017-09-21 | Seereal Technologies S.A. | A display for two-dimensional and/or three-dimensional images |
US20180120559A1 (en) * | 2016-10-28 | 2018-05-03 | Magic Leap, Inc. | Method and system for large field of view display with scanning reflector |
WO2018146326A2 (de) | 2017-02-13 | 2018-08-16 | Seereal Technologies S.A. | Lichtleitvorrichtung und anzeigevorrichtung zur darstellung von szenen |
WO2018211074A1 (de) | 2017-05-19 | 2018-11-22 | Seereal Technologies S.A. | Anzeigevorrichtung mit einem lichtleiter |
WO2019012028A1 (de) | 2017-07-13 | 2019-01-17 | Seereal Technologies S.A. | Anzeigevorrichtung zur vergrösserung des sichtfelds |
WO2019076963A1 (de) | 2017-10-18 | 2019-04-25 | Seereal Technologies S.A. | Anzeigevorrichtung und verfahren zur erzeugung eines grossen sichtfeldes |
Also Published As
Publication number | Publication date |
---|---|
US20220247982A1 (en) | 2022-08-04 |
DE112020002420A5 (de) | 2022-02-24 |
KR20220008315A (ko) | 2022-01-20 |
CN114174893A (zh) | 2022-03-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
DE112015002814B4 (de) | Anzeigevorrichtung für virtuelle Bilder | |
DE102007025069B4 (de) | Holographisches Rekonstruktionssystem | |
WO2019185510A1 (de) | Anzeigevorrichtung | |
DE102006024356B4 (de) | Holographische Projektionsvorrichtung zur Rekonstruktion von Szenen und Verfahren zur holographischen Rekonstruktion | |
WO2019012028A1 (de) | Anzeigevorrichtung zur vergrösserung des sichtfelds | |
WO2018011285A1 (de) | Anzeigevorrichtung | |
EP3548954B1 (de) | Head-up-display | |
WO2012062681A1 (de) | Anzeigegerät, insbesondere ein head-mounted display, basierend auf zeitlichen und räumlichen multiplexing von hologrammkacheln | |
WO2007099458A2 (de) | Holographische projektionsvorrictung zur vergrösserung eines rekonstruktionsbereichs | |
EP3254152A2 (de) | Verfahren und vorrichtungen zur dateneinspiegelung mittels holografischer optischer elemente | |
DE102015213376A1 (de) | Projektionsvorrichtung für eine Datenbrille, Datenbrille und Verfahren zum Betreiben einer Projektionsvorrichtung für eine Datenbrille | |
WO2005060270A1 (de) | Autostereoskopische anzeigevorrichtung mit positionsnachführung für mehrere benutzer | |
DE112007003043B4 (de) | Holographische Projektionsvorrichtung und Verfahren zum Beobachten einer rekonstruierten Szene | |
WO2008092852A1 (de) | Holographisches rekonstruktionssystem mit optischen wellennachführmitteln | |
DE102008001644A1 (de) | Vorrichtung zur Darstellung von dreidimensionalen Bildern | |
DE102014008153A1 (de) | Anzeigevorrichtung sowie Verfahren zur augmentierten Darstellung eines Bildobjektes | |
WO2018037077A2 (de) | Holographische anzeigevorrichtung | |
DE102014217180A1 (de) | Bilderzeugungsvorrichtung für ein Head-up-Display, Head-up-Display und Verfahren zum Erzeugen eines Bilds für ein Head-up-Display | |
DE102017211932A1 (de) | Projektionsvorrichtung für eine Datenbrille, Datenbrille sowie Verfahren zum Betreiben einer Projektionsvorrichtung | |
DE102014001710A1 (de) | Vorrichtung und Verfahren zur augmentierten Darstellung eines virtuellen Bildobjektes in einer realen Umgebung | |
WO2018015496A1 (de) | Vorrichtungen zur dateneinspiegelung | |
WO2020229599A1 (de) | Anzeigevorrichtung mit reduziertem energieverbrauch | |
WO2021105130A1 (de) | Augmented-reality-head-up-display | |
WO2011038825A1 (de) | Verfahren zum erzeugen eines stereobilds durch eine projektionseinheit für ein head-up-display und projektionseinheit für ein head-up-display | |
WO2019076963A1 (de) | Anzeigevorrichtung und verfahren zur erzeugung eines grossen sichtfeldes |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20727188 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 20217040763 Country of ref document: KR Kind code of ref document: A |
|
REG | Reference to national code |
Ref country code: DE Ref legal event code: R225 Ref document number: 112020002420 Country of ref document: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20727188 Country of ref document: EP Kind code of ref document: A1 |