WO2023075291A1

WO2023075291A1 - Mid-air imaging device and method of its operation, projection optical system for mid-air imaging device, interaction mid-air imaging display system, method for operating the interaction mid-air imaging display system

Info

Publication number: WO2023075291A1
Application number: PCT/KR2022/016037
Authority: WO
Inventors: Ilia Valer'evich MALYSHEV; Svetlana Vladimirovna DANILOVA; Dmitriy Evgenyevich PISKUNOV; Stanislav Aleksandrovich Shtykov; Mikhail Vyacheslavovich POPOV; Alexander Alekseyevich ASPIDOV
Original assignee: Samsung Electronics Co., Ltd.
Priority date: 2021-10-25
Filing date: 2022-10-20
Publication date: 2023-05-04

Abstract

The disclosure refers to devices forming mid-air images in free space. The device comprises a projection optical system; a waveguide comprising a first in-coupling diffractive optical element, a second in-coupling diffractive optical element; an out-coupling diffractive optical element; an optical element with a positive optical power. The projection optical system comprises: an image projector, an optical transfer unit, an intermediate image area, a projection unit. When using the invention, the user observes a real image in a large field of view.

Description

MID-AIR IMAGING DEVICE AND METHOD OF ITS OPERATION, PROJECTION OPTICAL SYSTEM FOR MID-AIR IMAGING DEVICE, INTERACTION MID-AIR IMAGING DISPLAY SYSTEM, METHOD FOR OPERATING THE INTERACTION MID-AIR IMAGING DISPLAY SYSTEM

The disclosure refers to the field of optics and is intended to create integrated optical devices, namely, augmented reality devices forming flying images in a free space.

The actively developing field of mobile technologies requires more and more original solutions with high information content and comfort. One idea that requires technical implementation is a mid-air image display. Technology needs: a mid-air image display/large field of view 3D display, capable of displaying without additional diffusing medium. The following requirements are imposed on such a display:

high quality large image;

large field of view so that the image can be seen from multiple points of view or by multiple users;

image rendered in front of the display plane;

no moving parts of the system;

safe and contactless user interface.

Known from the prior art are augmented reality glasses based on a waveguide and an in-coupling and out-coupling diffractive optical element (DOE), the field of view of the image in such systems is rather small, and the brightness of the image strongly depends on the angle of view. Also, the used augmented reality glasses are based on an architecture containing many in-coupling, out-coupling and multiplying DOEs; the field of view of the image in such systems expands. Also mid-air imaging systems for mobile devices are known, in which the field of view of the image is larger compared to the field of view obtained in augmented reality glasses, and, in addition, the image can be seen by several users at the same time. However, a mid-air image itself is small in such systems, and when the image is scaled, it is difficult to achieve good brightness, image uniformity and image quality.

A display system comprising a radially symmetric mirror, a display screen, and a radial array of lenses is known from the prior art (document US 2019/0222828 A1, publication date 18.07.2019). Light from the screen may pass through the radial array and then reflect from the mirror, to create a 360-degree automultiscopic display. The automultiscopic display may display multiple rendered views of a 3D scene, each of which shows the scene from a different virtual camera angle. This known solution has a large size, reduced resolution due to the use of a raster system.

A three-dimensional imaging device based on an array of angle reflectors, which can transfer the plane of the primary display into free space is known from the prior art (document US 20190285904 A1, publication date 09.19.2019). To render the image, the display of the mobile device is positioned at an angle to the array of angle reflectors, the image is formed in free space. The drawback of the known device is that the array of angle reflectors operates with a small field of view, which does not exceed 20 degrees. Also, this known device forms a low-resolution image and has large dimensions. In addition, the known solution does not enlarge a mid-air image.

An apparatus for displaying an image is known from the prior art (document US9933684 B2, publication date 03.04.2018), the apparatus comprising: a first optical substrate comprising at least one waveguide layer configured to propagate light in a first direction, wherein the at least one waveguide layer of the first optical substrate comprises at least one grating lamina configured to extract the light from the first substrate along the first direction; and a second optical substrate comprising at least one waveguide layer configured to propagate the light in a second direction, wherein the at least one waveguide layer of the second optical substrate comprises at least one grating lamina configured to extract light from the second substrate along the second direction; wherein the at least one grating lamina of at least one of the first and second optical substrates comprises an SBG in a passive mode. The drawback of the known solution is low image brightness, limited field of view, no mid-air image is formed.

A system for generating multi-depth image sequence comprising a modulation array is known from the related art (document US 20170199496 A1, publication date 13.07.2017). The modulation array comprising a plurality of light modulators which may shift light incident upon the modulators by a number of degrees. The plurality of light modulators may shift light in concert according to a modulation shift pattern. The modulation shift pattern can be configured to focus incident light to a voxel or to form a 3-D image. One or more modulation shift patterns can be changed to raster one or more image objects in one or more image depth planes. The drawbacks of the known system are that no mid-air image with a positive relief is formed, and different parameters of in-coupling DOEs are required to increase the field of view.

A projection objective and a waveguide display are known from the related art (document WO 2018220265 A1, publication date 06.12.2018). The objective is adapted to project an image from a first plane to a second plane and comprises in order from the second plane a first optical element group (G1), having a positive effective focal length, a second optical element group (G2) placed between the first plane and the first optical element group and having a negative effective focal length, and a third optical element group (G3) placed between the first plane and the second optical element group and having a positive effective focal length. Counting from the second plane, the first refractive surface of the second optical element group is concave towards the second plane and the second refractive surface of the third optical element group is convex towards the first plane. The objective suits well for projecting images to diffractive optical displays. The drawback of the known system is that it is small in size and can be used, for example, for augmented reality glasses, and cannot project large mid-air images.

An optical lens and a head-mounted display device are known from the related art (document US 2020409034 A1, publication date 31.12.2020). The optical lens includes a first lens, a second lens, a third lens, and a fourth lens sequentially arranged from a light exit side to a light incident side. An image generator is disposed at the light incident side. The optical lens is configured to receive an image beam provided by the image generator. The image beam forms a stop at the light exit side. The drawback of the known system is that it is small in size and can be used, for example, for augmented reality glasses, and cannot project large mid-air images.

An eyepiece with increased exit pupil is known from the related art (document RU 2427864 Ρ1, publication date 27.08.2011). The eyepiece contains four components in the direction of the beam path. The first component is a negative meniscus which is turned by a concave surface to the plane of objects. The second component is single plane-convex lens turned by a flat surface to the plane of objects. The third component is a doublet consisting of biconcave and biconvex lenses. The fourth component is a flat-convex lens with a convex surface facing the exit pupil. The drawback is inappropriate parameters that are not suitable for RGB projection systems.

An eyepiece with increased exit pupil is known from the related art (document RU 2652660 Ρ1, publication date 28.04.2018). The eyepiece contains three components, the first of which is a negative meniscus, which is turned by a concave surface to the plane of objects, the second is a single plane-convex lens, and the third is a positive doublet consisting of a biconvex positive lens and a negative meniscus. The drawback is inappropriate parameters that are not suitable for RGB projection systems.

A holographic waveguide optical tracker is known from the related art (document US20190041634A1, publication date 07.02.2019). The tracker comprises: a source of light; at least one waveguide optically coupled to said source; at least one detector waveguide containing a grating lamina for in-coupling and deflecting a first polarization of light reflected from said object into a first waveguide direction and deflecting a second polarization of light reflected from said object into a second waveguide direction; at least one detector optically coupled to said detector waveguide operative to receive light propagating in said first waveguide direction; and at least one detector optically coupled to said detector waveguide operative to receive light propagating in said second waveguide direction. The drawbacks of said device are a lot of additional elements, no mid-air image.

A heads-up display with integrated imaging system is known from the related art (document US 9606354 B2, publication date 28.03.2017). The apparatus comprises a light guide including a proximal end, a distal end, a display positioned near the proximal end, an ocular measurement camera positioned at or near the proximal end to image ocular measurement radiation, a proximal optical element positioned in the light guide near the proximal end and a distal optical element positioned in the light guide near the distal end. The proximal optical element is optically coupled to the display and the distal optical element is optically coupled to the proximal optical element, the ambient input region and an input/output optical element. The drawbacks of said apparatus are large dimensions, a lot of additional elements, no mid-air image.

Thus, there is a need in a mid-air imaging device with a large field of view and without moving parts, with a high quality image, and the device should not need a diffuse screen, in addition, a sufficiently large mid-air image should be formed.

According to the disclosure, a mid-air imaging device comprises:

a projection optical system for directing the image light to a first in-coupling diffractive optical element;

a waveguide on which the diffractive optical elements are located, wherein the waveguide is configured to propagate the light from in-coupling diffractive optical elements to an out-coupling diffractive optical element;

the first in-coupling diffractive optical element located on a first surface of the waveguide for in-coupling the light from the projection optical system into the waveguide and diffraction of the in-coupled light;

a second in-coupling diffractive optical element located on a second surface of the waveguide, opposite to the first surface of the waveguide, opposite the first in-coupling diffractive optical element, wherein the second in-coupling diffractive optical element is located in such a way that the center of the first in-coupling diffractive optical element and the center of the second diffractive optical element are located on one axis, wherein a light reflective coating is applied on the second in-coupling diffractive optical element, wherein the second in-coupling diffractive optical element is for in-coupling the "0" diffraction order light, which moves to the upper surface of the waveguide from the first in-coupling diffractive optical element, the reflective coating is for reflecting the light having one of the diffraction orders, and directing it back to the second in-coupling diffractive optical element;

the out-coupling diffractive optical element located on the first side of the waveguide, wherein the out-coupling diffractive optical element is for out-coupling the image light from the waveguide;

an optical element with a positive optical power, located in vicinity of the second side of the waveguide opposite the out-coupling diffractive optical element, wherein the optical element with a positive optical power is for focusing the light out-coupled from the waveguide through the out-coupling diffractive optical element in the mid-air image plane.

A mid-air imaging device is proposed, comprising the following elements:

a projection optical system;

a waveguide comprising a first in-coupling diffractive optical element, a second in-coupling diffractive optical element;

an out-coupling diffractive optical element;

an optical element with a positive optical power;

wherein

the projection optical system is configured to project the image light onto the first in-coupling diffractive optical element and comprises:

an image projector, an optical transfer unit, an intermediate image area, a projection unit, wherein

the optical transfer unit is configured to display the image from the projector in the intermediate image plane, wherein the optical transfer unit comprises an optical system including a first optical element group having a positive optical power and located at a distance from the projector, and a second optical element group having a negative optical power and located at a distance from the first optical element group,

the projection unit is configured to focus the light at a distance equal to the value of the positive exit pupil relief of the projection optical system onto the first in-coupling diffractive optical element, wherein the projection unit comprises an optical system having a positive optical power;

the waveguide is configured to multiply the light received from the projection optical system, wherein the first in-coupling diffractive optical element is located on the first side of the waveguide so that the light from the projection optical system goes to the waveguide through the first in-coupling diffractive optical element;

the second in-coupling diffractive optical element is located on a second side of the waveguide opposite to the first side of the waveguide, wherein a reflective coating is applied to the second in-coupling diffractive optical element;

the out-coupling diffractive optical element is configured to out-couple the light from the waveguide in the form of a projection field, which is a cone having a divergence angle, the out-coupling diffractive optical element is located on the first side of the waveguide,

wherein, each of the in-coupling diffractive optical elements is located relative to the out-coupling diffractive optical element in such a way that

the angle between the vector of the in-coupling diffractive optical element and the side closest to said vector of the out-coupling diffractive optical element shall be not larger than the difference between the diagonal angle of the out-coupling diffractive optical element and an angle equal to half of the angle of divergence of the projection field;

wherein, the optical element with positive optical power is configured to focus the projection field out-coupled from the out-coupling diffractive optical element in the mid-air image plane.

Wherein, the optical powers of the optical elements in the optical element groups and the distance between the optical elements in the optical element groups are selected in such a way that the size of the exit pupil of the projection optical system and the size of the first in-coupling diffractive optical element are matched. Wherein, the projector consists of three projectors for obtaining a color RGB image. Wherein, additional optical elements are located between the projector and the unit of optical elements to control the light of the projection optical system, namely: the direction of propagation and/or the amplitude and/or phase and/or frequency and/or polarization of the light. Wherein, the additional optical elements can be one of: a prism, an optical cube, a mirror. Wherein, the center of each of the in-coupling diffractive optical elements lays in the range:

Xin≤0.1*W,

Yin≤0.1*H

where W is waveguide measurement along the X axis,

H is waveguide measurement along the Y axis.

The diffraction efficiency of the first in-coupling diffractive optical element is less than the diffraction efficiency of the second in-coupling diffractive optical element.

The device may further comprise a waveguide containing diffractive optical elements identical to the main waveguide, wherein the additional waveguide is superimposed on the main waveguide in such a way that the layout of the in-coupling diffractive optical elements and the out-coupling diffractive optical element of the additional waveguide coincides with the corresponding layout of the in-coupling diffractive optical elements and the out-coupling diffractive optical element, wherein the vectors of the first and second in-coupling diffractive optical elements of the additional waveguide have a direction different from the direction of the vectors of the first and second in-coupling diffractive optical elements of the main waveguide, and the vector of the out-coupling diffractive optical element of the additional waveguide has a direction corresponding to the directions of the vectors of the first and second in-coupling diffractive optical elements of the additional waveguide. Wherein, the out-coupling diffractive optical element is an arbitrary segmented arrangement. Wherein, the device comprises at least an additional waveguide containing diffractive optical elements identical to the main waveguide, wherein at least the additional waveguide and the main waveguide form a stack of waveguides in which the angles of the vectors of the in-coupling diffractive optical elements coincide. Wherein, the device comprises at least an additional waveguide containing diffractive optical elements identical to the diffractive optical elements of the main waveguide, wherein at least the additional waveguide and the main waveguide form a stack of waveguides, wherein each out-coupling diffractive optical element is only one segment of the waveguide surface, wherein the location of the segment of the out-coupling diffractive optical element differs from the location of the segment of the out-coupling diffractive optical element in the subsequent waveguide of the waveguide stack.

According to the disclosure, a method for operating the mid-air imaging device comprises the following steps:

A) the image light from the projection optical system falls on the first in-coupling diffractive optical element, the light is diffracted on the first in-coupling diffractive optical element, as a result of diffraction, the following is formed:

B) "+1^st" diffraction order light which moves along the waveguide towards the out-coupling diffractive optical element, propagating along the waveguide due to the effect of total internal reflection from the waveguide walls;

"-1^st" diffraction order light, which propagates towards the waveguide edge, where it is attenuated;

"0" diffraction order light, which moves to the upper surface of the waveguide and falls on the second in-coupling diffractive optical element, on the surface of which the reflective coating is applied, wherein the "0" diffraction order light, falling on the second in-coupling diffractive optical element, diffracts on the second in-coupling diffractive optical element, as a result of diffraction, the following is formed:

C) the "+1^st" diffraction order light, which moves along the waveguide towards the out-coupling diffractive optical element, propagating along the waveguide due to the effect of total internal reflection from the waveguide walls, and the light having this diffraction order propagates with an offset relative to the "+1^st" diffraction order light, which began to propagate immediately after being formed on the first in-coupling diffractive optical element;

the "-1^st" diffraction order light, which propagates towards the waveguide edge, where it is attenuated;

the "0" diffraction order light, which falls on the reflective coating of the second diffractive optical element, is reflected from the reflective coating of the second diffractive optical element, the "0" diffraction order light falls on the second diffractive optical element, in which it diffracts, as a result of diffraction, the following is formed:

D) the "-1^st" diffraction order light, which propagates along the waveguide due to the effect of total internal reflection from the waveguide walls towards the out-coupling diffractive optical element, wherein the "-1^st" diffraction order light propagates along the same path along which the "+1^st" diffraction order light, formed after diffraction of light from the projection optical system on the first in-coupling diffractive optical element, propagates;

the "+1^st" diffraction order light, which propagates towards the waveguide edge, where it is attenuated;

the "0" diffraction order light, which falls on the first in-coupling diffractive optical element 1a, diffracts on the in-coupling diffractive optical element 1a, as a result of diffraction, the following is formed:

E) the "+1^st" diffraction order light, which propagates towards the waveguide edge, where it is attenuated;

the "-1^st" diffraction order light, which propagates along the waveguide due to the effect of total internal reflection from the waveguide walls towards the out-coupling diffractive optical element;

the "0" diffraction order light, which leaves the waveguide towards the projection optical system;

then steps (B) - (E) are repeated;

wherein, the light falling on the out-coupling diffractive optical element is out-coupled from the waveguide, falls on the optical element with a positive optical power, which focuses the light out-coupled from the out-coupling diffractive optical element in the mid-air image plane.

Also a projection optical system for the mid-air imaging device according to the disclosure is proposed, wherein

the optical power of the projection optical system is greater than or equal to 44 diopters,

the minimum value of the ratio of the distance between the projector and the first optical element of the optical transfer unit to the effective focal length is 1.1,

the maximum value of the ratio of the diameter of the largest optical element in the projection optical system to the effective focal length is 1.5,

the minimum value of the ratio of the exit pupil diameter of the projection optical system to the diameter of the largest optical element is 0.3,

the minimum value of the ratio of the positive exit pupil relief to the effective focal length is 0.8.

An interaction mid-air imaging display system is proposed, the system comprising

the mid-air imaging device according to the disclosure;

a beam splitter;

a detector,

a central control unit associated with the detector and the projector,

an executor; wherein

the beam splitter is configured to direct the light from the projector to the optical transfer unit, and direct the light scattered by the user's interaction with the mid-air image plane from the optical transfer unit to a scattered light detector;

the detector is configured to receive the scattered light that has passed through an optical element with a positive optical power, the out-coupling diffractive optical element, a waveguide, a first in-coupling diffractive optical element, a projection optical system;

the central control unit is configured to receive signals from the detector, process the signals from the detector, and issue a command to the executor, depending on the position of the location of user interaction with the mid-air image plane on the mid-air image plane.

Wherein, the executor is associated with transmitters of light of various ranges. Wherein, the projection optical system comprises an additional light source with a spectral range that is different from the spectral range of the projector, wherein the additional light source is configured to turn on when the projector is turned off. Wherein, the spectral range of said additional light source is invisible to the user. Wherein, the transmitters are ultrasonic transmitters for transmitting an ultrasonic signal to the location of user interaction with the mid-air image plane at the command of the central control unit provided to the executor, wherein the user is capable of feeling the ultrasonic signal at the location of interaction with the mid-air image plane.

A method for operating the interaction mid-air imaging display system according to the disclosure is proposed, the method comprising the steps of:

interacting of a user with a mid-air image plane, due to said interaction, a mid-air image light is scattered, the scattered light falls on the optical element with a positive optical power;

collimating the scattered light by means of an element with a positive optical power;

directing the collimated scattered light to an out-coupling diffractive optical element;

in-coupling the collimated scattered light into a waveguide by means of the out-coupling diffractive optical element;

propagating the in-coupled collimated scattered light in the waveguide;

out-coupling the collimated scattered light into a projection optical system by means of an in-coupling diffractive optical element;

projecting the collimated scattered light onto a detector by means of the projection optical system;

detecting the collimated scattered light by means of the detector and transmitting the signals to the central control unit;

recognizing the fact of user interaction with the mid-air image plane as well as the location of interaction on the mid-air image plane by means of the central control unit;

providing a command by means of the central control unit, depending on the interaction location on the mid-air image plane.

In one embodiment, a command is provided by the central control unit to change the image and/or to provide a sound signal. In one embodiment, a command is provided by means of the central control unit to provide ultrasonic light. Wherein, the projection optical system comprises an additional light source with a spectral range that is different from the spectral range of the projector, wherein the projection optical system projects the light of the additional light source onto the in-coupling diffractive optical element. Wherein, the spectral range of said additional light source is invisible to the user.

The above and other features and advantages of the disclosure are illustrated in the following description, illustrated by the drawings, in which:

Fig. 1A schematically illustrates the location of the in-coupling diffractive element and the path of the beams in the waveguide according to the related art.

Fig.1B schematically illustrates the location of the in-coupling diffractive element and the path of the beams in the waveguide according to the disclosure.

Fig. 1C shows four possible positions of the centers of the in-coupling diffractive elements in the case of a rectangular waveguide.

Fig. 2 illustrates the direction of the vector of the in-coupling diffractive optical element relative to the diagonal of the out-coupling diffractive optical element.

Fig. 3A illustrates the case where the center of the projection field is directed exactly to the opposite corner of the out-coupling aperture of the out-coupling diffractive optical element.

Fig. 3B illustrates the case where the center of the projection field is directed above the diagonal of the out-coupling aperture .

Fig. 4A schematically illustrates the proposed mid-air imaging device.

Fig. 4B illustrates an extreme negative beam of the projection optical system field.

Fig. 5A illustrates embodiments of the disclosure of two waveguides.

Fig. 5B illustrates embodiments of the disclosure of a stack of waveguides, wherein the out-coupling DOE has an arbitrary segmented arrangement.

Fig. 5C illustrates embodiments of the disclosure of a stack of waveguides and a segmented out-coupling aperture.

Fig. 6 schematically illustrates a projection optical system with an intermediate image effect.

Fig. 7 shows an example of an embodiment of the projection optical system.

Fig. 8A, Fig. 8B, and Fig. 8C are variants of location of the projection optical system and the waveguide.

Fig. 9 illustrates the system of interaction of the mid-air image with the user.

The problem to be solved by the disclosure is to produce a mid-air image with an enlarged field of view, wherein the mid-air image should be displayed without additional diffusing medium. It is necessary to produce a high quality enlarged image with a wide field of view so that the image can be seen from multiple viewpoints and/or by multiple users. Wherein, the mid-air imaging device should not have moving parts and should have a safe and contactless user interface.

A device for formation of a flying focused image in a free space, which can be seen with the naked eye in the field of view (Field of View, FoV) at a certain distance from the formed mid-air image, is proposed. The proposed technical solution combines the use of a waveguide, diffractive optical elements (DOE), an optical element with a positive optical power, monocentric projection optics and an array of focusing lenses.

When using the proposed disclosure, the user can observe a real image in space in a large field of view, the convenience of viewing the image by the user at a distance and the convenience of the user's interaction with the image are also improved. The proposed mid-air imaging device displays a mid-air image without additional diffusing medium, thus providing an enlarged high quality image, with a wide field of view, the image can be seen from multiple points of view and/or by multiple users. Moreover, the proposed device has no moving parts and has a safe and contactless user interface.

The proposed disclosure provides improved efficiency of using light directed from the projector, improved image uniformity regardless of the angle at which the user observes the image, excellent image quality, the presence of a system of contactless user interaction with the image.

The proposed disclosure is based on the architecture of a waveguide based on diffractive optical elements with sequential light in-coupling and low light losses during in-coupling. This approach improves the efficiency of light in-coupling and image quality.

The proposed system of interaction of the mid-air image with the user, which does not use additional optical elements, is based on the duplex (symmetry) design of the mid-air image display. The perception of a mid-air image of large dimensions, flying in front of the user, is provided, an enlarged and controlled field of view of the optical system is provided, wherein the resulting image has an increased brightness and uniformity compared to the related art. In the disclosure, the exit pupil size of the projection optical system is matched to the size of the in-coupling diffractive optical element of the waveguide, which results in increased efficiency and improved image quality. The possibility of user interaction with the mid-air image without using any additional elements is provided.

The following terms are used to describe the disclosure:

The optical system's field of view (angular field) is the cone of beams that out-coupled from the optical system and form an image. The center of the field of view corresponds to the center of the image, and the edge of the field of view corresponds to the edge of the largest possible image size.

The exit pupil (or pupil of the optical system) is a paraxial image of the aperture diaphragm in the image space, formed by the subsequent part of the optical system in the direct path of the beams. This term is well-established in optics. The main property of the exit pupil is that at any point there are all fields of the image. By multiplying the exit pupil, its size is thereby increased, without resorting to increasing the longitudinal dimensions of the optical system. Classical optics makes it possible to increase the size of the exit pupil, but at the same time the longitudinal dimensions of the optical system increase; waveguide optics, due to multiple reflection of the beams of rays inside the waveguide, allows this to be done without increasing the longitudinal dimensions.

The diffraction order determines the light diffraction angle. For example, if the diffraction order is "0", then the angle of light diffraction is equal to the angle of incidence of light on the diffractive element (i.e., diffraction does not occur as such). The light having "±1^st" diffraction orders has the smallest possible diffraction angle.

Diffraction efficiency is a property of a diffraction grating, measured as a percentage or a fraction of a unit. Diffraction efficiency is the ratio of the energy contained in one of the diffraction orders relative to the energy incident on the diffraction grating. This term is well established in the related art.

The vector of the diffraction grating is the wave vector of the diffraction grating (diffractive optical element) directed perpendicular to the grating lines and located in the same plane with its working surface. The vector length is

, where Λis the period of the diffraction grating.

Fig. 1A schematically illustrates the location of the in-coupling diffractive optical element 1 and the path of the beams in the waveguide 2 according to the related art, FIG 1B schematically illustrates the location of the in-coupling diffractive optical element 1 and the path of the beams in the waveguide 2 according to the disclosure.

As shown in Fig. 1A, in the related art, the in-coupling diffractive element 1 is located in the middle of one of the sides of the waveguide 2. In the known schemes, the light in-coupled into the waveguide propagates inside the waveguide in directions of ±60 degrees relative to the direction of light in-coupled into the waveguide, that is, the in-coupled light propagates in a waveguide in a cone with a base angle of 120⁰, reducing the working out-coupling aperture of the waveguide. That is, "dead zones" M are formed in the waveguide, where the light does not propagate. Fig. 1A, it can be seen that the zones M, where the light does not propagate, fall on the section of the working out-coupling aperture of the waveguide 2, from where the light could leave the waveguide 2.

According to the proposed disclosure, the mid-air imaging device comprises:

a projection optical system for directing the image light to the first in-coupling diffractive optical element;

an optical element with a positive optical power, located in vicinity of the second side of the waveguide opposite the out-coupling diffractive optical element, wherein the optical element with a positive optical power is for focusing the light out-coupled from the waveguide through the out-coupling diffractive optical element in the air-image plane.

Fig. 1B shows the coordinate system associated with the waveguide. According to the proposed disclosure, the centers of the in-coupling diffractive optical elements 1 are within the range of:

X_in≤0.1*W,

Y_in≤0.1*H

where W is measurement of the waveguide along the X-axis, H is measurement of the waveguide along the Y-axis.

The centers of the in-coupling diffractive optical elements 1 are located on the same axis, on opposite surfaces of the waveguide 2 opposite to each other.

The out-coupling diffractive optical element 3 can have the shape of a polygon, for example, in Fig. 1B the out-coupling diffractive optical element 3 is shown in the form of a pentagon. As it is seen in Fig. 1B, the center of each of the in-coupling diffractive optical elements 1 is equidistant from at least two vertices of the out-coupling diffractive optical element, or the center of each of the in-coupling diffractive optical elements 1 is located in vicinity of at least one vertex of the out-coupling diffractive optical element 3 with apex angle equal to or less than 120 degrees.

The area ratio of the in-coupling diffractive optical element 1 and the out-coupling diffractive optical element 3 is less than 0.001.

The ratio of the effective area of the out-coupling surface, that is, the area from which the image is displayed, to the surface area of the waveguide 2, on which the out-coupling diffractive optical element 1 is applied, is greater than or equal to 0.8.

The diffraction efficiency of the first in-coupling diffractive optical element is less than the diffraction efficiency of the second in-coupling diffractive optical element, which is necessary for equalizing the amount of light in a given (working/useful) diffraction order formed by the first in-coupling diffractive optical element and the second in-coupling diffractive optical element.

Due to the proposed arrangement of the architecture of the diffractive elements on the waveguide 2, the zones in which the light does not propagate do not fall on the section of the waveguide from where the light leaves the waveguide 2.

As it is shown in Fig. 1B, in order to avoid the appearance of inactive regions of the out-coupling diffractive optical element 3, the in-coupling diffractive optical element 1 is placed in the corner of the waveguide 2 so that the vector angle of the out-coupling diffractive optical element is fully placed in the 120-degree propagation cone of the exit pupil, and the inactive regions M are outside the region of the out-coupling diffractive optical element 3. That is, the light in-coupled into the waveguide 2 propagates inside the waveguide in the direction of the diagonal arrow and multiplies in the direction of the side arrows, while the "dead zones" M, where the light does not propagate, do not fall on a section of the waveguide, that is, on the out-coupling diffractive optical element, from where the light out-couples from the waveguide, that is, does not fall on the out-coupling diffractive optical element 3.

This approach allows optimizing the use of the surface of the waveguide 2, that is, to place a large-sized out-coupling diffractive optical element 3 in the waveguide 2.

Fig. 1C shows four possible positions of the centers of the in-coupling diffractive optical elements 1 in the case of a rectangular waveguide.

As it is seen in Fig. 1C, if the vertex of each of the corners of the waveguide 2 is taken for each center of coordinates, then for each of the coordinate systems the distances of the center of the in-coupling diffraction element 1 from the sides of the corner of the waveguide 2 lay within the range:

along the X coordinate: Xin ≤ 0.1 *W, where W is the size of the side of waveguide 2 along the X coordinate,

along the Y coordinate: Yin ≤ 0.1 * H, where H is the size of the side of waveguide 2 along the Y coordinate.

As it is shown in Fig. 2, the in-coupling diffractive optical element shall be located relative to the out-coupling diffractive optical element so that the angle (indicated by three arcs) between the vector of the in-coupling diffractive optical element and the side closest to said vector of the out-coupling diffractive optical element shall be not larger than the difference between the diagonal angle (denoted by one arc) of the out-coupling diffractive optical element and an angle (denoted by two arcs) equal to half of the angle of divergence of the projection system field. In other words, the angle (indicated by three arcs) between the vector of the in-coupling diffractive optical element and the side closest to said vector of the out-coupling diffractive optical element shall be less than the diagonal angle of the out-coupling diffractive optical element (indicated by one arc), at least by half of the angle divergence of the projection field (indicated by two arcs).

If the above condition is not met, then a cropped mid-air image will be generated, as it is explained below. The projection field is the area of light out-coupled from the out-coupling diffractive optical element. The projection field is in the form of a cone having an angle of divergence (that is, the angle at the apex of the cone).

Fig. 3A illustrates cropping of a mid-air image due to incorrect positioning of the in-coupling diffractive optical element when the wrong direction of the in-coupling diffractive optical element vector is selected. It should be noted that the entire plane of the out-coupling diffractive optical element is the out-coupling aperture. As it can be seen in Fig. 3A, in the case where the center (bisector) of the projection field, which has the form of a cone, is directed exactly to the angle of the out-coupling aperture of the out-coupling diffractive optical element (DOE), part of the projection field does not propagate over the entire out-coupling aperture. Fig. 3A shows the projection field (angle indicated by two arcs). It can be seen on Fig. 3A that in the case when the center of the projection field coincides with the diagonal angle of the out-coupling diffractive element (the angle indicated by one arc), a part of the projection field (lower dashed arrow) does not propagate in the projection area of the out-coupling optical element indicated by the dashed line. This means that this part of the field is not removed from this part of the aperture of the out-coupling diffractive optical element. As a result, the part of the mid-air image corresponding to this part of the projection field is cut off.

In Fig. 3B the lower part of the projection field (left dashed arrow) coincides with the diagonal angle of the out-coupling diffractive element (the angle indicated by one arc). The projection field spreads over the entire area of the out-coupling diffractive element, and therefore is out-coupled from its entire area. A full mid-air image is observed. Arrows indicate only the edges (dashed arrows) and the center (solid arrow) of the projection field. In fact, inside the angle of divergence of the projection field (indicated by two arcs), many such beams of light propagate (in the direction indicated by the arrows), each of the arrows is the "future" point of the mid-air image. In the case shown in Fig. 3A, the extreme part of the projection field (left dashed arrow, that is, the plurality of light beams) propagates past the region shown by the dashed rectangle, i.e. goes beyond the border of the out-coupling area of the out-coupling diffractive element, which means that it does not multiply or out-coupled in this dotted area. Thus, the part of the dots of the mid-air image to which the light beams propagating past the area shown by the dotted rectangle correspond do not form an image, and the image is displayed incomplete/cropped.

It should be emphasized that in order to avoid the loss of the projection field, it is necessary that the angle between the vector of the in-coupling diffractive optical element and the side closest to said vector of the out-coupling diffractive optical element be less than the difference in the diagonal angle of the out-coupling diffractive optical element (denoted by one arc) and half of the angle of divergence of the projection system field (indicated by two arcs), as it is proposed in the disclosure. In this case, the center of the projection field P will be directed above the diagonal of the out-coupling aperture and a full mid-air image will be formed. In other words, the angle of the vector of the in-coupling diffractive optical element 1, that is, the direction of light in-coupling into the waveguide, shall be not larger than the diagonal angle of the out-coupling aperture of the out-coupling diffractive optical element minus half of the divergence angle of the projection field.

Fig. 4A shows the proposed device for displaying a mid-air image, comprising a projection optical system 5 and a waveguide with the architecture based on diffractive optical elements according to the disclosure.

The projection optical system 5 out-couples the light in the direction of the in-coupling diffractive optical element 1a of the waveguide 2. The projection optical system 5 comprises projection optics, the minimum optical power of said projection optics is 44 diopters, the minimum value of the ratio of the rear working distance, that is, the distance between the image source (projector)) and the first optical surface of the projection optics, to the effective focal length is 1.1, while the effective focal length is understood as the distance from the optical center of the projection optics to the focal point, and the maximum value of the ratio of the diameter of the largest optical element in the projection optics system to the effective focal length is 1.5. The minimum value of the ratio of the exit pupil diameter to the diameter of the largest optical element is 0.3. The minimum value of the ratio of the positive exit pupil relief to the effective focal length is 0.8.

The aforementioned minimum and maximum values provide an optimal implementation of said projection optics, in particular, efficient light in-coupling without loss. The difference between the minimum and maximum values from those mentioned above will lead to light losses in the system, which will degrade quality of the resulting image.

The architecture (structure) of diffractive optical elements located on the waveguide, i.e. a waveguide with an architecture based on diffractive optical elements, has in-coupling and out-coupling diffractive optical elements with low losses, the architecture includes at least two in-coupling diffractive

optical elements

1a and 1b, located in the corner of the waveguide 2, wherein the in-coupling diffractive

optical elements

1a and 1b are located on opposite surfaces of the waveguide 2.

The diffraction efficiency of the first in-coupling diffractive optical element is less than that of the second in-coupling diffractive optical element (diffraction efficiency is the ratio of the energy of the flux that diffracts into a given diffraction order to the energy of the flux that falls on the diffractive optical element). The second in-coupling diffractive optical element located in the light propagation path is covered with a reflective coating.

The ratio of areas of the in-coupling diffractive optical element and the out-coupling diffractive optical element is less than 0.001;

the ratio of the area of the effective out-coupling of the out-coupling diffractive optical element to the area of the waveguide is greater than or equal to 0.8.

The out-coupling diffractive optical element has such a gradient of diffraction efficiency (a function of the dependence of the diffraction efficiency on the coordinate on the surface of the diffractive optical element), due to which the uniformity of brightness of the output image from the out-coupling aperture is kept.

The out-coupling diffractive optical element out-couples the light from the waveguide towards the optical power optical element, which forms a mid-air image. The shape of the aperture of the optical element with the optical power is matched with the shape of the out-coupling aperture of the out-coupling diffractive optical element.

The mid-air image in space is formed by means of the element 12 with a positive optical power. In ordinary augmented reality glasses, as it is known from the related art, the user sees an image formed by a system consisting of a projector and a waveguide with an architecture based on diffractive optical elements, wherein the image after the out-coupling diffraction grating is formed at infinity, and it can only be seen by the user who is wearing augmented reality glasses. In the proposed device, an optical element with a positive optical power is located on the image path after the out-coupling diffractive optical element. An optical element with a positive optical power collects beams coming from infinity into a focal plane located behind this element. Thus, when an element with a positive power is used, a real image is formed in the plane of a mid-air image, which, due to its location in front of the waveguide and said optical element with a positive optical power, can be observed by multiple users from the side.

The user interaction system includes the mid-air imaging device described above and a detector for detecting the scattered light received as a result of getting of an object such as a user's finger the mid-air in the image plane.

In this case, the light scattered from the object (user's finger) moves from the mid-air image plane back to the projection system through an optical element with a positive optical power, which collimates the scattered light. The collimated light is in-coupled into the waveguide by means of an out-coupling diffractive optical element, propagates along the waveguide and is out-coupled through the in-coupling diffractive optical element in the direction of the projection optical system and then to the detector. The detector comprises at least one light-sensitive area and is used to determine the presence and/or location of an object (user's finger) in the mid-air image plane.

As it is illustrated in Fig. 4A, the light from the projection optical system 5 falls on the first in-coupling diffractive optical element 1a located on the lower part of the waveguide 2. The first in-coupling diffractive optical element transmits the light from the projection optical system 5. The light is diffracted on the first in-coupling diffractive optical element 1a, as a result of diffraction, the following is formed: the "+1^st" diffraction order light, which moves along the waveguide towards the out-coupling diffractive optical element, by propagating along the waveguide due to the effect of total internal reflection from the waveguide walls towards the out-coupling diffractive optical element 3;

the "-1^st" diffraction order light, which propagates towards the waveguide edge, where it is attenuated; the "0" diffraction order light, which moves to the upper surface of the waveguide and falls on the second in-coupling diffractive optical element 1b, on the surface of which a reflective coating 4 is applied (moreover, the direction of movement of the "0" diffraction order light coincides with the movement direction of the initial light, i.e. if the initial light falls on the in-coupling element perpendicularly, then the "0" diffraction order light is perpendicular to the upper surface of the waveguide);

the "0" diffraction order light falls on the second in-coupling diffractive optical element 1b, diffracts on the second in-coupling diffractive optical element 1b, as a result of diffraction the following is formed:

the "+1^st" diffraction order light, which moves along the waveguide towards the out-coupling diffractive optical element, by propagating along the waveguide due to the effect of total internal reflection from the waveguide walls towards the out-coupling diffractive optical element 3, wherein the light having this diffraction order propagates from offset with respect to the "+1" diffraction order light, which began to propagate immediately after being formed on the first in-coupling diffractive optical element 1a;

the "0" diffraction order light, which falls on the reflective coating of the second in-coupling diffractive optical element 1b. After being reflected from the reflective coating of the second in-coupling diffractive optical element 1b, this "0" diffraction order falls on the second in-coupling diffractive optical element 1b, in which it is diffracted, as a result of diffraction the following is formed:

the "-1^st" diffraction order light, which propagates along the waveguide due to the effect of total internal reflection from the waveguide walls towards the out-coupling diffractive optical element 3, wherein the "-1^st" diffraction order light propagates along the same path along which the "+1^st" diffraction order light, generated after diffraction of light from the projection optical system 5 on the first in-coupling diffractive optical element 1a, propagates;

the "0" diffraction order light, which falls on the first in-coupling diffractive optical element 1a, diffracts on the first in-coupling diffractive optical element 1a, as a result of diffraction, the following is formed:

the "-1^st" diffraction order light, which propagates through the waveguide due to the effect of total internal reflection from the waveguide walls towards the out-coupling diffractive optical element 3;

the "0" diffraction order light that is out-coupled from the waveguide 2 towards the projection optical system.

Thus, 2 series of exit pupils are formed, which are displaced relative to each other by half the exit pupil diameter. Thus, in comparison with the related art, the light is twice as often incident on the out-coupling diffractive optical element 3, that is, the light with increased brightness is out-coupled. If we take a waveguide with a certain thickness and an exit pupil of the projection optical system 5, the diameter of which is less than the product of twice the thickness of the waveguide and the tangent of the diffraction angle for the extreme negative beam of the field of the projection optical system (shown in Fig. 4B), then when using systems, known from the related art, in which only the in-coupling diffractive optical element and the out-coupling diffractive optical element, located on one waveguide side, are used, an image with "cellular" uneven illumination is formed at the waveguide output. That is, in the case when the diameter of the exit pupil of the projection system is less than the product of twice the thickness of the waveguide and the tangent of the diffraction angle for the extreme negative beam of the field of the projection system (Fig. 4B), the mid-air image formed by waveguides whose architecture relates to the related art, will have uneven brightness over its area, i.e. will be striped. This phenomenon arises due to the fact that the replicas of the pupil of the projection system, which multiplies in the waveguide, do not intersect/do not overlap when leaving the waveguide, and gaps are formed between them. Due to the architecture of the waveguide described in the disclosure, when the in-coupling diffractive elements are located on both surfaces of the waveguide, even if the condition concerning the waveguide thickness and the exit pupil diameter of the projection system is not met, it is possible to avoid the banding effect (uneven brightness of the flying image). This is due to the fact that in the case of this architecture, the pupil of the projection system is inputted twice, forming two series of multiplying pupils. And due to the fact that the in-coupling elements are located at a distance equal to the waveguide thickness from each other, these series of multiplying pupils are displaced from each other by half the diameter of the pupil. Thus, even if the thickness of the waveguide and the diameter of the exit pupil of the system are inconsistent, the gaps arising during derivation of the replicas of the pupils are filled with replicas of the pupils of the second series.

In the related art, in order to avoid the appearance of uneven illumination of the image, either the width of the exit pupil of the projection system is increased, or the thickness of the waveguide is reduced. However, with an increase in the exit pupil of the projection optical system, it is necessary to increase the size of the in-coupling diffractive optical element, that is, the system becomes cumbersome. If the thickness of the waveguide is decreased, the system loses its rigidity and can bend or collapse under its own weight.

The proposed disclosure solves this problem. In contrast to the known solutions when using the proposed disclosure, the output image illumination is bright and uniform, due to the fact that it is possible to achieve omnidirectional multiplication of light out-coupled from the projection optical system.

Fig. 5 shows embodiments of a waveguide 2 with in-coupling diffractive optical elements and out-coupling diffractive optical elements. Variant (a) illustrates a device consisting of two waveguides 2 superimposed on each other and having the same arrangement of the in-coupling diffractive optical elements 1, but multidirectional vectors of the in-coupling diffractive optical elements, and the same arrangement of the out-coupling diffractive optical elements 3. Thanks to this arrangement, the efficiency of the device is improved, the brightness and uniformity of the image is increased. Variant (b) illustrates a device consisting of a stack of waveguides 2, wherein the out-coupling diffractive optical element 3 is an arbitrary segmented arrangement, and the angles of the vectors of the in-coupling diffractive optical elements coincide. Variant (c) illustrates a device consisting of a stack of waveguides 2 with a segmented out-coupling aperture, that is, each waveguide 2 has an out-coupling diffractive optical element 3 in the form of only one segment of the common surface of the waveguide for out-coupling of light, wherein the location of the segments differs from the location of the segments in the previous waveguide. Thanks to such an arrangement, the brightness of the image is increased, and it is also possible to obtain a large out-coupling aperture of the display using such an arrangement. Since the manufacture of large-sized diffractive elements is extremely complicated, it is difficult to create a large out-coupling aperture from a single diffractive element; therefore, a composite out-coupling aperture is the simplest solution to this problem.

The calculation of the optimal number of in-coupling diffractive elements depending on the maximum possible diffraction efficiency of the element is shown below.

The formula for calculating the optimal number of in-coupling diffractive elements n is as follows:

where I₀ is the fraction of the total incident light that can be in-coupled into the waveguide using the in-coupling DOE, wherein

; η₀is out-coupling efficiency of one diffractive element (before it is divided into several smaller segments), wherein

; m is a number that shows how many times more the out-coupling efficiency of a single diffractive element

shall be obtained by dividing one diffractive element into many, namely n, smaller ones;

is the maximum light out-coupling efficiency associated with the technical limitations of the DOE production. In this case, the elementary light flux from one whole (before dividing into n parts) of the element is equal

, then when dividing into n segments, the elementary light flux from each individual segment with uniform out-coupling looks like this

:, where I₁is the fraction of the total light that is in-coupled into a single segment.

Then the target formula is obtained from the solution of the following system of inequalities, by taking into account that the maximum possible fraction of the total light will be in-coupled into each separate segment I_j, i.e. wherein

:

Then, by setting I₀,m,η_o,η_max, one can calculate the optimal number of elements n by the formula:

.

The calculation allows calculating the minimum required and sufficient number of segments n. The calculation can be applied for cases of a stack of waveguides.

Initially, the out-coupling diffractive optical element is not segmented and is a single element.

The number N_prim of interactions of the primary beams with the out-coupling diffractive optical element on the path of their primary propagation in the waveguide is calculated. Also N_prim is equal to the number of secondary beams that arise as a result of this interaction and propagate in the direction of secondary propagation.

The out-coupling diffractive optical element is divided into segments, the number of which in the direction of the primary beam propagation is equal to N_prim.

The energy of the secondary beams E_cut is calculated by the formula

,

where E_in is the energy of the primary beams, i.e. beams that are formed in the waveguide as a result of light diffraction of the projection system on the in-coupling diffractive optical element and propagate in the direction of primary propagation. E_cut is the energy of the secondary beams. Secondary beams are "cut" from the main beams by segments of the out-coupling diffractive optical element. The diffraction efficiency of each segment of the out-coupling diffractive optical element in the direction of primary beam propagation (DE_{cut_i}) is calculated using the formula:

where i is the segment number, N_sec of interactions of each secondary beam with the out-coupling diffractive optical element along the secondary propagation path in the waveguide. For different secondary beams, N_sec can be different; to obtain uniform light out-coupling from the waveguide, the maximum N_sec is taken.

The out-coupling DOE is divided into segments, the number of which is equal to N_sec, in the direction of the secondary beam propagation.

The total number of light out-couplings from the waveguide is calculated by the formula

E_out is calculated by the formula

E_out is the energy that is "cut off" from the secondary beams by the segments of the out-coupling diffractive optical element and is out-coupled from the waveguide.

The diffraction efficiency of each segment of the out-coupling diffractive optical element (DE_{out_i}) is calculated by the formula

where i is the segment number.

Fig. 6 illustrates in detail the structure of the projection optical system 5 with obtaining an intermediate image 6. The light propagates from the projector 7, passes into the I optical unit, called the optical transfer unit, by means of which the image is transferred from the projector 7 to the intermediate image plane 6. The I unit comprises at least two optical elements. In the direction of light, the first optical element 8, which can be a lens or a mirror, has a positive optical power and is located at a distance A from the projector 7. A is the rear working distance. In this case, the distance A should be large enough to be able to place additional optical elements in this area, such as a beam splitting cube, a prism, a mirror, etc., to control the light from one or more sources, in particular, three RGB projectors. Moreover, the control of projection optical system light is to control the direction of propagation and/or the amplitude and/or phase and/or frequency and/or polarization of light. The second optical element 9 has a negative optical power and is located at a distance B from the first optical element 8. Thanks to the

elements

8 and 9 at a distance C from the second optical element 9, an intermediate image is formed, and then the beams from the intermediate image fall into the projection unit II, where there is a third optical element 10 with a positive optical power, which focuses the beams at a distance D, the distance D being the value of the positive offset of the exit pupil of the projection optical system 5 to the in-coupling diffractive optical element 1 of the waveguide 2.

Due to selection of the distances A, B, C, D and selection of the optical power of each optical element, the size E of the exit pupil from the projection optical system 5 is matched with the size of the in-coupling diffractive optical element 1, thus avoiding light losses. It should be noted that the exit pupil E of the optical projection system 5 is the input pupil for the in-coupling diffractive optical element 1, and will hereinafter be referred to as the input pupil E.

The minimum optical power of the projection optical system is the sum of the optical power of Unit I (Dr) and Unit II (Do) and is at least 44 diopters (D=Do+Dr ≥44 diopters).

The minimum value of the ratio of the rear working distance (that is, the distance between the image source (projector) 7 and the first optical surface of the projection optics) to the effective focal length is 1.1, wherein the effective focal length is understood as the distance from the optical center of the projection optical system 5 to the focus point.

The maximum value of the ratio of the diameter of the largest optical element in the projection optical system 5 to the effective focal length is 1.5 (D_lens/EFL ≤ 1.5).

The minimum value of the ratio of the diameter of the exit pupil E of the projection optical system 5 to the diameter of the largest optical element is 0.3 (EPD/D_lens ≥ 0.3).

The minimum value of the ratio of the positive exit pupil relief to the effective focal length is 0.8 (EPR/EFL ≥ 0.8).

The mentioned above ranges of values are the optimal ranges for obtaining the best mid-air image effect. Using values outside said ranges it is also possible to get a mid-air image, but the image quality will be much worse.

Formation of the intermediate image allows using small lenses (up to 35 mm in diameter) in the projection optical system, and also allows obtaining an increased back distance A between the image source and the first optical surface of the projection optics. Distance A shall be large enough to be able to place additional optical elements in this area, such as a beam splitting cube, prism, mirror, etc., to control the light from one or more sources, in particular, three RGB projectors.

The formation of an intermediate image leads to a positive offset of the exit pupil of the projection optical system 5, that is, a distance appears between the last optical surface of the projection optics, that is, the third optical element 10, and the position of the exit pupil of the projection optical system 5. In this case, for ease of use, the distance of the positive offset shall be at least 5 mm.

The diameter of the exit pupil of the projection system is matched with the size of the in-coupling diffractive optical element 1 of the waveguide 2, which is determined by the period of the diffractive optical element, the thickness of the waveguide and the material from which the waveguide is made.

The possibility of forming an intermediate image leads to high brightness and uniformity of the mid-air image, that is, the efficiency of the device increases, primarily due to the achievement of matching between the exit pupil of the projection system and the in-coupling diffractive optical element, which shall match to achieve maximum efficiency. On the other hand, the efficiency of the device is achieved through the use of a telecentric system. In telecentric systems, the main beams of all off-axis light beams are parallel to the optical axis in object space or in image space, wherein the term "telecentricity" is well established in optics. An important advantage of telecentric optical systems is the almost complete absence of distortion and vignetting, which leads to the most efficient use of all incident light from the source, as well as to improved quality of the resulting image. The increased distance between the image source and the first optical surface of the projection optics leads to improved image quality; in addition, additional optical elements, such as optical cubes or prisms, can be placed in this gap, due to which it is possible to obtain a color image. Due to the positive offset of the exit pupil, a gap is formed in which the in-coupling diffractive optical element of the waveguide can be placed. The in-coupling diffractive optical element is located directly on the waveguide itself, therefore the waveguide is located relative to the projection system so that the positions of the in-coupling diffractive optical element and the exit pupil of the projection system are matched.

In the described optical units I and II, it is possible to use any number of optical elements that form the

optical elements

8, 9, 10, wherein one can use any shape of the lens aperture used in the

optical elements

8, 9, 10, that is, the shape of the lenses can be a traditional round, as well as semicircular, segmental, annular, semicircular, polygonal, etc.

An example of implementation of the projection optical system 5 is shown in Fig. 7. Block I, closest to the light source 7, consists of five lenses, one of which (shown by an arrow) has a negative optical power, and the rest ones have a positive power. Block II is located farther from the light source and consists of one lens with a positive optical power and an additional optical element (namely, a light filter to select light in a certain optical range in order to improve the quality of the resulting image), which is located closest to the exit pupil of the projection system and is not a required component of the system. This system is given only as an example of implementation of the disclosure.

The projector 7 of the projection optical system 5 may include several image sources. To obtain a color RGB image, it is possible to use either three R, G, B projectors with an additional beam splitting cube, or a white light lamp with the additional use of three R, G, B light filters and further transmission/reflection of light from the matrix. In any of these cases, additional space is required between the image source and the first optical surface of the projection system, sufficient to accommodate additional elements for realizing an RGB image.

In the intermediate image plane, a spatial light modulator can be placed, which makes it possible to additionally control the incident light and superimpose visual effects on the resulting image.

The projection optical system 5 and the waveguide 2 with diffractive optical elements can be positioned relative to each other in different ways, as shown in Fig. 8. Figure 8A shows the parallel arrangement of the projection optical system 5 and the waveguide 2 with diffractive optical elements. In this case, to integrate the projection optical system 5 and the waveguide 2 with the in-coupling diffractive optical element 1 and the out-coupling diffractive optical element 3, it is possible to use the prism 11.

In the example of Fig. 8B the prism/mirror 11 is placed directly in the projection optical system 5, for example, in the intermediate image region. This is done to reduce the overall dimensions of the system, since such an arrangement of the prism/mirror 11 makes it possible to control the direction of light propagation and, as a consequence, to arrange the projection optical system 5 parallel to the plane of the waveguide 2, which makes the entire device compact in at least one direction.

The system shown in the example, in Fig. 8C, has the largest overall dimensions, since the projection optical system 5 is located perpendicular to the plane of the waveguide 2. However, it is the simplest to implement, since it does not require additional optical elements, for example, mirrors or prisms, to change the direction of light propagation.

In all these cases, one in-coupling diffractive optical element 1 is shown on the waveguide 2, the position of which coincides with the position of the exit pupil of the projection optical system 5.

In accordance with the disclosure, the user can not only see the mid-air image, but also interact with the mid-air image, that is, the disclosure can be used as an interactive display of the mid-air image. The user interaction system shown in Fig. 9 is designed in such a way that the user can interact with the system and the mid-air image device can immediately or after some time respond to the user's input.

The beams shown in Fig. 9 by solid lines pass from the projection optical system 5 into the waveguide 2 with the in-coupling diffractive optical element 1a, the in-coupling optical diffractive element 1b and the out-coupling diffractive optical element 3, after passing through the waveguide 2 they fall on the optical element 12 with a positive force, with the help of which a mid-air image is formed, wherein Fig. 8 shows the mid-air image plane 13. A mid-air image is formed by the optical element 12 with a positive dioptric power, as described above.

When any object, for example, the user's hand, occurs the image plane 13, the light is scattered by this object, and the beams of the scattered light fall on the element 12 with optical power, become collimated and fall on the out-coupling diffractive optical element 3, which in this case will play the role of an in-coupling diffractive optical element for this scattered light. The out-coupling diffractive optical element 3 in-couples the light into the waveguide 2, then the light moves through the waveguide 2 and is removed from the waveguide 2 using the in-coupling diffractive optical element 1, then the light enters the detector 14, which detects this scattered light. Thus, a user feedback system is implemented.

The system of interaction with the user (interactive mid-air image display) works as follows. The user brings a finger or some object, for example, a pointer (that is, an object with which the user interacts with the mid-air image, to any object depicted in the mid-air image, wherein the light is scattered and falls on the element 12 with optical power. The element 12 with optical power collimates the scattered light, the collimated scattered light is directed to the out-coupling diffractive optical element 3, which in-couples the collimated scattered light into the waveguide 2, where it propagates and is out-coupled through the in-coupling diffractive optical element 1 into the projection optical system 5, which projects this light through the beam splitter 15 onto the detector 14. In this case, the beam splitter 15 is configured to direct the light from the projector 7 to the optical transfer unit, and direct the scattered light received from the user's interaction with the plane 13 of the mid-air image from the optical transfer unit and into the scattered light detector 14.

Thus, a response system is formed without additional light sources, wherein the detector 14 receives the light from the mid-air image area, which the user "touched", since the system is symmetric and there is no distortion in the image projected for viewing and obtained on the detector 14, not entered. That is, the detector 14 detects not only the fact of user's interaction with the mid-air image plane 13, but also the area in the image in which the interaction took place. The data from the detector 14 goes to the central control unit (CPU), which processes the signals received from the detector and determines which part of the interactive display, that is, which part of the plane of the mid-air image the user "pressed" on. In this case, the central control unit is connected to the projector 7 and, basing on the signals received from the detector, the central control unit issues a command to the executor to change the image on the projector. In this case, the user, in response to his interaction with themed-air image plane, receives a modified mid-air image in accordance with the current content and commands (that is, interaction with certain parts of the mid-air image) of the user. The executor can be connected to light transmitters of various ranges.

It is possible to add an additional light source to the user interaction system, which will emit light with a spectral range that differs from the spectral range of light that forms a mid-air image, while additional light, passing through the projection optical system 5, an in-coupling diffractive optical element 1, a waveguide 2, the out-coupling diffractive optical element 3 and the optical element with positive power 12 will be in the mid-air plane. This additional light will allow feedback from the user when, for example, the image projector is turned off. The spectral range of such additional light may not be visible to the user. When the main projector of the mid-air image is turned off, and the user wants to turn it on, the user only has to move his hand in the plane of the mid-air image, while the invisible light will be scattered and hit the detector 14. The system can be configured so that upon receiving a signal from the detector about the reception of scattered additional light will turn on the projector, projecting the mid-air image.

In addition, the system can be configured in such a way that when one "clicks" on a certain part of the mid-air image, that is, upon receiving a signal from the detector 14 about the reception of scattered light in a certain part of the mid-air image, the system will emit a sound signal corresponding to exactly this part of the mid-air image. Also, the user can receive a response from interacting with the mid-air image in the form of image changes.

It is possible to use an array of ultrasonic transmitters in conjunction with the mid-air image display. Due to modulation of the wave phase of each transmitter, it is possible to focus the signal from the ultrasonic transmitters into any region of the mid-air image space. That is, when receiving signals about the user's interaction with the mid-air image plane, the CPU issues a command to the executor to transmit an ultrasonic signal to the area where the user's finger is located. Thus, a reverse tactile response can take place, which will signal the user about "pressing" on any element of the mid-air image, i.e. the user will feel the pressure tactilely, that is, the user will feel the ultrasonic signal with his finger and the user will have the feeling that he actually touched the image,

Thus, the central control unit can be connected to any necessary transmitters, which, upon the command of the central control unit, can transmit the light of the visible range, invisible range, sound, ultrasound to the mid-air image plane, that is, the light of any ranges suitable for interaction with the user.

It should be noted that from a technical point of view, due to the aberrations of the element with a positive optical power 12, the image is spherical, i.e. it is projected onto a sphere of a certain radius. Because of this, it can be considered voluminous, because the point in the center of the mid-air image is closer to the user than the point at the edge of the image. On the other hand, the user does not notice this curvature, for him the image is in the plane.

Thus, thanks to the proposed disclosure, a mid-air image projected in the air is obtained, the image has a large size, a wide viewing angle, that is, the image can be seen from different angles, the brightness of the mid-air image does not depend on the viewing angle of the mid-air image, the user has the ability to interact with the mid-air image by receiving feedback.

The disclosure excludes the physical interaction of the user with any surface to obtain information/response or to turn on and operate with any device. The user simply brings his finger to a place in the air where the mid-air image of a button is visible, and the device, which has an areal control panel, performs an action corresponding to "pressing" this button.

The proposed mid-air imaging device can be used not only as a display of images, but also when creating a holographic user interface when a user interacts, for example, with household appliances such as a refrigerator, hob, TV, air conditioner, intercom, etc. .p., as well as the proposed device can be used in hazardous industries. That is, controls can appear "floating" in space. In this case, using an additional camera, you can detect:

- explicit interaction, which can be expressed by user gestures. Gestures can be symbolic (for example, raising the thumb up), deictic (for example, pointing gestures), iconic (for example, reproducing a specific movement), and pantomimic (for example, using an invisible instrument);

- implicit interaction (proxemics). In this case, proxemics is understood as a sign system in which the space and time of organization of the communication process have a semantic load. For example, if two users with mobile devices with the proposed display form a mid-air volumetric image of the interlocutor (called in this case a hologram and, possibly, not identical to the size of the user's body) using the proposed display, then since the proposed display allows projecting dynamic images, holograms of interlocutors can change over time and the context of communication. In this case, such a modification of the volumetric image can occur both with the participation of the user (using gestures, pressing buttons, voice control, moving the user's eyes, etc.), and without his participation using a pre-programmed reaction (i.e., visual changes 3D images) to the message of the interlocutor. In this case, it should be understood that communication between the interlocutors' holograms can occur without active actions on the part of users, for example, if you use the proposed display with additional sensors for the position and reactions of the user's body.

Using multiple handheld and portable devices can add additional context-sensitive functionality to interact with the generated mid-air images. For example, they can act as a temporary space to transfer information from one hologram to another.

Although the disclosure has been described with some illustrative embodiments, it should be understood that the essence of the disclosure is not limited to these specific embodiments. On the contrary, the essence of the disclosure is intended to include all alternatives, corrections, and equivalents that may be included within the spirit and scope of the claims.

In addition, the disclosure includes all equivalents of the claimed disclosure, even if the claims are changed in the course of consideration.

Claims

A mid-air imaging device comprising:

a projection optical system;

a waveguide comprising a first in-coupling diffractive optical element, a second in-coupling diffractive optical element;

an out-coupling diffractive optical element;

an optical element with a positive optical power;

wherein

the projection optical system is configured to project the image light onto the first in-coupling diffractive optical element and comprises:

an image projector, an optical transfer unit, an intermediate image area, a projection unit, wherein

the optical transfer unit is configured to display the image from the projector in the intermediate image plane, wherein the optical transfer unit comprises an optical system including a first optical element group having a positive optical power and located at a distance from the projector, and a second optical element group having a negative optical power and located at a distance from the first optical element group,

the projection unit is configured to focus the light at a distance equal to the value of the positive exit pupil relief of the projection optical system onto the first in-coupling diffractive optical element, wherein the projection unit comprises an optical system having a positive optical power;

the waveguide is configured to multiply the light received from the projection optical system, wherein the first in-coupling diffractive optical element is located on the first side of the waveguide so that the light from the projection optical system goes to the waveguide through the first in-coupling diffractive optical element;

the second in-coupling diffractive optical element is located on a second side of the waveguide opposite to the first side of the waveguide, wherein a reflective coating is applied to the second in-coupling diffractive optical element;

the out-coupling diffractive optical element is configured to out-couple the light from the waveguide in the form of a projection field, which is a cone having a divergence angle, the out-coupling diffractive optical element is located on the first side of the waveguide,

wherein, each of the in-coupling diffractive optical elements is located relative to the out-coupling diffractive optical element in such a way that

the angle between the vector of the in-coupling diffractive optical element and the side closest to said vector of the out-coupling diffractive optical element shall be not larger than the difference between the diagonal angle of the out-coupling diffractive optical element and an angle equal to half of the angle of divergence of the projection field;

wherein, the optical element with positive optical power is configured to focus the projection field out-coupled from the out-coupling diffractive optical element in the mid-air image plane.
The device according to claim 1, wherein the optical powers of the optical elements in the optical element groups and the distance between the optical elements in the optical element groups are selected in such a way that the size of the exit pupil of the projection optical system and the size of the first in-coupling diffractive optical element are matched.
The device according to any one of claims 1 and 2, wherein the projector consists of three projectors for obtaining a color RGB image.
The device according to any one of claims 1 and 2, wherein additional optical elements are located between the projector and the unit of optical elements to control the light of the projection optical system, namely: the direction of propagation and/or the amplitude and/or phase and/or frequency and/or polarization of the light.
The device according to claim 4, wherein the additional optical elements can be one of: a prism, an optical cube, a mirror.
The device according to any one of claims 1 and 2, wherein the center of each of the in-coupling diffractive optical elements lays in the range:

Xin≤0.1*W,

Yin≤0.1*H

where W is waveguide measurement along the X axis,

H is waveguide measurement along the Y axis.
The device according to any one of claims 1 and 2, wherein the diffraction efficiency of the first in-coupling diffractive optical element is less than the diffraction efficiency of the second in-coupling diffractive optical element.
The device according to any one of claims 1 and 2, comprising an additional waveguide containing diffractive optical elements identical to the main waveguide, wherein the additional waveguide is superimposed on the main waveguide in such a way that the layout of the in-coupling diffractive optical elements and the out-coupling diffractive optical element of the additional waveguide coincides with the corresponding layout of the in-coupling diffractive optical elements and the out-coupling diffractive optical element, wherein the vectors of the first and second in-coupling diffractive optical elements of the additional waveguide have a direction different from the direction of the vectors of the first and second in-coupling diffractive optical elements of the main waveguide, and the vector of the out-coupling diffractive optical element of the additional waveguide has a direction corresponding to the directions of the vectors of the first and second in-coupling diffractive optical elements of the additional waveguide.
The device according to claim 1, wherein the device comprises at least an additional waveguide containing diffractive optical elements identical to the main waveguide, wherein at least the additional waveguide and the main waveguide form a stack of waveguides in which the angles of the vectors of the in-coupling diffractive optical elements coincide.
A method for operating the mid-air imaging device according to any one of claims 1 and 2, the method comprising the following steps:

A) the image light from the projection optical system falls on the first in-coupling diffractive optical element, the light is diffracted on the first in-coupling diffractive optical element, as a result of diffraction, the following is formed:

B) "+1^st" diffraction order light which moves along the waveguide towards the out-coupling diffractive optical element, propagating along the waveguide due to the effect of total internal reflection from the waveguide walls;

"-1^st" diffraction order light, which propagates towards the waveguide edge, where it is attenuated;

"0" diffraction order light, which moves to the upper surface of the waveguide and falls on the second in-coupling diffractive optical element, on the surface of which the reflective coating is applied, wherein the "0" diffraction order light, falling on the second in-coupling diffractive optical element, diffracts on the second in-coupling diffractive optical element, as a result of diffraction, the following is formed:

C) the "+1^st" diffraction order light, which moves along the waveguide towards the out-coupling diffractive optical element, propagating along the waveguide due to the effect of total internal reflection from the waveguide walls, and the light having this diffraction order propagates with an offset relative to the "+1^st" diffraction order light, which began to propagate immediately after being formed on the first in-coupling diffractive optical element;

the "-1^st" diffraction order light, which propagates towards the waveguide edge, where it is attenuated;

the "0" diffraction order light, which falls on the reflective coating of the second diffractive optical element, is reflected from the reflective coating of the second diffractive optical element, the "0" diffraction order light falls on the second diffractive optical element, in which it diffracts, as a result of diffraction, the following is formed:

D) the "-1^st" diffraction order light, which propagates along the waveguide due to the effect of total internal reflection from the waveguide walls towards the out-coupling diffractive optical element, wherein the "-1^st" diffraction order light propagates along the same path along which the "+1^st" diffraction order light, formed after diffraction of light from the projection optical system on the first in-coupling diffractive optical element, propagates;

the "+1^st" diffraction order light, which propagates towards the waveguide edge, where it is attenuated;

the "0" diffraction order light, which falls on the first in-coupling diffractive optical element 1a, diffracts on the in-coupling diffractive optical element 1a, as a result of diffraction, the following is formed:

E) the "+1^st" diffraction order light, which propagates towards the waveguide edge, where it is attenuated;

the "-1^st" diffraction order light, which propagates along the waveguide due to the effect of total internal reflection from the waveguide walls towards the out-coupling diffractive optical element;

the "0" diffraction order light, which leaves the waveguide towards the projection optical system;

then steps (B) - (E) are repeated;

wherein, the light falling on the out-coupling diffractive optical element is out-coupled from the waveguide, falls on the optical element with a positive optical power, which focuses the light out-coupled from the out-coupling diffractive optical element in the mid-air image plane.
A projection optical system for the mid-air imaging device according to claim 1, wherein

the optical power of the projection optical system is greater than or equal to 44 diopters,

the minimum value of the ratio of the distance between the projector and the first optical element of the optical transfer unit to the effective focal length is 1.1,

the maximum value of the ratio of the diameter of the largest optical element in the projection optical system to the effective focal length is 1.5,

the minimum value of the ratio of the exit pupil diameter of the projection optical system to the diameter of the largest optical element is 0.3,

the minimum value of the ratio of the positive exit pupil relief to the effective focal length is 0.8.
An interaction mid-air imaging display system, comprising

the mid-air imaging device according to any one of claims 1 and 2;

a beam splitter;

a detector,

a central control unit associated with the detector and the projector,

an executor; wherein

the beam splitter is configured to direct the light from the projector to the optical transfer unit, and direct the light scattered by the user's interaction with the mid-air image plane from the optical transfer unit to a scattered light detector;

the detector is configured to receive the scattered light that has passed through an optical element with a positive optical power, the out-coupling diffractive optical element, a waveguide, a first in-coupling diffractive optical element, a projection optical system;

the central control unit is configured to receive signals from the detector, process the signals from the detector, and issue a command to the executor, depending on the position of the location of user interaction with the mid-air image plane on the mid-air image plane.
The system according to claim 12, wherein the executor is associated with transmitters of light of various ranges.
The system according to any one of claims 12 and 13, wherein the projection optical system comprises an additional light source with a spectral range that is different from the spectral range of the projector, wherein the additional light source is configured to turn on when the projector is turned off.
The system according to claim 12, wherein the transmitters are ultrasonic transmitters for transmitting an ultrasonic signal to the location of user interaction with the mid-air image plane at the command of the central control unit provided to the executor, wherein the user is capable of feeling the ultrasonic signal at the location of interaction with the mid-air image plane.