GB2555332A

GB2555332A - Holographic systems

Info

Publication number: GB2555332A
Application number: GB1720709.3A
Authority: GB
Inventors: James Cable Adrian; Benet Ballester Raul
Original assignee: Dualitas Ltd
Current assignee: Dualitas Ltd
Priority date: 2010-11-22
Filing date: 2010-11-22
Publication date: 2018-04-25
Anticipated expiration: 2030-11-22
Also published as: WO2012069811A3; WO2012069811A2; GB2485609B; GB201720709D0; GB2555332B; GB2485609A; GB201019749D0

Abstract

A method of generating an image holographically comprises displaying a hologram on a first spatial light modulator (SLM) that has a plurality of light modulating pixels (502, fig.5a); illuminating the first SLM, SLM1, with a beam of coherent light to generate an illumination pattern by diffracting the beam of coherent light; displaying, on a second SLM, SLM2 that has a plurality of light modulating pixels, an intensity image comprising a spatial pattern of intensity modulation; illuminating the second SLM with the illumination pattern; and modulating the illumination pattern with the intensity image to project the image. An apparatus for generating an image holographically is also disclosed that includes a hologram data generator 202 coupled to the first SLM and a controller 202 coupled to the second SLM that is arranged to cause the second SLM to display the intensity image. Also disclosed is replacing one or more of the light modulating pixels with one or more light sensing pixels (504, fig.5a) to sense an intensity of said beam of coherent light to determine an intensity of the generated image.

Description

(54) Title of the Invention: Holographic systems

Abstract Title: A holographic image projection system that employs dual SLM modulation (57) A method of generating an image holographically comprises displaying a hologram on a first spatial light modulator (SLM) that has a plurality of light modulating pixels (502, fig.5a); illuminating the first SLM, SLM1, with a beam of coherent light to generate an illumination pattern by diffracting the beam of coherent light; displaying, on a second SLM, SLM2 that has a plurality of light modulating pixels, an intensity image comprising a spatial pattern of intensity modulation; illuminating the second SLM with the illumination pattern; and modulating the illumination pattern with the intensity image to project the image. An apparatus for generating an image holographically is also disclosed that includes a hologram data generator 202 coupled to the first SLM and a controller 202 coupled to the second SLM that is arranged to cause the second SLM to display the intensity image. Also disclosed is replacing one or more of the light modulating pixels with one or more light sensing pixels (504, fig.5a) to sense an intensity of said beam of coherent light to determine an intensity of the generated image.

IMAGE

DATA

Figure 2

200

1/9

IMAGE

DATA

110

SCREEN FOR \ IMAGE

Figure 1

2/9

LASER POWER CONTROL DATA

IMAGE

200

Figure 2

3/9 image data from source

Input Buffer

Phase modulation data φχγ

4/9

INPUT BUFFER

TO SLM TO SLM (SLM FRAME (SLM FRAME

BUFFER) BUFFER)

Figure 3d

5/9

JS ex.

to σ

ο

Ε ,ο ro

	g-CM
CD	“ο 2>
Ε co	2 ε ο C®
4=	~ 4=
jQ	Ε -Ω
Ζ3	~ =>
ω	ι2 to

Q-g' to ' Ύ3 θ 2 ^Εο 2 Ε 35 ιθ ω

Figure 4a

6/9 >> φ (0 CO Q. 0> φ £ ω ° §

:§ i c '5 Φ .£ ft't: E o

-22 hr 03 φ ,e£ <u 2 o S ,ΐτ.φ ° £2 ULJ m— -d v~ o

c

CD

C co

Φ

Φ σ>

7/9

Figure 5a

I-_)8M-1 I....................................... c “^J .ζιι

602 604b

1 1	_li I	J (_1	U 1_	ί ri	a □ t	7 ZZZi □ EZZ
	□ CZZ3 O	□ □		1 □ L—Ji □ LZJ Q
□ ΐ	□ O	cz	J u	L„J l! 1 L	. 1 U LX
	.............	1 ’	1			.........../
		I	t 1			k
		606	¹ 608	606		606
600	/		Figure 5b

604c

604b

604c

604a ί

604

8/9

IMAGE

DATA

110

PROJECTION LENS ASSEMBLY

L5 \ DIFFRACTION

SCREEN FOR IMAGE

Figure 6

9/9

202

IMAGE

DATA

HOLOGRAM

DATA

PROCESSOR

HIGH SPATIAL FREQUENCY INTENSITY MODULATION DATA

206

LASER POWER CONTROL DATA

-TOW SPATIAL FREQUENCY HOLOGRAM DATA;

V BEAM PROFILE DATA ⁷⁰⁴ DIFFRACTION OPTICS/

SLM2

212

PROJECTION OPTICS

214

D1

700

Figure 7

Holographic Systems

FIELD OF THE INVENTION

This invention relates to apparatus and methods for generating an image holographically, and also to spatial light modulators for use in such techniques.

BACKGROUND TO THE INVENTION

In many applications for which use of a spatial light modulator is efficacious, including the formation of images (by refraction or diffraction) and wavefront correction (in adaptive optics applications such as astronomy and microscopy), it is important to be able to measure of the structure of the light illuminating the spatial light modulator.

In some applications, the light source used to illuminate the SLM can be characterised outside the complete system and then stabilised (through precise temperature and current control) so that it does not change with time, and the recorded illumination data can be used in any processing required to calculate data to display on the SLM. However, in many applications, such as a diffractive imaging system designed for mass production, such control may be uneconomic, insufficiently robust, restrict the operating condition range of the device and/or add unacceptably to its size. In these applications, it is desirable to be able to sense the structure of incident illumination in real time without adding significant additional cost or complexity to the system.

SUMMARY OF THE INVENTION

According to a first aspect there is provided a method of generating an image holographically, the method comprising: displaying a hologram on a first spatial light modulator (SLM) comprising a plurality of light modulating pixels; illuminating the first SLM with a beam of coherent light to generate an illumination pattern by diffracting the beam of coherent light in accordance with the displayed hologram; displaying, on a second spatial light modulator (SLM) comprising a plurality of light modulating pixels, an intensity image comprising a spatial pattern of intensity modulation; illuminating the second SLM with the illumination pattern; and modulating the illumination pattern in accordance with the intensity image to form the image.

According to a second aspect, there is provided an apparatus for generating an image holographically, the apparatus comprising: a first spatial light modulator (SLM) comprising a plurality of light modulating pixels; an illumination source to illuminate the first SLM with a beam of coherent light; a hologram data generator coupled to the first SLM to provide hologram data to the first SLM to display a hologram on the first SLM to generate an illumination pattern by diffracting the beam of coherent light in accordance with the displayed hologram; a second spatial light modulator (SLM) comprising a plurality of light modulating pixels; and a controller coupled to the second SLM and arranged to cause the second SLM to display an intensity image comprising a spatial pattern of intensity modulation, the second SLM arranged to be illuminated with illumination pattern to modulate the illumination pattern in accordance with the intensity image to form the image.

Also disclosed is a method of generating an image holographically, the method comprising: displaying a hologram on a spatial light modulator (SLM) comprising a plurality of light modulating pixels; and illuminating said spatial light modulator with a beam of coherent light to generate said image from said displayed hologram by diffracting said beam of coherent light using said light modulating pixels; and wherein the method further comprises: providing at least one light sensing pixel on said SLM to sense an intensity of said beam of coherent light at a position of said light sensing pixel; and determining an intensity of said generated image by sensing said beam with said light sensing pixel.

In preferred examples of this method, the light sensing is performed by replacing one or more of the light modulating pixels with one or more corresponding light sensing pixels. This is based on the recognition that in a diffractive imaging system such as a holographic imaging system the displayed diffraction pattern or hologram can undergo significant degradation whilst preserving the displayed image, albeit with increased noise. Thus a number of light modulating pixels may be replaced by light sensing pixels without substantially affecting the displayed image, in particular in a system as described later in which multiple temporal sub-frames are averaged to provide the displayed image, thus reducing the noise in the replay field.

One advantage of replacing light modulating pixels with light sensing pixels is that this makes fabrication of the SLM much easier than it would be if, somehow, light sensing pixels had to be fitted in amongst the light modulating pixels. Further, sensing in the SLM plane is desirable because this can significantly reduce the size of an optical assembly in a holographic image projection system because it is not necessary to split the beam to direct a portion of the beam to a light sensor or light sensor array. Sensing in the SLM plane also facilitates accurate measurement of light intensity because this measurement is made on the actual beam generating the hologram.

Also disclosed is a spatial light modulator (SLM) comprising a plurality of light modulating pixels for diffracting a beam of coherent light to generate an image from a hologram displayed on said SLM, wherein at least one of said light modulating pixels is replaced with a light sensing pixel to sense an intensity of said beam of coherent light at a position of said light sensing pixel. In some preferred examples of this SLM the light modulating pixels comprise MEMS (microelectromechanical systems) structures, fabricated on a silicon substrate, in which case a light sensing pixel may be fabricated in COMS (complementary metal oxide semiconductor) technology.

In some preferred examples of this method of generating an image holographically, preferably plurality of light sensing pixels is provided distributed over the plane of the SLM so that a spatial distribution of the intensity of the holographically generated image can be determined. Alternatively such light sensing pixels may be employed to determine a more accurate average beam intensity. In this latter case the sensed (average) beam intensity may be employed in a feedback loop to control the brightness of the holographically generated image, in particular to compensate for brightness variations resulting from selective diffraction of light into illuminated portions of the image — unlike a system which directly displays an image on the SLM, in a holographic image display system light is selectively diffracted into illuminated regions of the image and thus if such a region is small it will appear correspondingly brighter unless brightness compensation is provided dependent on (proportional to) the coverage of the image.

In some preferred examples the light-sensing SLM is employed as the first SLM of two SLMs in the system. The first SLM displays a hologram which generates an image defining an illumination pattern on the second SLM, which displays an intensity image comprising a spatial pattern of intensity modulation, and this in turn is projected, preferably via a diffuser.

In such a system the intensity field which is seen by the eye of a viewer is in effect a product of the intensity field defined by the replay or reconstruction of the hologram on the first SLM, multiplied by the intensity modulation pattern on the second SLM, broadly speaking the second SLM adding the high frequency detail to the image. (The skilled person will appreciate that the eye in fact perceives the squared modulus of the intensity field).

In such a system it will be appreciated that it is important to be able to accurately determine the illumination field of the second SLM, so that this can be adjusted as desired in order to produce the desired intensity field actually seen by the viewer's eye. In practice the beam illuminating the first SLM tends to approximate a Gaussian beam, which has a quasi-low pass filtering effect (recalling that the first SLM effectively performs a Fourier transform). However if the spatial distribution of the intensity of the illumination pattern from the generated image from the first SLM can be determined, the spatial pattern of intensity modulation on the second SLM can be adjusted in order to compensate for the nonuniformities introduced by the spatial beam intensity profile of the beam illuminating the first

SLM. In mathematical terms, the first SLM performs a Fourier transform of a product of the beam spatial amplitude profile and the hologram spatial phase profile (both in two dimensions, x and y). The result of the Fourier transform is a complex reconstruction field, and the intensity perceived by the eye is the squared modulus of this field, modified by the spatial pattern and intensity modulation on the second SLM. Using a plurality of spatially distributed light sensing pixels on the first SLM facilitates accurately determining the intensity field perceived by the eye, that is generating a displayed image which is an accurate representation of that desired.

In preferred examples replacing light modulating pixels of the first SLM by light sensing pixels has little effect on the overall image generation process and substantially reduces the cost and complexity of fabricating the SLM.

Also disclosed is an apparatus for generating an image holographically, the apparatus comprising: a spatial light modulator (SLM) comprising a plurality of light modulating pixels; an illumination source to illuminate said SLM with a beam of coherent light; a hologram data generator coupled to said SLM to provide hologram data to said SLM to display a hologram on said SLM to generate said image by diffracting said beam of coherent light using said light modulating pixels; wherein said SLM includes at least one light sensing pixel to sense an intensity of said beam of coherent light at a position of said light sensing pixel; and wherein the apparatus further comprises a data processor coupled to said at least one light sensing pixel to receive a signal from said at least one light sensing pixel and to determine an intensity of said generated image by sensing said beam with said light sensing pixel.

In preferred examples this apparatus uses an SLM according to an example of the invention as previously described.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other examples of the invention will now be further described, by way of example only, with reference to the accompanying figures in which:

Figure 1 shows a first example of a holographic image projection system for use with an SLM;

Figure 2 shows an improved holographic image projection system for use with an SLM;

Figures 3a to 3d show an example of a holographic image display system without aberration correction illustrating, respectively, a block diagram of a hologram data calculation system, operations performed within the hardware block of the hologram data calculation system, energy spectra of a sample image before and after multiplication by a random phase matrix, and an example of a hologram data calculation system with parallel quantisers for the simultaneous generation of two subframes from real and imaginary components of complex holographic sub-frame data;

Figures 4a and 4b show, respectively, an outline block diagram of an adaptive OSPRtype system, and details of an example implementation of the system;

Figures 5a and 5b show, respectively, a schematic illustration of a 25 x 25 SLM array with active (white) and photodiode (black) pixels according to an embodiment of the invention; and a schematic representation of four pixels from a MEMS piston SLM with one pixel replaced by a photodiode circuit;

Figure 6 shows a first example of a holographic image projection system including an SLM according to an embodiment of the invention; and

Figure 7 shows an improved holographic image projection system including an SLM according to an embodiment of the invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Broadly speaking we will describe a spatial light modulator comprising an array of pixels in which one or more of the pixels is replaced with a photosensitive element, for example a photodiode. Such a spatial light modulator may be employed in wavefront correction or other applications, but is particularly useful for diffractive image formation, and more specifically in a diffractive holographic projector.

To aid understanding of the invention we therefore first describe some preferred implementations of holographic image display systems with which the calibration techniques we describe may be used.

Hologram Generation

We will describe applications of embodiments of the invention to an OSPR-type holographic image display system, and we therefore describe examples of such systems below. However applications of embodiments of the invention are not restricted to this type a hologram generation procedure and may be employed with holographic image display systems employing other types of hologram generation procedure, for example: a Gerchberg-Saxton procedure (R. W. Gerchberg and W. O. Saxton, “A practical algorithm for the determination of phase from image and diffraction plane pictures” Optik 35, 237-246 (1972)) or other procedure. The techniques may also be employed more generally with systems which display a diffraction pattern on an SLM, and in principle in other fields.

Optical system

Figure 1 shows an example optical layout for a first example of a holographic image projection system 100 to project a 2D image onto a screen 110. In the full colour holographic image projector of Figure 1 there are red R, green G, and blue B lasers. The system also includes the following additional elements:

• SLM is the hologram SLM (spatial light modulator).

• L1, L2 and L3 are collimation lenses for the R, G and B lasers respectively (optional, depending upon the laser output).

• M1, M2 and M3 are corresponding dichroic mirrors; they may be implemented as a prism assembly.

• PBS (Polarising Beam Splitter) transmits the incident illumination to the SLM. Diffracted light produced by the SLM - naturally rotated in polarisation by 90 degrees (with a liquid crystal SLM) - is then reflected by the PBS towards L4.

• Mirror M4 folds the optical path.

• Lenses L4 and L5 form an output telescope (demagnifying optics). The output projection angle is proportional to the ratio of the focal length of L4 to that of L5. In embodiments L4 may be encoded into the hologram(s) on the SLM, for example using the techniques we have described in W02007/110668, and/or output lens L5 may be replaced by a group of projection lenses. In embodiments L5 may comprise a wide-angle or fisheye lens, mounted for translation perpendicular to the output optical axis (e.g left-right in Figure 1), to enable configuration of the output optical system as an off-axis system for tabledown projection.

• D1 is a diffuser located at intermediate image plane to reduce speckle. It may comprise a plastic plate, and optionally, may be piezoelectrically-actuated so that it can be moved rapidly in two orthogonal directions to reduce streaking (see our GB2456170). The diffuser increases the etendue of the source, increasing safety and reducing speckle.

A processor 102 acts as a system controller and performs signal processing in either dedicated hardware, or in software, or in a combination of the two, as described further below. Thus processor 102 inputs image data and provides hologram data 104 to the SLM. The different colours are time-multiplexed and the sizes of the replayed images are scaled to match one another, for example by padding a target image for display with zeros (the field size of the displayed image depends upon the pixel size of the SLM not on the number of pixels in the hologram).

In embodiments the SLM may be a liquid crystal device. Alternatively, other SLM technologies to effect phase modulation may be employed, such as a pixellated MEMS-based piston actuator device.

Figure 2 shows an optical architecture for a second example of a holographic image projection system 200, in which like elements to Figure 1 are indicated by like reference numerals. This is described in more detail later.

OSPR

We now describe an example OSPR-type hologram generation procedure, with reference to Figures 3a to 3d: Broadly the SLM is modulated with holographic data approximating a hologram of the image to be displayed. However this holographic data is chosen in a special way, the displayed image being made up of a plurality of temporal sub-frames, each generated by modulating the SLM with a respective sub8 frame hologram, each of which spatially overlaps in the replay field (in embodiments each has the spatial extent of the displayed image).

Each sub-frame when viewed individually would appear relatively noisy because noise is added, for example by phase quantisation by the holographic transform of the image data. However when viewed in rapid succession the replay field images average together in the eye of a viewer to give the impression of a low noise image. The noise in successive temporal subframes may either be pseudo-random (substantially independent) or the noise in a subframe may be dependent on the noise in one or more earlier subframes, with the aim of at least partially cancelling this out, or a combination may be employed. Such a system can provide a visually high quality display even though each sub-frame, were it to be viewed separately, would appear relatively noisy.

The procedure is a method of generating, for each still or video frame I = l_xy, sets of N binary-phase holograms h^<1)... h^fW). In embodiments such sets of holograms may form replay fields that exhibit mutually independent additive noise. An example is shown below:

1. Let =Gexp(y'^/⁾) where is uniformly distributed between 0 and 2π for 1 < η < N/2 and 1 < x,y <m

2. Let g⁽u^y = F~^l where F^_1 represents the two-dimensional inverse Fourier transform operator, for 1 < η < N/2

3. Let = 9ψ’} forl<n<JV/2

4. Let m^(n+NI2) uv = 3(^} for \<n< N/2

5- Let C’ =

+Ufm^>Q^w where Q^w = median ) and 1 < η < N.

Step 1 forms N targets G^ equal to the amplitude of the supplied intensity target l_xy, but with independent identically-distributed (i.i.d.), uniformly-random phase. Step 2 computes the N corresponding full complex Fourier transform holograms g¹/. Steps 3 and 4 compute the real part and imaginary part of the holograms, respectively. Binarisation of each of the real and imaginary parts of the holograms is then performed in step 5: thresholding around the median of ensures equal numbers of -1 and 1 points are present in the holograms, achieving DC balance (by definition) and also minimal reconstruction error. The median value of may be assumed to be zero with minimal effect on perceived image quality.

Figure 3a, from our W02006/134398, shows a block diagram of a hologram data calculation system configured to implement this procedure. The input to the system is preferably image data from a source such as a computer, although other sources are equally applicable. The input data is temporarily stored in one or more input buffer, with control signals for this process being supplied from one or more controller units within the system. The input (and output) buffers preferably comprise dual-port memory such that data may be written into the buffer and read out from the buffer simultaneously. The control signals comprise timing, initialisation and flow-control information and preferably ensure that one or more holographic sub-frames are produced and sent to the SLM per video frame period.

The output from the input comprises an image frame, labelled I, and this becomes the input to a hardware block (although in other embodiments some or all of the processing may be performed in software). The hardware block performs a series of operations on each of the aforementioned image frames, I, and for each one produces one or more holographic sub-frames, h, which are sent to one or more output buffer. The subframes are supplied from the output buffer to a display device, such as a SLM, optionally via a driver chip. Use of a ferroelectric liquid crystal SLM can be advantageous because of its fast switching time. The SLM may be binary phase or multi-phase - binary phase devices can be convenient but binary quantization results in a conjugate image whereas the use of a multi-phase SLM suppresses this.

Figure 3b shows details of the hardware block of Figure 3a; this comprises a set of elements designed to generate one or more holographic sub-frames for each image frame that is supplied to the block. Preferably one image frame, l_xy, is supplied one or more times per video frame period as an input. Each image frame, l_xy, is then used to produce one or more holographic sub-frames by means of a set of operations comprising one or more of: a phase modulation stage, a space-frequency transformation stage and a quantisation stage. In embodiments, a set of Λ/sub-frames, where N is greater than or equal to one, is generated per frame period by means of using either one sequential set of the aforementioned operations, or a several sets of such operations acting in parallel on different sub-frames, or a mixture of these two approaches.

The purpose of the phase-modulation block is to redistribute the energy of the input frame in the spatial-frequency domain, such that improvements in final image quality are obtained after performing later operations. Figure 3c shows an example of how the energy of a sample image is distributed before and after a phase-modulation stage in which a pseudo-random phase distribution is used. It can be seen that modulating an image by such a phase distribution has the effect of redistributing the energy more evenly throughout the spatial-frequency domain. The skilled person will appreciate that there are many ways in which pseudo-random binary-phase modulation data may be generated (for example, a shift register with feedback).

The quantisation block takes complex hologram data, which is produced as the output of the preceding space-frequency transform block, and maps it to a restricted set of values, which correspond to actual modulation levels that can be achieved on a target SLM (the different quantised phase retardation levels may need not have a regular distribution). The number of quantisation levels may be set at two, for example for an SLM producing phase retardations of 0 or π at each pixel, or more for a multi-phase SLM.

In embodiments the quantiser is configured to separately quantise real and imaginary components of the holographic sub-frame data to generate a pair of holographic subframes, each with two (or more) phase-retardation levels, for the output buffer. Figure 3d shows an example of such a system. It can be shown that (depending on the implementation of the procedure) for discretely pixellated fields, the real and imaginary components of the complex holographic sub-frame data are uncorrelated, which is why it is valid to treat the real and imaginary components independently and produce two uncorrelated holographic sub-frames. In other approaches only the real or only the imaginary part may be used.

Adaptive OSPR

In the OSPR approach we have described above subframe holograms are generated independently and thus exhibit independent noise. In control terms, this is an open-loop system. However better results can be obtained if, instead, the generation process for each subframe takes into account the noise generated by the previous subframes - in order to cancel it out, effectively “feeding back” the perceived image formed after, say, n OSPR frames to stage n+1 of the algorithm. In control terms, this is a closed-loop system.

One example of this approach comprises an adaptive OSPR algorithm which uses feedback as follows: each stage n of the algorithm calculates the noise resulting from the previously-generated holograms H₁ to H_n-i, and factors this noise into the generation of the hologram H_n to cancel it out. As a result, it can be shown that noise variance falls as 1//V in comparison to the 1/Λ/ falloff for (non-adaptive) OSPR. An example procedure takes as input a target image T, and a parameter /V specifying the desired number of hologram subframes to produce, and outputs a set of N holograms H, to H_n which, when displayed sequentially at an appropriate rate, form as a far-field image a visual representation of 7which is perceived as high quality.

An optional pre-processing step performs gamma correction to match a CRT display by calculating T(x, y)¹³. Other pre-processing may include colour space conversion and geometry correction (if projecting at an angle). Then at each stage n (of N stages) an array F (zero at the procedure start) keeps track of a “running total” (desired image, plus noise) of the image energy formed by the previous holograms H₁ to H,^ so that the noise may be evaluated and taken into account in the subsequent stage: F(x, y) :=F(x, y) + |F [//^(^,^)]]². A random phase factor φ is added at each stage to each pixel of the target image, and the target image is adjusted to take the noise from the previous stages into account, calculating a scaling factor a to match the intensity of the noisy “running total” energy Fwith the target image energy (T)². The total noise energy from the previous n - 1 stages is given by a F - (n - 1)(7^-)²ΣΓ(χ,γ)⁴ according to the relation a·.x,y ^F(x,y).T'(x,y)² x_ty and therefore the target energy at this stage is given by the difference between the desired target energy at this iteration and the previous noise present in order to cancel that noise out, i.e. (7^-)² - [a F - (n - 1)(7^-)²] = n(7)² + aF. This gives a target amplitude \Tj equal to the square root of this energy value, i.e.

T(x,y):=\ yj2T\x, y)² - aF .exp{j(/>(x, y)} if 2T\x, yY > aF 0 otherwise

At each stage n, /-/represents an intermediate fully-complex hologram formed from the target 7” and is calculated using an inverse Fourier transform operation. It is quantized to binary phase to form the output hologram H_n, i.e.

H(x,yY=F /τΧχ,γΥ fl if Re[H(x,y)]>0 tf„(vy)= . .

-1 otherwise

Figure 4a outlines this method and Figure 4b shows details of an example implementation, as described above.

Thus, broadly speaking, an ADOSPR-type method of generating data for displaying an image (defined by displayed image data, using a plurality of holographically generated temporal subframes displayed sequentially in time such that they are perceived as a single noise-reduced image), comprises generating from the displayed image data holographic data for each subframe such that replay of these gives the appearance of the image, and, when generating holographic data for a subframe, compensating for noise in the displayed image arising from one or more previous subframes of the sequence of holographically generated subframes. In embodiments the compensating comprises determining a noise compensation frame for a subframe; and determining an adjusted version of the displayed image data using the noise compensation frame, prior to generation of holographic data for a subframe. In embodiments the adjusting comprises transforming the previous subframe data from a frequency domain to a spatial domain, and subtracting the transformed data from data derived from the displayed image data. More details, including a hardware implementation, can be found in WO2007/141567 hereby incorporated by reference.

Super-resolution ADOSPR

The eye perceives not the field amplitude Fbut its intensity |F]² and thus manipulation of the phases allows one to influence the pixel values between the sampling grid, using inter-pixel interference to create structure at increased spatial frequencies. Superresolution can be implemented using an ADOSPR-type procedure to generate OSPR hologram sets of resolution Μ x M that form image reproductions at double (in each dimension) the resolution of that of the hologram, i.e. 2M x 2M (2M x M for a binary phase modulator).

One can extend the ADOSPR procedure so that, in addition to feeding forward the reproduction error present at each of the Μ x M sampling points (x, y), the errors present between the sampling points after stage N- 1, i.e. at (x¹/2, y), (x, yVz) and (x¹/2, yVz), are also fed forwards and compensated for when calculating the hologram H_w in stage N. In embodiments this uses a modified inter-pixel Fourier transform operation to evaluate the frequency components every half-sample, instead of every sample. As an alternative to half-sample evaluation, such a transform can be implemented by padding each Mx M hologram up to 2Mx 2Mby embedding it in a matrix of zeros. Taking the Fourier transform of this padded hologram then produces a 2M x 2M field, which can be adjusted for error as desired before taking the inverse Fourier transform to obtain a 2Mx 2M hologram, which is then band-limited to form the next Mx M hologram in the output OSPR set. For further details reference may be made to detailed example calculations for Stage 1, Stage 2 and Stage N in WO 2007/085874 (hereby incorporated by reference).

Brightness control

For a holographic imaging system one can define a parameter p which is a measure of the total noise energy or background noise level independent of the image displayed. For example, for OSPR p = 0.6321, implying that about 30% of the light energy goes into noise as compared with the image. One can further define a parameter c which defines the coverage of an image, that is the energy in the desired image as a proportion of the maximum available energy. The average contrast ratio in a n

holographically replayed image is given by the expression 1+—-—. In general, c(l-p) because different images have different coverages, they will form reproductions with n

different relative brightness, which for an image /_xy varies with where we define c{I} the coverage c{l} as follows:

phase levels > 2

M²

M² binary phase (due to conjugate image)

For a phase hologram with Mx M pixels the normalization factor of Λ/ arises because each pixel, being phase only, has an amplitude (and therefore energy) of 1 so the total replay field energy is /Vf regardless of image content. Since only an approximate value for c{ 1} is needed, if amplitude modulation is employed a similar or the same expression may be used.

In order to compensate for coverage one can, for example, display each subframe of image / for a time proportional to c{I}/ p ; and/or display each subframe of image / for the same length of time t, but illuminate for a time t’< / proportional to c{I}/ p and/or modulate the illumination power proportional to c{I}/ p . In embodiments only a very approximate value for c is employed, say 2 data bits, and a simplified expression may thus be used, for example without the squaring.

Dual modulation architecture

We now describe the improved holographic image projection system architecture 200 of Figure 2. This employs dual SLM modulation - low resolution phase modulation and higher resolution amplitude (intensity) modulation. This can provide substantial improvements in image quality, power consumption and physical size. The primary gain of holographic projection over imaging is one of energy efficiency. Thus the low spatial frequencies of an image can be rendered holographically to maintain efficiency and the high-frequency components can be rendered with an intensity-modulating imaging panel, placed in a plane conjugate to the hologram SLM. Effectively, diffracted light from the hologram SLM device (SLM1) is used to illuminate the imaging SLM device (SLM2). Because the high-frequency components contain relatively little energy, the light blocked by the imaging SLM does not significantly decrease the efficiency of the system, unlike in a conventional imaging system. The hologram SLM is preferably a fast multi-phase device, for example a pixellated MEMS-based piston actuator device.

In Figure 2:

• SLM1 is the hologram SLM (spatial light modulator), for example a 160 χ 160 MEMS or ferroelectric liquid crystal device with pixels of size Δ; it may have physically small lateral dimensions, e.g <1mm.

• L1, L2 and L3 are the collimation lenses.

• M1, M2 and M3 are dichroic mirrors a implemented as prism assembly.

• M4 is a turning beam mirror.

• SLM2 is the imaging SLM and has a resolution at least equal to the target image resolution (e.g. 854 χ 480); it may comprise a LCOS (liquid crystal on silicon) panel.

• Diffraction optics 210 comprises lenses LD1 and LD2, forms an intermediate image plane on the surface of SLM2, and has effective focal length /such that /λ/Δ covers the active area of imaging SLM2. Thus optics 210 perform a spatial Fourier transform to form a far field illumination pattern in the Fourier plane, which illuminates SLM2.

• PBS2 (Polarising Beam Splitter 2) transmits incident light to SLM2, and reflects emergent light into the relay optics 212. PBS2 preferably has a clear aperture at least as large as the active area of SLM2.

• Relay optics 212 relay light to the diffuser D1.

• M5 is a beam turning mirror.

• D1 is a diffuser to reduce speckle, as previously described.

• Projection optics 214 project the object formed on D1 by the relay optics 212, and preferably provide a large throw angle, for example >90°, for angled projection down onto a table top (the design is simplified by the relatively low entendue from the diffuser).

A system controller and hologram data processor 202 performs signal processing in either dedicated hardware, or in software, or in a combination of the two, as described further below. Thus controller 202 inputs image data and provides low spatial frequency hologram data 204 to SLM1 and higher spatial frequency intensity modulation data 206 to SLM2. The controller also provides laser light intensity control data 208 to each of the three lasers.

In an example procedure:

1. The hologram SLM size is Μ χ M pixels.

2. The input image target amplitude, T, is of size P χ P pixels. Amplitude range for the input is between 0 (black) and 1 (white).

3. N ADOSPR subframes are generated.

4. D is a diffraction efficiency boost parameter controlling the trade-off between reconstruction error and diffraction efficiency A value of 1.0 gives theoretically perfect reconstruction; larger values of D increase the optical efficiency at the expense of increasing the noise. (Simulations suggest using a value for D of approximately 1.5).

We assume the illumination incident on the SLM is Gaussian, with the 1/e^{1 2 3} intensity at the edges of the SLM. The steps are:

1. Form a 2M χ 2Mtarget image, R, for hologram generation comprising peak values of blocks of the image. Subdivide the input (P χ P) image Tinto 2M χ 2M blocks, each of size P/2M χ P/2M. Set each pixel of the target Ft to be the peak amplitude of the image data within the corresponding P/2M χ P/2M block of the image.

2. Generate a hologram set H of N holograms of size Μ χ M from R. For example the above ADOSPR algorithm optionally with super-resolution may be employed, optionally iteratively optimising the holograms, for example using a Gerchberg-Saxton procedure.

3. Calculate the reconstruction intensity I of the hologram set, oversampled to P χ P pixels. Sum the intensities of the reconstructions of each of the N holograms and divide the final intensity by Λ/. (An example of reconstruction of an image from hologram data is described above, as part of the ADOSPR procedure).

4. Calculate the intensity image Fto display on the imaging SLM. Set each pixel of Fto the corresponding pixel of the target image intensity 7*. Divide each pixel in Fby the corresponding pixel intensity in I. Let m be the maximum value in the new field F. Then multiply each pixel in Fby Dim. Finally, set every pixel greater than 1 in Fto 1.

5. The relative laser power K used to display this frame is given by m/D.

The image is projected by displaying F on the imaging SLM, while sequentially displaying the N hologram subframes on the hologram SLM. For further details reference may be made to WO2010/007404 (hereby incorporated by reference).

Spatial light modulator with embedded photosensitive pixels

Broadly speaking we will now describe a spatial light modulator comprising an array of optically-active pixels (which may modulate phase or amplitude, or some combination of both) wherein one or more pixels are replaced by photosensitive elements, such as photodiodes. Those skilled in the art will recognise that many other embodiments of photosensitive elements are also possible, including phototransistors, photovoltaic elements, and so forth. Many applications that rely on phase modulation, such as diffractive image formation, are fundamentally robust to a small number of nonmodulating pixels, since the relationship between SLM data and projected image is given by a Fourier transform, and it can be shown that a small number (up to around 1%) of inactive SLM pixels does not produce visible defects in the projected image. Therefore, in these applications, replacement of a small number of SLM active pixels with photodiodes would not significantly impede the primary functionality of the device as a wavefront modulator. Such photodiode elements may be produced using the same standard CMOS fabrication technology as employed for a conventional device’s existing backplane, making this a minimal impact change in terms of cost and process.

In some preferred embodiments, the photodiodes are distributed throughout the array in such a manner that the spatial intensity profile of the beam can be extracted with some accuracy. For example, single-mode laser diode illumination is substantially Gaussian in structure, given by 6 parameters (beam centre given by μ_χ and μ₇, beam variance given by σ/ and σ/, beam covariance given by σ_χ/, peak beam intensity given by /), and so 5 or preferably more (for improved accuracy) appropriately-positioned photodiodes may be employed to determine these parameters. Because photodiode response is fast, the beam profile (and total intensity of incident light) can be determined in real time with minimal computation.

Referring now to Figure 5a, this shows a schematic illustration of a spatial light modulator 500 with, in this example, an array of 25 x 25 square, light modulating pixels 502 in which 25 of these are replaced by photodiodes 504 for the purpose of beam sensing (profiling). It will be appreciated that the techniques we describe are applicable to arrays of different sizes and different pixel shapes, and that whilst for illustration Figure 5a depicts photodiodes which are equally spaced, in other arrangements the photodiode spacing may not be equal. For example the photodiodes may be more densely or less densely packed towards the centre of the array, or otherwise arranged.

We now describe first implementation in the SLM, and then processing of intensity data from the photodiodes to obtain beam profile estimates.

Many architectures of spatial light modulator comprise a standard silicon (CMOS) backplane, with additional material (e.g. liquid crystal or MEMS) on top of the backplane to effect the required optical modulation. Since a photodiode is a standard CMOS component, such an architecture is amenable to replacement of one or more active pixels with photodiodes. Figure 5b for example shows a cross-sectional representation of four pixels from a MEMS piston pixel device 600, comprising MEMS structures 602 deposited on top of a standard 4-metal-layer silicon CMOS backplane 604, with one of the active pixel driver circuits 606 being replaced by a photodiode 608, optionally in combination with an interface circuit.

The MEMS structures 602 are shown schematically and comprise an electrode 602a mechanically coupled to a mirror 602b. The CMOS back plane 604 comprises a silicon substrate 604a and, in the illustrated example, four layers of Inter-Layer Dielectric (ILD) 604b and metal/ILD 604c. It will appreciated that many alternative layouts are possible including other numbers of layers, and it will also be appreciated that a backplane may be fabricated using a technology other than CMOS.

We now give an example of how to utilise captured intensity data from Λ/ photodiodes 5 (numbered from 1 to Λ/) to estimate the Gaussian beam parameters. Suppose that the /th photodiode is located at spatial position (x„ y,) on the SLM, and returns an intensity reading of pj.

• iand define constants A, B, C, D, Sand Fby:

Now let

/A	/*->400	c ©310	^220	‘S’soo	©210	C' \ *· ©200 \	/¹¾ 01 \
			^220	©130	^210	s₁₂₀	Sno Ϊ		©111
.r-<		c	c		C	c	c		c
c		©220	°l30	©040		©030	©020		©021
TA		c	c	c	c	c	c		c
D		©300	^210	©120	^200	©110	©100		©101
		C ©210	^120	C ©030	*^110	©020	©010 i		c i©Oil i
\ J7 /		\ e	c	c·	c	c	c /	\ c /
	^s©200	^aiio	©020		©010	°000^z	^X©001^z

The beam centre (μ_χ, μ₇) is then given by:

AE/

The beam size (σ/, σ/) and beam covariance o_xy (corresponding to the rotation of the beam) are given by:

The total intensity of the beam (including the part that overfills the SLM) is given by /:

440

V4AC — S'³

Those skilled in the art will appreciate that other fitting methods for systems with Gaussian beams may also readily be employed. Further, this general method can be applied to any beam distribution, and is not limited to a beam with a Gaussian profile, using an appropriate fitting algorithm.

Referring now to Figures 6 and 7, these show holographic image projection systems 600, 700 which broadly correspond to those shown in Figures 1 and 2 (like elements are indicated by like reference numerals). Each incorporates a spatial light modulator as described above providing beam profile data over a respective connection 604, 704 to the hologram data processor 102, 202.

The system at Figure 6 may employ this information for example to determine an average beam intensity, which may then be used for image brightness control as previously described.

In the system of Figure 7 again this data may be employed for controlling the relative laser power K(see above), but preferably Gaussian beam parameters determined from the profile data as described above are used to define a profile of beam amplitude over the hologram SLM (SLM 1) so that the illumination pattern on the second SLM (SLM 2) can be more accurately determined from:

I = | F {beam _ amplitude hologram _ phase )| where F is a Fourier transform

This reconstruction intensity can be employed as a more accurate reconstruction intensity at step (3) of the above described dual modulation architecture procedure.

Broadly speaking we have described a system for sensing incident illumination parameters, integrated within the active array of pixels of a spatial light modulator, comprising one or more photosensitive elements such as photodiodes. The pixels may be non-rectangular, of unequal width and height. In embodiments the pixels are configured to modulate the phase of light incident on the spatial light modulator. The SLM may be a liquid crystal SLM, magneto-optic SLM, acousto-optic SLM or an optically-addressed SLM. Phase modulation may be effected using the Kerr or Pockels effect, or the SLM may be, for example, a MEMS SLM. In MEMS SLM embodiments the MEMS pixels may move along an axis substantially perpendicular to the device surface or they may tilt around one or more axes substantially parallel to the device surface; in embodiments movement of the pixels of the SLM may be controlled by electrostatic means.

No doubt many other effective alternatives will occur to the skilled person. It will be understood that the invention is not limited to the described examples and encompasses modifications apparent to those skilled in the art lying within the scope of the claims appended hereto.

Also disclosed is:

Item 1. A method of generating an image holographically, the method comprising: displaying a hologram on a spatial light modulator (SLM) comprising a plurality of light modulating pixels; and illuminating said spatial light modulator with a beam of coherent light to generate said image from said displayed hologram by diffracting said beam of coherent light using said light modulating pixels; and wherein the method further comprises: providing at least one light sensing pixel on said SLM to sense an intensity of said beam of coherent light at a position of said light sensing pixel; and determining an intensity of said generated image by sensing said beam with said light sensing pixel.

Item 2. A method as in item 1 comprising providing a plurality of said light sensing pixels at a plurality of spatially distributed positions on said SLM; and wherein said intensity determining comprises determining a spatial distribution of said intensity of said generated image.

Item 3. A method as in item 2, comprising determining a spatial profile of an amplitude or intensity of said beam from signals from said plurality of said light sensing pixels, wherein said determining of a spatial distribution of said intensity of said generated image comprises performing a space-frequency transform on a product of a phase pattern of said hologram and said determined spatial profile.

Item 4. A method as in item 1, 2 or 3 for projecting a displayed image, wherein a said generated image comprises an illumination pattern, the method further comprising displaying on a second spatial light modulator (SLM) an intensity image comprising a spatial pattern of intensity modulation, and illuminating said second SLM using said illumination pattern from said generated image and modulating said illumination pattern with said intensity image to project said image.

Item 5. A method as in item 4 further comprising adjusting said intensity image displayed on said second SLM responsive to said determined intensity of said generated image comprising said illumination pattern.

Item 6. A method as in any preceding item wherein said providing of said at least one light sensing pixel comprises replacing one or more of said light modulating pixels respectively with one or more of said light sensing pixels.

Item 7. A method as in any preceding item further comprising controlling one or both of said beam of light and a said hologram responsive to said sensed intensity.

Item 8. A method as in item 7 wherein said controlling comprises adjusting an intensity of said beam of light to control a brightness of said holographically generated image.

Item 9. A method as in item 8 wherein said adjusting comprises compensating for brightness variations of said holographically generated image resulting from selective diffraction of said coherent light by said hologram into illuminated portions of said image.

Item 10. A method as in item 7, 8 or 9 comprising replacing a plurality of said light modulating pixels at spatially distributed positions over said spatial light modulator with a respective plurality of said light sensing pixels.

Item 11. A method as in item 10 further comprising determining an average intensity of said beam from signals from said plurality of light sensing pixels, and wherein said controlling comprises adjusting an intensity of said beam of light to control a brightness of said holographically generated image responsive to said determined average intensity.

Item 12. A method as in item 10 or 11 further comprising determining a spatial profile of an amplitude or intensity of said beam from signals from said plurality of light sensing pixels.

Item 13. A method as in item 12 wherein said controlling comprises controlling responsive to said determined spatial profile.

Item 14. Apparatus for generating an image holographically, the apparatus comprising: a spatial light modulator (SLM) comprising a plurality of light modulating pixels; an illumination source to illuminate said SLM with a beam of coherent light; a hologram data generator coupled to said SLM to provide hologram data to said SLM to display a hologram on said SLM to generate said image by diffracting said beam of coherent light using said light modulating pixels; wherein said SLM includes at least one light sensing pixel to sense an intensity of said beam of coherent light at a position of said light sensing pixel; and wherein the apparatus further comprises a data processor coupled to said at least one light sensing pixel to receive a signal from said at least one light sensing pixel and to determine an intensity of said generated image by sensing said beam with said light sensing pixel.

Item 15. An apparatus as in item 14 wherein said at least one light sensing pixel replaces a said light modulating pixel.

Item 16. An apparatus as in item 15 wherein said SLM comprises a plurality of said light sensing pixels at a plurality of spatially distributed positions on said SLM each replacing a corresponding said light modulating pixel, and wherein said data processor receives signals from said plurality of light sensing pixels for determining a spatial distribution of said intensity of said generated image.

Item 17. A spatial light modulator (SLM) comprising a plurality of light modulating pixels for diffracting a beam of coherent light to generate an image from a hologram displayed on said SLM, wherein at least one of said light modulating pixels is replaced with a light sensing pixel to sense an intensity of said beam of coherent light at a position of said light sensing pixel.

Item 18. An SLM as in item 17 wherein said SLM comprises a plurality of said light sensing pixels at a plurality of spatially distributed positions on said SLM each replacing a corresponding said light modulating pixel.

Item 19. An SLM as in item 17 or 18 wherein said light modulating pixels comprise MEMS structures, and wherein a said light sensing pixel comprises a CMOS light sensing device.

Item 20. An optical assembly for a holographic image projection system, comprising the SLM of item 17, 18 or 19 in combination with a laser illumination system to illuminate the SLM, and an optical system between the SLM and an optical output of the assembly to output an image.

Item 21. An Apparatus including an SLM as in item 18 or 19 or an optical assembly as claimed in item 20, and a data processor configured to process signals from said light sensing pixels to determine a spatial intensity profile of a light beam illuminating said SLM.

Item 22. An apparatus as in item 21 wherein said light beam is a coherent light beam, and wherein the apparatus is configured to display a diffraction pattern on said SLM.

Item 23. An apparatus as in item 22 wherein said data processor is configured to determine a spatial distribution of said intensity of said generated image from said spatial intensity profile of said light beam illuminating said SLM.

Claims

CLAIMS:

1. A method of generating an image holographically, the method comprising: displaying a hologram on a first spatial light modulator (SLM) comprising a plurality of light modulating pixels;

illuminating the first SLM with a beam of coherent light to generate an illumination pattern by diffracting the beam of coherent light in accordance with the displayed hologram;

displaying, on a second spatial light modulator (SLM) comprising a plurality of light modulating pixels, an intensity image comprising a spatial pattern of intensity modulation;

illuminating the second SLM with the illumination pattern; and modulating the illumination pattern in accordance with the intensity image to form the image.

2. The method of claim 1, further comprising determining an intensity of the illumination pattern.

3. The method of claim 2, further comprising adjusting the intensity image displayed on the second SLM responsive to the determined intensity ofthe illumination pattern.

4. A method as claimed in claim 2 or claim 3, further comprising controlling the beam of light responsive to said determined intensity.

5. A method as claimed in claim 4 wherein said controlling comprises adjusting an intensity of said beam of light to control a brightness of the image.

6. A method as claimed in claim 5 wherein said adjusting comprises compensating for brightness variations of the image resulting from selective diffraction of the beam of coherent light by the displayed hologram into illuminated portions ofthe image.

7. The method of any preceding claim, further comprising projecting the image.

8. The apparatus of any one of claims 5 to 7, further comprising a projection optic arranged to project the holographically generated image.

21 03 18

Intellectual

Property

Office

Application No: GB1720709.3 Examiner: Mrs Rachel Morgans

8. An apparatus for generating an image holographically, the apparatus comprising:

a first spatial light modulator (SLM) comprising a plurality of light modulating pixels;

an illumination source to illuminate the first SLM with a beam of coherent light;

a hologram data generator coupled to the first SLM to provide hologram data to the first SLM to display a hologram on the first SLM to generate an illumination pattern by diffracting the beam of coherent light in accordance with the displayed hologram;

a second spatial light modulator (SLM) comprising a plurality of light modulating pixels;

5 and a controller coupled to the second SLM and arranged to cause the second SLM to display an intensity image comprising a spatial pattern of intensity modulation, the second SLM arranged to be illuminated with illumination pattern to modulate the illumination pattern in accordance with the intensity image to form the image.

9. The apparatus of claim 8, wherein the controller is configured to determine an intensity ofthe illumination pattern.

10. The apparatus of claim 9, wherein the controller is configured to adjust the intensity 15 image displayed on the second SLM responsive to the determined intensity of the illumination pattern.

11. The apparatus of claim 9 or claim 10, wherein the controller is further configured to control the beam of light responsive to said determined intensity.

12. The apparatus of claim 11, wherein the controller is configured to adjust an intensity of said beam of light to control a brightness of the image.

13. The apparatus of any one of claims 8 to 12, further comprising a projection optic 25 arranged to project the image.

Amendments to the claims have been filed as follows

21 03 18

CLAIMS:

1. A method of generating an image holographically, the method comprising: displaying a hologram on a first spatial light modulator (SLM) comprising a plurality

5 of light modulating pixels;

sensing an intensity of the beam of coherent light;

determining the intensity of the illumination pattern based on the sensed intensity of the 10 beam of coherent light;

displaying, on a second spatial light modulator (SLM) comprising a plurality of light modulating pixels, an intensity image comprising a spatial pattern of intensity modulation, wherein the intensity image displayed on the second SLM is adjusted responsive to the determined intensity of the illumination pattern;

illuminating the second SLM with the illumination pattern; and modulating the illumination pattern in accordance with the intensity image to form the holographically generated image.

2. A method as claimed in claim 1, further comprising controlling the beam of light responsive to said determined intensity.

3. A method as claimed in claim 2 wherein said controlling comprises adjusting an intensity of said beam of light to control a brightness of the holographically generated image.

25 4. The method of any preceding claim, further comprising projecting the holographically generated image.

5. An apparatus for generating an image holographically, the apparatus comprising: a first spatial light modulator (SLM) comprising a plurality of light modulating pixels;

30 an illumination source to illuminate the first SLM with a beam of coherent light;

a second spatial light modulator (SLM) comprising a plurality of light modulating pixels; and a controller coupled to the second SLM and arranged to cause the second SLM to display an intensity image comprising a spatial pattern of intensity modulation, wherein the

5 controller is configured to determine an intensity of the illumination pattern based on a sensed intensity of the beam of coherent light, and wherein the controller is further configured to adjust the intensity image displayed on the second SLM responsive to the determined intensity of the illumination pattern; and wherein the second SLM is arranged to be illuminated with illumination pattern to

10 modulate the illumination pattern in accordance with the intensity image to form the holographically generated image.

6. The apparatus of claim 5, wherein the controller is further configured to control the beam of light responsive to said determined intensity.

7. The apparatus of claim 6, wherein the controller is configured to adjust an intensity of said beam of light to control a brightness of the holographically generated image.