GB2499869A

GB2499869A - Image capture system with wavefront encoding and simultaneous representation

Info

Publication number: GB2499869A
Application number: GB1220513.4A
Authority: GB
Inventors: Holger Vogel; Juergen Hartmaier
Original assignee: Cassidian Optronics GmbH
Current assignee: Hensoldt Optronics GmbH
Priority date: 2011-11-16
Filing date: 2012-11-14
Publication date: 2013-09-04
Anticipated expiration: 2032-11-14
Also published as: GB2499869B; GB201220513D0; DE102011118697A1; DE102011118697B4

Abstract

An image capture system has optics 1 and a detector 4 for capturing image information and an image representation device, wherein the detector 4 captures punctiform information and passes this onto the image representation device and in the optics 1 a wavefront encoding takes place. For improved representation a permanent homogenisation of the image captured via blurring by means of wave front encoding and a simultaneous representation by a reversal of the wave front encoding take place. A phase mask or plate 5 may in be used and the system may be a thermal or infra red imaging device and may have a variable focal length.

Description

1

Image capture system

The invention concerns an image capture system with optics and a detector for capture of image information and an image representation device, wherein punctiform image information is captured by the detector and passed to the image representation device and in the optics wavefront encoding takes place.

An image capture system comprises optics for, for example, visible light and/or infrared radiation and a detector for capturing image information. Here the image information is for example captured and processed by thermal imaging devices with a resolution of640 x 512 or 1280 x 1024 or more picture elements and passed to an image representation device such as a monitor, screen, display or similar for visual rendering. At other wavelengths, such as for example visual or ultraviolet considerably higher resolutions are also possible. For the image capture system one or more cameras can be used in the visible and/or thermal range, e.g. long-wave infrared (LWIR), medium wave infrared (MWIR), very long-wave infrared (VLWIR), far infrared (FIR) and also short-wave infrared (SWIR), near-infrared (NIR) and in the UV range. Other image-based representations are also possible.

Such image capture systems are used in many areas of engineering, including as daylight cameras for example in driver assistance systems in automobile engineering, in video monitoring systems or in thermal imaging devices, which are designed for the capture of infrared radiation. Such thermal imaging devices can for example be used for monitoring an area or a building in times of darkness or poor visibility, in order, inter alia, to be able to prevent unauthorised border crossings. In armoured vehicles thermal imaging devices are used to represent an environment for occupation.

It is also known, for example to increase the depth of field of a visible imaging system by means of so-called wavefront encoding. This is achieved in that a known aberration is introduced into the optics of an image capture system which performs a modulation transfer function (MTF) of the optical system in a known range independently of the focal position. For this the modulation transfer function must have no zero points in order to allow a back calculation.

2

With such wave front encoding the depth of field of a captured image can also be increased, as has been suggested by Qioptiq of St Asaph (Great Britain) in the journal Optolines, No 27, 2011, page 32.

With thermal imaging devices, in particular, with for example 640 x 512 or 1280 x 1024 or more picture elements, each picture element or each of the many semiconductor elements, which form the actual detector sensitive to radiation behaves differently, so that even during manufacture and the subsequent measurement, the different detection behaviour in each case of an individual semiconductor element has to be determined and taken into account in the subsequent image processing. In operation, however, such sensors exhibit a marked change in their sensitivity, so that the originally measured values are only valid for a limited period. Different and changing sensitivities or features of each individual detector point are also dependent upon the temperature of the individually observed scene. This means for example that adjacent picture elements provide different values although they are considering the same temperatures in a scene. For a uniform temperature of the scene captured all detector points should really have the same output signal, but because of the different features of each individual semiconductor element this is not the case, on the contrary, different signals will occur here.

In order to achieve a constant image quality therefore, for example in thermal imaging devices, the user, in particular for thermal imaging devices without cooling, also brings about at regular intervals a homogenisation of the thermal imaging device. In principle such a homogenisation can also be performed on other image capture systems. For this, for example, a defocusing lens can be swivelled into the optical beam, in order to reproduce the average scene brightness or scene temperature in particular over the entire detector area. Here complete defocusing takes place. If then the detector or its individual detector points then captures/capture further deviating measured values, then these cannot be caused by the scene itself but are a result of the differing behaviour of an individual detector element. Thus this can be used as an instantaneous correction value, in order, following swivelling away of the defocusing lens, to be taken into account in the further image processing. In the same way a heat source of known temperature can be swivelled into the beam, in order to likewise reproduce a uniform temperature distribution on the detector. With partial defocusing from a certain limiting frequency determined by the degree of defocusing, higher frequency

components are filtered out. The values obtained in this way are used for non-uniformity correction (NUC).

However, it can be considered a disadvantage here that during such homogenisation a complete loss of image takes place, so that a scene cannot be monitored during homogenisation. For markedly changing scenes with different temperatures this must take place, in particular with thermal imaging devices without cooling, relatively often, for example every minute, so that a continuous image representation is not possible. This is particularly undesirable if the thermal imaging device is used for image representation for the driver of a vehicle.

On the basis of this state of the art a person skilled in to the art will be faced with the task of improving an image capture system of the aforementioned type such that the image quality is maintained or increased and a continuous capture of a scene image is made possible.

According to the invention this task is met by an image capture system with the features of claim 1. In addition a corresponding method and the use of a phase mask are indicated.

The central idea of the invention is that in a known image capture system with optics and a detector for punctiform capture of image information a permanent homogenisation of the image captured can take place via blurring by means of wave front encoding. This means that the device for wave-front encoding, which is preferably designed as described in the following, is continuously arranged in the optical beam path of the image detection system. By introducing a known blurring into the optical beam path a blurred image is generated on the detector wherein, however, the nature, extent and scale of the blurring are known and can be taken into account in the correction of the image information. Similarly known are all features of the imaging optical system, which for example consists of a plurality of lenses or lens modules. Since it is a case of a known effect on the reproduction this can be calculated mathematically by an inverse function, in order to obtain a sharp image. It is obvious that this wave front encoding must be a reversible function, in particular without zero points, in order to back calculate or reverse without errors the blurring introduced or the wave front encoding during image processing. The known optical features of the optics and for example of the phase mask described in the following are described overall by a transfer function that as a result is known. This means that with a known scene that is being viewed it is possible to

calculate which image should be visible on the detector. Then the image actually captured by the detector or an internal intermediate image corrected by NUC can be back calculated again by means of a correction function of the transfer function, in order in particular to eliminate a blurring for example by a phase mask and to reproduce a corrected image for the viewer on the image representation device. Here an uninterrupted representation of the scene observed takes place on the image representation device for an observer and at the same time homogenisation of the image captured in the detector is possible.

It is obvious that in order to perform all the functions described here a controller of for example a thermal imaging device is designed with hardware and software in order to perform the corresponding steps of the image processing.

Further configurations of the invention are the subject-matter of subclaims.

In a preferred manner the continuous homogenisation is performed in a thermal imaging device or the optics of a thermal imaging device are continuously equipped with a device for homogenisation or wavefront encoding. In a particularly advantageous manner this can take place in thermal imaging devices without cooling in which previously frequent homogenisation with a total loss of image was necessary.

In a simple manner, for wavefront encoding or for homogenisation, in the optical beam path a phase mask can be introduced, the optical features of which are of course known. Here it is a case for example of an optical transmission grating, for example in quartz glass with defined linear-shaped recesses. Such a material is suitable in particular for optically visible light.

With thermal imaging devices a phase mask in silicon, germanium, zinc selenide, zinc sulphide or other suitable materials with desired infrared optical features can be selected. Since both the features of the actual optics of the image capture system and the features of the phase mask are known, a transfer function of the image capture can be calculated, meaning that on the basis of a real object viewed or a scene it can be determined which internal intermediate image should be reproduced in the image plane on the detector. To this end the transfer function is of course selected such that it has no zero points, in order to be reversible. The actual intermediate image captured by the detector and corrected with NUC is used for homogenisation of the camera, since because of the blurring introduced it can be automatically detected that when there are sharp transitions between adjacent pixels or

picture elements in each case these have differing reception features. Here a scene that is positioned in an object plane is reproduced via the optics and for example the phase mask introduced on an image plane or the detector. The phase mask is preferably designed in such a way that a marked blurring for example of 5 to 50 pixels is achieved in the intermediate image captured. If the image information of neighbouring pixels differs by more than an in particular adjustable limit or a limiting frequency, then the image capture system itself can automatically conclude that the image information above the transferrable limiting frequency cannot actually come from the captured scene itself. Use is made here of the fact that a blurred image cannot contain any high frequencies, since through the blurring virtually only a "fuzzy" image should be visible. For this correction values are determined automatically for the corresponding picture elements in order to filter out those frequencies that are above the transferrable limiting frequency. The values obtained in this way will be used for non uniformity correction (NUC) and a further internal, corrected intermediate image generated. The NUC is by way of example described in EP 1 698 873 Al, paragraph [0037]. This corrected intermediate image then undergoes homogenisation, wherein the correction values obtained in this way are in turn employed for NUC of the next or a later image captured by the detector. This takes place in a looped or iterative, continuously repeating manner, in order to maintain or improve the reproduction quality. In parallel with this or at the same time the NUC corrected internal intermediate image, via a correction function of the transfer function, resulting from the reproduction features of the known optics and the known phase mask, is back calculated to provide a sharp image for representation to a user.

In a further configuration a pure defocusing of the optics of the image capture system takes place which is then through a deconvolution reversed and the image thereby becomes sharp again. However, only a very limited range of blurring can be used here in which a reversal is possible. But the essential advantage of this configuration is that no additional optical element is necessary in the beam path, in particular no phase mask. The other processes such as generation of a corrected intermediate image by NUC, the homogenisation of this and simultaneous representation for a viewer take place as described above.

It is also proposed that the image capture system is equipped with a zoom lens with variable focal length. Here, however, it is necessary for the optical features of the lens to be calculated at various grid points, in order to adapt the inverse function by means of interpolation to the actual focal length.

It is evident to a person skilled in the art that for the image capture system it is not only semiconductors that can be used as sensors, but also any other sensors. Similarly, image capture can take place in principle in any frequency ranges of the electromagnetic spectrum, in particular however in the infrared ranges described above.

An embodiment of the invention is explained in the following in more detail using a drawing. This shows as follows:

Figure 1 a schematic representation of an image capture system;

Figures 2 to 5 various reproductions of a real scene, and Figures 6 to 9 schematic representations.

In the representations of Figures 2 to 5, Figure 2 shows a real scene 3, Figures 3 and 4 each show intermediate images 10, 6 and Figure 5 an example of a corrected image 9 shown on the monitor of a viewer. In figures 6 to 9 corresponding schematic representations are shown, which seek to illustrate the difference between the real scene 3 in Figure 6, the intermediate images 10, 6 in Figures 7 and 8 and the corrected image 9 in Figure 9. For a person skilled in the art it is evident here that the real scene 3 in Figure 6 cannot be reproduced exactly in the corrected image 9 in Figure 9.

The embodiment represented in Figure 1 concerns optics 1 of an image capture system which for ease of presentation is not illustrated here, in particular a thermal imaging device. The optics 1 comprise one or more lenses 2a, 2b, etc. and serve to reproduce a real scene 3, see Figures 2 and 6, on an image plane of a detector 4 with for example 640 x 512 or 1280 x 1024 picture elements. Here each picture element is comprised of a semiconductor element or sensor, the features of which such as sensitivity, temperature profile and suchlike differ from adjacent picture elements and can frequently also change spontaneously.

The image capture system also comprises an image representation device, which is provided with the image information captured by the detector 4. Here it is a case for example of a monitor. It is evident to a person skilled in the art that the image capture system also contains an electronic controller for image processing and operational controls such as for a zoom lens or similar. The optical features of the optics 1, thus for example the focal length,

magnification and suchlike can be calculated from the shapes and distances of the lenses 2a, 2b, etc.

In a first embodiment, in the optics 1 a phase mask 5 is continuously arranged for wavefront encoding, the optical features of which are likewise known. For example, with the phase mask 5 a blurring is introduced into the optics 1, such that a real scene 3 appears in the image plane of the detector 4 as a captured internal intermediate image 10, see Figures 3 and 7. Here the known optical features of the optics 1 and of the phase mask 5 combine to form a transfer function 7, which for example can be calculated mathematically. Here the blurring is in particular selected such that in the internal intermediate image 10 captured a blurring of for example 5 to 50 pixels is achieved. This leads to an equalisation of the adjacent image information or a homogenisation of the image temperature. Alternatively to the wavefront encoding by means of phase mask 5 a defocusing can also be performed.

If now, however, in the detector 4 for example on two adjacent pixels a considerable difference in the respective signals received or the signal strength is detected, then the image capture system can itself automatically conclude that this information was brought about by a change in the behaviour of the semiconductor element and not by actual image information. Therefore the image information of the virtually captured intermediate image 10 is converted by a NUC (see step 11 above) into a correct internal intermediate image 6, see Figures 4 and 8. This internal intermediate image 6 now undergoes homogenisation as indicated by the loop 12. This means that the detected differences between the picture elements are used in order to obtain corrected offset values for the NUC. The NUC corrected in this way by the homogenisation 12 is then in step 11 in turn used so that the next or a later internal intermediate image 10 captured by the detector 4 undergoes improved NUC so that the image quality of the corrected internal intermediate image 6 is maintained or improved.

At the same time as the homogenisation 12 and to improve the NUC in step 11 and especially without interruption the corrected internal intermediate image 6, however, through a reversal of the transfer function 7 or a correction function 8 is converted and represented continuously on the image representation device of a viewer as a corrected image 9, see Figure 5 and 9. It is obvious that with the reproduction of a real scene 3 the corrected images 9 displayed reproduce the actual passage of time, thus for example like a person passing through a scene

8

Accordingly, by an inversion of the transfer function 7 or a correction function 8, the errors in the optics 2 are eliminated by the phase mask 5, so that ultimately on an image representation device such as a monitor a corrected image 9 is represented.

Here the phase mask 5 remains continuously in the optical path of the optics 1 and despite the elimination of the erroneous measurement behaviour of an individual pixel there is no interruption in the image representation so that for example an area can be continually monitored with a thermal imaging camera. Similarly in a vehicle a driver can be shown an environment without interruption.

9

List of references:

1 Optics

2 Lens

3 Scene

4 Detector

5 Phase mask

6 Corrected internal intermediate image

7 Transfer function

8 Correction function of the transfer function

9 Corrected image

10 Captured internal intermediate image

11 NUC

12 Homogenisation

10

Claims

Claims:

1. Image capture system with optics (1) and a detector (4) for capture of image information and with an image representation device, wherein punctiform image information is captured by the detector (4) and passed to the image representation device and in the optics (1) wavefront encoding takes place, characterised in that a permanent homogenisation of the image captured can take place via blurring by means of wave front encoding and simultaneously representation can take place by reversal of the wavefront encoding.

2. Image capture system according to claim 1, characterised in that the image capture system is a thermal imaging device.

3. Image capture system according to claim 1 or 2, characterised in that a phase mask (5) is arranged continuously in the optics (1).

4. Image capture system according to claim 1 or 2, characterised in that a pure defocusing can be performed.

5. Image capture system according to one of claims 1 to 4, characterised in that the optics (1) have a variable focal length.

6. Method for operating an image capture system with optics (1) and a detector (4) for capturing image information and with an image representation device, wherein punctiform image information is captured by the detector (4) and passed to the image representation device and in the optics (1) wavefront encoding takes place, characterised in that a permanent homogenisation of the image captured takes place via blurring by means of wave front encoding and simultaneous representation takes place by a reversal of the wave front encoding.

7. Method according to claim 6, characterised in that the homogenisation of a thermal imaging device takes place.

11

8. Method according to claim 6 or 7, characterised in that a phase mask (5) permanently arranged in the optics (1) is used.

9. Method according to claim 6 or 7, characterised in that optics (1) are constantly defocused.

10. Method according to one of claims 6 to 9, characterised in that optics (1) with variable focal length are used.

11. Use of a phase mask (5) in a thermal imaging device for continuous homogenisation of a captured image (6).