US20200264415A1 - Method for determining a height position of an object - Google Patents
Method for determining a height position of an object Download PDFInfo
- Publication number
- US20200264415A1 US20200264415A1 US16/061,639 US201616061639A US2020264415A1 US 20200264415 A1 US20200264415 A1 US 20200264415A1 US 201616061639 A US201616061639 A US 201616061639A US 2020264415 A1 US2020264415 A1 US 2020264415A1
- Authority
- US
- United States
- Prior art keywords
- intensity
- location
- imaging
- calibration
- wide field
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/008—Details of detection or image processing, including general computer control
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/02—Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness
- G01B11/06—Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness for measuring thickness ; e.g. of sheet material
- G01B11/0608—Height gauges
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0032—Optical details of illumination, e.g. light-sources, pinholes, beam splitters, slits, fibers
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0052—Optical details of the image generation
- G02B21/0072—Optical details of the image generation details concerning resolution or correction, including general design of CSOM objectives
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
-
- H04N5/23238—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10056—Microscopic image
Definitions
- the invention relates to a method for determining a height position of an object at a lateral first location of the object by using a microscope, which images the object with a point spread function along a z-direction that coincides with the height direction.
- the invention further relates to a microscope for producing a partially confocal image and a wide field image of an object, wherein the microscope comprises a detector device, a pinhole device for partially confocal imaging and for imaging in wide field, a focusing device which is adapted for setting a z-position of a focus of the partially confocal image, and a control device for controlling the pinhole device, which control device is connected to the detector device.
- Methods are known from the prior art, which can be used to determine the topography of an object using an optical microscope.
- confocal microscopes are used herefor.
- the focus of a partially confocal image is shifted along a z-direction, i.e. along a direction of the height extension of the topography, and a partially confocal image is recorded at each z-position.
- the partially confocal imaging is typically provided such that the image comprises a purely confocal component and a component of a wide field.
- the intensity is determined for each of said images in the z-direction, which are collectively also referred to as a stack, and an intensity profile in the z-direction is calculated.
- the maximum intensity can be determined, the z-coordinate of which coincides with the height position of the object at the measured point.
- the above-described method is repeated at various locations. For determining the maximum intensity, it is typically necessary to carry out a multiplicity of measurements along the z-direction in order to obtain a proper accuracy for the intensity maximum, and, thus, the elevation of the object. For this reason, the method can require with a great expenditure of time.
- US 2004/0008515 A1 relates to an improvement of fluorescence illumination for three-dimensionally resolving microscopy.
- the publication Hiraoka Y. et al., Biophys. J., Vol. 57, February 1990, pp. 325-333 explains the ascertainment of the three-dimensional imaging properties in an optical microscope.
- McNally J. et al., SPIE, Vol. 2984, 1997, pp. 52-63 compare several 3D microscopy methods using an accurately known test object.
- the method is preferably used to determine a topography of a laterally extending object.
- the elevation at a plurality of first locations can be determined, wherein the first locations have a lateral distance from one another.
- the first locations consequently relate in particular to the lateral extent of the object.
- the first locations are preferably spaced apart perpendicularly to the z-direction.
- the method is preferably used in a scanning manner, wherein the claimed method or parts of the method is/are performed at each scanning point, which can correspond to the first location.
- the topography of the object can be determined from the height positions of the individual first locations.
- the microscope is in particular adapted to image the object in a partially confocal manner.
- Partially confocal in this context is intended to mean that the pinhole limits imaging in the z-direction, but not in diffraction-limited manner. Then, the size of the pinhole determines the region of the first location in the z-direction that is imaged by the partially confocal imaging. As is generally known, the resolution in the z-direction increases as the size of the pinhole decreases.
- Confocal imaging in this context is intended to mean that the pinhole has an extent that corresponds to the diffraction limit of the imaging.
- Partially confocal imaging in particular comprises components of a purely confocal image and of the wide field image.
- the pinhole is in particular larger than in confocal imaging.
- Partially confocal imaging serves in particular for capturing a so-called composite image.
- the microscope determines the depth of field range in the z-direction.
- the depth of field range is thus a property of the microscope which can be determined in advance by known methods.
- the object in particular has a topography whose height variation is in the nanometer to millimeter range.
- the object can be any sample which can be optically imaged.
- the object can in particular be a biological sample.
- the partially confocal image defines the depth of field range.
- the depth of field range depends on the size of the pinhole, wherein the depth of field range in the z-direction increases as the size of the pinhole increases. Consequently, confocal imaging has a smaller depth of field range in the z-direction than partially confocal imaging. Further, the depth of field range of a partially confocal imaging is smaller than the depth of field range of a wide field imaging. If the size of a confocally imaging pinhole is decreased further, the depth of field range does not change, but only the intensity of the confocal image is reduced.
- a wide field imaging is intended to mean in particular an imaging of the object which has no, or a low, spatial resolution in the z-direction. This can be done by providing no pinhole, or a pinhole having a large diameter, in an imaging beam path which represents the beam path of the light, from emission or reflection at the first location to a detector device of the microscope. After wide field imaging, the intensity thereof is determined, which corresponds to the wide field intensity.
- Wide field imaging can be performed at the first location or at any other location of the object and preferably serves for capturing the optical properties, such as for example the reflectance of the sample.
- the wide field intensity is preferably determined only once, such that, when the method is repeated, the wide field intensity of a previous measurement is utilized. Determining the wide field intensity only once is suitable in particular if the object is one whose optical properties, in particular the reflectance, are constant or approximately constant or change by less than 10%, 20% or 50% over the extent of the object. Consequently, if the method illustrated here is repeated, preferably wide field imaging of the first location is performed only once.
- the maximum intensity expected at the first location is calculated, which corresponds to the maximum intensity that was obtained in partially confocal imaging or confocal imaging. This calculation is based on the finding that the maximum intensity in confocal or partially confocal imaging is proportional to the wide field intensity. This relationship corresponds to the scaling factor, which is preferably determined in advance for the microscope, and in particular on the object.
- the calculation of the expected maximum intensity at the first location can be performed only once if the determination of the wide field intensity is likewise performed only once. For this reason, the determination of the expected maximum intensity must not be performed again upon repetition of the method.
- the partially confocal imaging of the object at the first location is performed upon repetition of the method in particular at each of the first locations.
- the focus of this image is provided at a z measurement position which is situated in the depth of field range of the partially confocal imaging of the microscope.
- the z measurement position is situated in a depth of field range which is established by confocal imaging.
- the choice of the z measurement position is preferably random, or the same z measurement position is used for the partially confocal imaging at each of the first locations. Establishing the z measurement position such that it is situated in the depth of field range of the partially confocal image can be effected by trial and error or manually.
- the partially confocal intensity determined from the partially confocal imaging can preferably be used to discover whether the z measurement position is situated in the depth of field range. If this is not the case, this method step can be repeated until it is certain that the z measurement position is located in the depth of field range of the partially confocal imaging.
- the partially confocal imaging is preferably performed by providing a pinhole in the imaging beam path.
- the partially confocal imaging consequently has a better resolution in the z-direction as compared to the wide field image. Due to the fact that the point spread function for the microscope can be determined, in particular the precise profile of the point spread function is known, for example the deviation of the point spread function from a Gaussian curve and the standard width. Purely confocal imaging preferably images the object with the point spread function.
- the maximum intensity of the point spread function is referred to in particular as the intensity corresponding to the point spread function.
- the z measurement position in the z-direction which coincides with the maximum intensity of the point spread function can be calculated based on the assumption that the intensity of a partially confocal imaging at the first location is equal to the sum of the intensity of a confocal imaging at the first location and the wide field intensity multiplied by a linking factor.
- the linking factor is determined preferably before performing the method, for example for the observed object or generally for the microscope. From this relationship it is possible to calculate from the measured variables, specifically the partially confocal intensity and the wide field intensity, the intensity corresponding to the point spread function at the first location. This method step is preferably performed during the topography ascertainment for each first location.
- the desired z-coordinate is the position of the focus in the z-direction at which the intensity of the point spread function is maximum. It is calculated using the information relating to the form of the point spread function and two points which are known of the point spread function. This is possible since the point spread function in the z-direction coincides with the intensity profile of a confocal image. The calculation of the position at which the point spread function is maximum can be performed for example by fitting the point spread function to the two known points.
- One of these two known points corresponds to the intensity of the partially confocal imaging corresponding to the point spread function and the other corresponds to the expected maximum intensity which was determined previously, as described above.
- the z-coordinate of the focus at which the intensity profile of the point spread function is maximum is the height positon of the object at the first location.
- an ambiguity may arise, but can be resolved as described below.
- this problem can be solved by way of the determined z-coordinate of the focus being compared to previously determined z-coordinates, for example by using the z-coordinate of the two determined z-coordinates that is closest to the previous measurement as the true position. It is moreover possible for further information that is known of the object to be used to resolve the ambiguity.
- the intensity of the partially confocal imaging comprises components of a wide field image.
- An advantage of this embodiment is that the intensity profile of the partially confocal imaging depending on the z-position has a larger depth of field range as compared to a purely confocal image, i.e. an image in which the pinhole has an opening that corresponds to the diffraction limit.
- the depth of field range in this embodiment is consequently larger and is easier to set. Moreover, greater height differences can be determined.
- the scaling factor is determined by performing the following steps: producing a z-stack by repeatedly shifting the focus positon in the z-direction and partially confocal imaging a lateral first calibration location of a first calibration object or of the object at each focus position, and determining the partially confocal intensities for each of the partially confocal images thus obtained; calculating a z-direction intensity profile at the first calibration location, and determining a maximum calibration intensity of the intensity profile at the first calibration location; imaging the first calibration location with the microscope in wide field, and determining a first calibration wide field intensity at the first calibration location; and calculating the scaling factor as a ratio between the maximum calibration intensity and the first calibration wide field intensity.
- the scaling factor can be determined either on a first calibration object, which differs from the object, or on the object itself.
- the first calibration object preferably has similar optical properties as the object to be measured, especially with respect to reflection and scattering of light.
- the object and the first calibration object may deviate with respect to their optical properties by less than 10%, 20% or 50%.
- the use of a first calibration object has the advantage that the determination of the scaling factor does not need to be performed every time before the method.
- Ascertaining the scaling factor on the object to be measured itself has the advantage that the scaling factor is determined with a particularly great accuracy.
- the scaling factor is determined in particular before the method is performed, as was described above.
- the first calibration location can be any desired location of the first calibration object or of the object. It is possible for example for the first calibration location to coincide with the first location, for example if the determination of the scaling factor is performed right before the described method is performed.
- the z-direction intensity profile of the partially confocal image is determined.
- a pinhole is provided in the imaging beam path, the opening of which has a size as is used in the method described below.
- a pinhole is in particular used which is used both for determining the scaling factor and also for determining the height position of the object at the first location.
- the plurality of partially confocal images having foci shifted in the z-direction form a z-stack and the partially confocal intensity is determined for each partially confocal image, such that the intensity profile in the z-direction can be determined therefrom.
- the maximum calibration intensity is calculated on the basis of said intensity profile and correlated to the likewise captured calibration wide field intensity, which was determined at the first calibration location.
- the scaling factor is the ratio of the maximum calibration intensity to the first calibration wide field intensity.
- the calibration wide field intensity is ascertained in particular like the wide field intensity determined above.
- the linking factor is determined by performing the following steps: partially confocally imaging a lateral second calibration location of a second calibration object or of the object with the focus located at a calibration measurement position in the z-direction which position is within the depth of field range, and determining a partially confocal calibration intensity of the partially confocal image at the second calibration location; confocally imaging the second calibration location with the focus located at the calibration measurement position, and determining a confocal calibration intensity of the confocal image at the second calibration location; imaging the second calibration location with the microscope in wide field, and determining a second calibration wide field intensity at the second calibration location; and calculating the linking factor as a ratio between the difference between the partially confocal calibration intensity and the confocal calibration intensity to the second calibration wide field intensity.
- the second calibration object can be identical with the first calibration object.
- the second calibration object is embodied similarly to the object with respect to reflection and scattering of light, for example the optical parameters differ by less than 10%, 20% or 50%.
- determination of the linking factor is preferably performed on the object itself.
- a partially confocal image is produced at an arbitrarily chosen second calibration location, which can coincide with the first calibration location or the first location.
- the size of the pinhole in the imaging beam path is the same as in the above-described method.
- the pinhole is preferably identical to the above-described method which is performed after the linking factor was determined.
- the confocal imaging is performed with a pinhole whose diameter corresponds to the diffraction limit of the imaging.
- the pinhole for the partially confocal imaging is thus larger than the pinhole for the confocal imaging.
- the confocal calibration intensity of the confocal image, determined at the second calibration location corresponds to a calibration intensity corresponding to the point spread function, because the point spread function is obtained in confocal imaging.
- the second calibration wide field intensity can coincide with the first calibration wide field intensity if it is determined at the same location as the first calibration wide field intensity.
- the step of imaging the second calibration location of the microscope in wide field and determining a second calibration wide field intensity at the second calibration location can be omitted if the first calibration location coincides with the second calibration location.
- the calculation of the linking factor is performed inversely to the determination of the intensity corresponding to the point spread function on the basis of the above-described relationship between intensity of the partially confocal image, intensity of the confocal image and wide field intensity.
- the first location is imaged with the microscope in wide field and a current wide field intensity is determined at the first location.
- a wide field image is recorded at each of the first locations and consequently the wide field intensity at each first location is determined.
- This procedure has the advantage that, if the object exhibits strong variations with respect to optical properties along the extent of the object, specifically with respect to reflectance or scattering of light, the appropriate wide field intensity is used in each case. This, thus, increases the accuracy of the method by determining the wide field intensity at each first location.
- the method furthermore to have the following steps: partially confocally imaging the first location with the first focus at least at two z measurement positions which are spaced apart in the z-direction and are situated in each case within the depth of field range, and determining partially confocal intensities for each partially confocal image at the first location; calculating the intensities corresponding to the point spread function at the first location by way of a respective difference between the respective partially confocal intensity and a product of wide field intensity and the predefined linking factor; calculating the z-coordinate of the focus in the z-direction at which the point spread function is maximum using the form of the point spread function, the calculated intensities corresponding to the point spread function and the calculated expected maximum intensity.
- two or more partially confocal images of the location are taken, wherein the foci of the partially confocal images are spaced apart in the z-direction.
- the size of the pinhole is preferably identical for each partially confocal image.
- the second location is imaged in partially confocal fashion and that, for calculating a z-coordinate of the focus in the z-direction at the second location, the z-coordinate of the focus in the z-direction at the first location is taken into consideration.
- the information relating to the height position of the object at a previous point is used to resolve the ambiguity of the above-described method. Provision is made in particular to use the z-coordinate that is closest to the previous measurement as a true z-coordinate.
- the second location can be any desired location on the object that is distant from the first location. In particular, the second location can be selected by a user of the microscope.
- this embodiment Since for this embodiment the second location is imaged in partially confocal fashion only once, this embodiment is particularly fast, with the result that it is even possible to display to the observer live images of the elevation of the object. This is helpful in particular if the user can manually select the second location. It is also possible for the manually selected second locations to be linked together to form a topography image of the object.
- the method furthermore to comprise the following steps: partially confocally imaging the first location with at least two different pinholes and with the focus at a z measurement position in the z-direction which is situated within the depth of field range, and determining a further partially confocal intensity for the partially confocal image with the further pinhole; and calculating the z-coordinate using the point spread function, the calculated intensity corresponding to the point spread function, the further partially confocal intensity and the calculated expected maximum intensity.
- two partially confocal images are recorded at the first location, wherein the partially confocal imaging is performed with differently sized pinholes.
- An intensity corresponding to the point spread function is preferably calculated only for one of the partially confocal images, as has been explained above.
- the further partially confocal intensity may be used to resolve the above-described ambiguity.
- the further partially confocal intensity can be used to determine the slope side of the image point spread function on which the intensity corresponding to the point spread function is situated.
- the advantage of this embodiment is that for determining the height position of the object at the first location, the focus of the partially confocal image does not need to be changed, but only the size of the pinhole.
- a laser scanning microscope as microscope, wherein, for partially confocal imaging and imaging in wide field, the size of the pinhole is changed by adjusting or exchanging the pinhole.
- the laser scanning microscope used can comprise a pinhole device in which the size of the pinhole can be adjusted manually or automatically.
- This embodiment of the method in particular uses a laser scanning microscope having only one detector, which records both the partially confocal image and the image in wide field. Moreover, it is possible in this embodiment to determine the ambiguity in the determination of the height position of the object at the first location by way of two images using pinholes of different sizes.
- a laser scanning microscope as a microscope, wherein, for partially confocal imaging and imaging in wide field, two detectors are provided behind pinholes of different size.
- This embodiment can be implemented as an alternative or in addition to the previously described embodiment.
- one pinhole of individual size is provided for each of two detectors, which can be part of one detector device.
- the two pinholes in particular form one pinhole device.
- the greater one of the two pinholes is specifically provided for producing an image in wide field, while the smaller pinhole can be used to produce the partially confocal image.
- the wide field image and the partially confocal image can be produced at the same time, for example by providing a beam splitter. Consequently, it is possible in this embodiment for the wide field intensity to be determined simultaneously with the partially confocal intensity in each determination of the elevation at the first location, with the result that the method in this embodiment of the microscope is particularly fast.
- a laser scanning microscope be used as microscope, wherein a wide field camera is used for imaging in wide field.
- the laser scanning microscope is provided with a wide field camera, onto which a fraction of the radiation of the imaging beam path is directed, for example using a beam splitter.
- the image in wide field and the partially confocal image are laterally and vertically overlaid.
- the partially confocal image and the image in wide field to be recorded at the same time.
- a confocal topography microscope is used as the microscope, in which a grating is used as pinhole, wherein a first camera with the grating is used for partially confocal imaging and a second camera is used for imaging in wide field.
- Confocal topography microscopes serve in particular for determining the topography of an object, wherein the object is imaged in a large wavelength range.
- Confocal topography microscopes are known from the prior art.
- this embodiment provides a beam splitter which directs the imaging radiation onto the first camera and the second camera in the imaging beam path.
- the first camera and the grating which is provided in front of it produces a partially confocal image of the object, whereas the second camera takes a wide field image of the object. It is once again advantageous here that the partially confocal image and the image in wide field can be recorded simultaneously.
- the grating can be part of a pinhole device, while the first camera and second camera can be part of a detector device.
- a confocal Airy microscope is used as the microscope, in which the object is imaged onto a detector device which comprises a plurality of pixels and resolves a diffraction structure of the partially confocal image, wherein the wide field intensity and the partially confocal intensity are determined from one recording.
- Airy microscopes make it possible to image the first location of the object such that the diffraction structure of the image can be resolved using the pixels of the detector device. Since the diffraction profile of the imaging is present on the detector, it is possible for the partially confocal intensity and the intensity in wide field imaging to be captured at the same time on the same detector device. It is thus possible for the partially confocal image and the image in wide field to be recorded at the same time.
- a confocal Airy microscope is used as the microscope, in which the object is imaged onto a detector device which comprises a plurality of pixels and resolves a diffraction structure of the partially confocal image, wherein a wide field camera is used for imaging in wide field.
- the Airy microscope is provided with a wide field camera, onto which a component of the radiation of the imaging beam path is directed, for example using a beam splitter.
- the image in wide field and the partially confocal image are here preferably laterally and vertically overlaid. In this configuration it is in particular possible for the partially confocal image and the image in wide field to be recorded at the same time.
- the invention furthermore relates to a microscope for producing a partially confocal image and a wide field image of an object, wherein the microscope comprises a detector device for capturing the image of the object, a pinhole device for providing partially confocal imaging and a recording in wide field, a focusing device which is embodied for setting a focus of the partially confocal image, and a control device for controlling the pinhole device, which is connected to the detector device.
- the microscope images the object in a z-direction, which coincides with the height direction of the object, with a point spread function.
- the control device is embodied to perform the method as has been explained above.
- the microscope can be a laser scanning microscope, an Airy microscope or a confocal topography microscope.
- the microscope can comprise a light source which provides white light or light of a specific wavelength, depending on what is necessary for imaging the object.
- the detector device in particular comprises detectors which can be used to convert incident radiation into electrical signals, wherein said electrical signals can be passed on to the control device.
- the pinhole device preferably comprises a pinhole, which can be used to image the object in partially confocal fashion onto the detector device.
- the pinhole size can be adjustable, with the result that the pinhole device can also be used to produce an image in wide field.
- the pinhole can be removed from the imaging beam path, with the result that a wide field image is taken, then.
- a plurality of pinholes or gratings are provided, which are arranged on a mask that is movable laterally in the imaging beam path.
- the pinholes have different diameters, or the gratings have different grating constants.
- the focusing device is adapted to adjust the focus of the partially confocal image.
- the focusing device can comprise optics which can be used to change the position of the focal plane at the object.
- the focusing device can move the object along the z-direction and consequently change the position of the focal plane of the imaging.
- only a relative displacement between focal plane and object is important.
- the microscope can further comprise a positioning device which is adapted to displace the first location with respect to the detector device laterally.
- the positioning device can be embodied as a scanner, which can be used to move the object with respect to the detector device.
- the beam path can be modulated, for illuminating the object, such that, as a consequence, the first location can be changed with respect to the detector device; this is implemented for example in a laser scanning microscope.
- the detector device comprises a first detector for the partially confocal imaging and a second detector for imaging in wide field.
- the control device in particular serves for controlling the above-mentioned devices and for performing the previously described method.
- the considerations, preferred embodiments and advantageous with respect to the method analogously apply with respect to the microscope.
- FIG. 1 shows a laser scanning microscope as a first embodiment of a microscope for performing a method for ascertaining an elevation of an object
- FIG. 2 shows a block illustration for illustrating the method steps
- FIG. 3 shows a diagram for illustrating the calibration of the method
- FIG. 4 shows a diagram for illustrating the determined and measured intensities used for the method
- FIG. 5 shows an Airy microscope as a second embodiment of the microscope for performing the method
- FIG. 6 shows a confocal topography microscope as a third embodiment of a microscope for performing the method.
- a laser scanning microscope 10 serves for the scanning imaging of an object 12 , wherein a method, described below, for determining height position of the object 12 can be performed on the laser scanning microscope 10 .
- the laser scanning microscope 10 comprises a light source 14 , a detector device 16 , a pinhole device 18 , a focusing device 20 , a positioning device 22 , a control device 24 and a beam splitter 26 .
- the light source 14 is e.g. a laser, a light-emitting diode (LED) or another monochrome light source that emits light of a wavelength that excites fluorescent dyes in the object 12 to emit fluorescent light.
- Radiation of an illumination beam path 28 , produced by the light source 14 is directed onto the beam splitter 26 , which is provided in the form of a dichroic mirror.
- the beam splitter 26 reflects the light of the illumination beam path 28 onto the positioning device 22 .
- the positioning device 22 comprises a scanner, i.e. two movably mounted mirrors, which are not illustrated in FIG. 1 .
- the positioning device 22 can direct the illumination beam path 28 onto different locations of the object 12 .
- the deflection of the illumination radiation is controlled by the control device 24 , which is connected to the positioning device 22 .
- the positioning device 22 guides the illumination beam path 28 in scanning fashion over the object 12 .
- a sample stage carrying the object 12 is shifted.
- the illumination beam path 28 travels from the positioning device 22 to the focusing device 20 , which is provided in the form of an objective.
- the focusing device 20 is used to shift the focus of the illumination beam path 28 on the object 12 in a z-direction.
- the z-direction coincides with the height direction of the object 12 and is perpendicular to a lateral extension of the object 12 , which is defined by an x- and y-direction.
- the focusing device 20 comprises a plurality of lenses, which are movable with respect to one another and are actuated e.g. using an electric motor.
- the focusing device 20 is likewise connected to the control device 24 , with the result that the control device 24 can change the focus of the laser scanning microscope 10 .
- the focus coincides with a z measurement position of the laser scanning microscope 10 .
- the illumination beam path 28 travels from the focusing device 20 to the object 12 , which is attached to a sample holder 30 .
- the focusing device 20 can be used to shift the focus in the z-direction.
- the focus of the illumination beam path 28 can be laterally shifted in an x-direction and a y-direction, which are both perpendicular to the z-direction.
- the object 12 Provided in the object 12 are fluorescent dyes, which are excited by the illumination radiation and emit light into an imaging beam path 32 . If the object 12 contains no fluorescent dyes, the illumination beam path is partially reflected by the object 12 into the imaging beam path 32 . Up to the beam splitter 26 , the imaging beam path 32 travels in the opposite direction of the illumination beam path 28 . Due to the fact that the fluorescent light has a different wavelength than the illumination radiation, the radiation in the imaging beam path 32 can pass through the dichroic beam splitter 26 and pass to the pinhole device 18 , which has a pinhole of adjustable size. Alternatively, a partially reflective beam splitter can be provided instead of the dichroic beam splitter 26 , especially if the illumination radiation is reflected by the object 12 . In particular, the pinhole device has an electric motor that continuously or incrementally changes the size of the pinhole. The pinhole device 18 is connected to the control device 24 , with the result that the control device 24 sets the size of the pinhole.
- the pinhole is situated in an intermediate image plane of the imaging beam path 32 .
- the detector device 16 Arranged downstream thereof is the detector device 16 , which comprises a detector for converting the radiation of the imaging beam path 32 to electrical signals.
- the electrical signals are passed on to the control device 24 , which can determine therefrom the intensity of the radiation in the imaging beam path 32 .
- the laser scanning microscope 10 produces a partially confocal image by setting the pinhole of the pinhole device 18 to be small, for example 50-300% greater than a pinhole that images the object 12 in diffraction-limited fashion.
- the pinhole of the pinhole device 18 is further enlarged as compared to the partially confocal imaging, with the result that the image predominantly comprises wide field components.
- the method for operating the laser scanning microscope 10 is explained using the block diagram in FIG. 2 .
- the method is used to determine a height position of the object 12 at a first location of the object 12 .
- the first location can be any location of the object 12 .
- the method is in particular repeated at a plurality of first locations, such that a plurality of elevations of the object 12 can be determined, from which the topography of the object 12 can be derived.
- Step S 1 serves for calibrating a scaling factor Sk
- step S 2 serves for calibrating a linking factor Vk
- step S 3 serves for calculating the height position of the first location.
- the calibration of the scaling factor Sk in step S 1 will be explained.
- the calibration of the scaling factor Sk can be performed on the object 12 or on a calibration object that has in particular similar properties with respect to reflection and scattering of light as the object 12 .
- the calibration of the scaling factor Sk at a calibration location which is a site on the object 12 , will be discussed.
- the calibration location is an example of the first calibration location.
- step S 1 . 1 a z-stack of a plurality of partially confocal images is recorded at the calibration location.
- the images cover the depth of field range of the partially confocal image, wherein the pinhole device 18 sets a first pinhole size.
- the intensity is determined for each partially confocal image, such that an intensity distribution I K (z) is obtained in dependence on the measurement position z of the focus in step S 1 . 2 . This is illustrated by way of example in FIG. 3 .
- the maximum calibration intensity I k,max can be determined at the position of the focus z k,max .
- a calibration wide field intensity I KW1 is determined by setting the pinhole device 18 to a second size for the pinhole, which is greater than the first size.
- the second size of the pinhole is such that the intensity of the wide field image dominates in the image.
- the second size is 5 to 10 times greater than the first size, for example.
- the scaling factor Sk satisfying the following equation is calculated:
- step S 2 . 1 of step S 2 a point spread function PSF(z) in the z-direction of the laser scanning microscope 10 is determined. This step is not necessary if the point spread function PSF(z) for the laser scanning microscope 10 is already known.
- step S 2 . 2 partially confocal imaging of the second calibration location is performed on a second calibration object, which can coincide with the first calibration object, or at a second calibration location on the object 12 , optionally coinciding with the first calibration location.
- the size of the pinhole is set to the first size.
- the focus of the partially confocal image is situated at a calibration measurement position z k1 which is located in the depth of field range of the partially confocal image.
- a partially confocal calibration intensity I k1 of the partially confocal image of the second calibration location is determined at the calibration measurement position z k1 .
- step S 2 . 3 confocal imaging of the second calibration location is performed.
- the focus of the confocal image is likewise situated at the calibration measurement position z k1 .
- a confocal calibration intensity I k1 * of the confocal image of the second calibration location is determined at the calibration measurement position z k1 .
- the size of the pinhole of the pinhole device 18 is set such that it is at imaging diffraction limit.
- the confocal calibration intensity I k1 * coincides with the intensity of the point spread function PSF(z) at the calibration measurement position z k1 .
- step S 2 . 4 a calibration wide field intensity I KW2 is also determined. This is done analogously to the determination of the calibration wide field intensity I KW1 in step S 1 . 3 .
- the linking factor Vk is determined in step S 2 . 5 from the partially confocal calibration intensity I k1 , the confocal calibration intensity I k1 * and the calibration wide field intensity I kW2 in accordance with the following equation:
- I k1 I* k1 +Vk ⁇ I kW2 (2)
- step S 3 the laser scanning microscope 10 partially confocally images the first location, wherein the focus is situated at a z measurement position z 1 located in the depth of field range.
- the pinhole is here set to the first size.
- the partially confocal intensity I 1 is determined. From the partially confocal intensity I 1 , the partially confocal intensity I 1 * that corresponds to the point spread function PSF is calculated in step S 3 . 2 using the linking factor Vk and the above-described equation (2).
- the intensity I 1 * that corresponds to the point spread function PSF corresponds to the intensity of the point spread function PSF at the z measurement position z 1 .
- step S 3 further partially confocal imaging is performed in step S 3 . 3 , in which the pinhole is changed with respect to the first performed partially confocal imaging.
- the pinhole can be embodied to be adjustable in terms of size or pinholes are exchanged. From said further partially confocal imaging using a different pinhole, a further partially confocal intensity I 1,pin can be calculated.
- a wide field image of the first location is produced in step S 3 . 4 , as described above, and the wide field intensity Iw is determined therefrom.
- the maximum intensity I max * of the point spread function PSF is calculated.
- a z-coordinate z h of the focus which coincides with the z-coordinate of the maximum intensity I max * of the point spread function PSF, is determined in step S 3 . 5 using equation (2). Since only two values are used for the determination, the determination is not always unique, as is indicated in FIG. 4 by the two regions in the z-direction ⁇ z 1 and ⁇ z 1 ′.
- FIG. 5 A second embodiment of a microscope for performing the method is schematically illustrated in FIG. 5 .
- an Airy microscope 100 is used, the schematic setup of which coincides with the laser scanning microscope 10 , except for the following differences that are shown:
- a positioning device 122 of the Airy microscope 100 does not operate by way of deflecting the radiation of the illumination beam path 28 , but by moving the object 12 with respect to the illumination beam path 28 using a scanner, which forms the positioning device 122 .
- the scanner moves the sample holder 30 .
- the positioning device 122 is again connected to the control device 24 here, such that the control device 24 can set the positioning of the object 12 with respect to the illumination beam path 28 .
- a detector device 116 comprises a plurality of pixels to resolve the diffraction structure of the partially confocal image.
- the mode of operation of the Airy microscope 100 does not differ from the mode of operation of the laser scanning microscope 10 with respect to the principle of determining the height position at the first location of the object 12 . Only the manner of producing the partially confocal image and the wide field image differs.
- the intensities of a single partially confocal image that are measured by the pixels comprise the partially confocal intensity I 1 and the wide field intensity I W . It is thus possible with a single partially confocal image to determine both intensities I 1 and I W that are required for the calculation of the elevation of the first location.
- step S 3 . 1 The above-described method for determining the partially confocal intensity I 1 (step S 3 . 1 ) and the partially confocal intensity I 1 * (step S 3 . 2 ) which corresponds to the point spread function PSF is repeated at a second measurement position z 2 , from which the partially confocal intensity I 2 and a partially confocal intensity I 2 * which corresponds to the point spread function PSF are obtained.
- step S 3 . 3 Further partially confocal imaging with a changed size of the pinhole (step S 3 . 3 ) is not performed in the Airy microscope 100 , but step S 3 . 5 is modified as follows.
- the PSF whose form is known, can be fitted on the basis of these values so that the Z coordinate z h of the maximum intensity I max * can be ascertained.
- Said z-coordinate z h corresponds to the elevation of the object 12 at the first location, such that the first location can be ascertained from the two partially confocal images at the measurement positions z 1 and z 2 .
- FIG. 6 schematically illustrates a confocal topography microscope 200 , which differs from the laser scanning microscope 10 and the Airy microscope 100 merely in the following differences:
- the confocal typography microscope 200 has, analogously to the Airy microscope 100 , a positioning device 222 which likewise comprises as a scanner for moving the sample holder 30 .
- the embodiment of the positioning device is not relevant here either.
- a light source 214 can be embodied as a laser, a light-emitting diode (LED) or any other monochrome light source, but the use of a white light source, which produces light in the entire visible range, so that the object 12 is illuminated with illumination radiation in the visible wavelength range, is also possible.
- a detector device 216 has a first camera 216 a , a second camera 216 b and a second beam splitter 216 c .
- the second beam splitter 216 c is embodied as a partially reflective mirror, which directs 50% of the radiation in the imaging beam path 32 onto the first camera 216 a and 50% of the radiation in the imaging beam path 32 onto the second camera 216 b .
- a pinhole device 218 is in the form of a mask having a plurality of gratings having different grating spacings, which is movable laterally through the imaging beam path 32 .
- the confocal topography microscope 200 can be used, owing to the second beam splitter 216 c , to simultaneously produce an image in wide field using the second camera 216 b , in front of which no pinhole is arranged, and a partially confocal image using the first camera 216 a .
- the mode of operation to this extent coincides with the mode of operation of the Airy microscope 100 , since here a partially confocal image and an image in wide field are also produced at the same time.
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Optics & Photonics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Microscoopes, Condenser (AREA)
Abstract
A method for determining a height position of an object at of using a microscope which images using a point-spread function along a z-direction (height direction), comprising the steps of imaging the object in the far field and determining a far-field intensity, calculating a maximum intensity expected by multiplying the far-field intensity by a scaling factor, partially confocally imaging the object with the focus in the z-direction within the depth-of-field range, and determining a partially-confocal intensity of the imaging, calculating the intensity of the point-spread function (at the first location) by forming a difference between the partially-confocal intensity and a product of the far-field intensity and a predefined combination factor, calculating the z-coordinate of the focus at a point-spread function maximum, using a previously-known form of the point spread function, its calculated intensity, and the calculated expected maximum intensity, and using the z-coordinate as the height position of the object.
Description
- The present application is a National Phase entry of PCT Application No. PCT/EP2016/080771, filed Dec. 13, 2016, which claims priority from German Patent Application Number 102016103736.2, filed Mar. 2, 2016, the disclosures of which are hereby incorporated by reference herein in their entirety.
- The invention relates to a method for determining a height position of an object at a lateral first location of the object by using a microscope, which images the object with a point spread function along a z-direction that coincides with the height direction. The invention further relates to a microscope for producing a partially confocal image and a wide field image of an object, wherein the microscope comprises a detector device, a pinhole device for partially confocal imaging and for imaging in wide field, a focusing device which is adapted for setting a z-position of a focus of the partially confocal image, and a control device for controlling the pinhole device, which control device is connected to the detector device.
- Methods are known from the prior art, which can be used to determine the topography of an object using an optical microscope. In particular, confocal microscopes are used herefor. In said known methods, the focus of a partially confocal image is shifted along a z-direction, i.e. along a direction of the height extension of the topography, and a partially confocal image is recorded at each z-position. The partially confocal imaging is typically provided such that the image comprises a purely confocal component and a component of a wide field. The intensity is determined for each of said images in the z-direction, which are collectively also referred to as a stack, and an intensity profile in the z-direction is calculated. On the basis of this calculated intensity profile, the maximum intensity can be determined, the z-coordinate of which coincides with the height position of the object at the measured point. In order to determine the height positions of the object over an area, the above-described method is repeated at various locations. For determining the maximum intensity, it is typically necessary to carry out a multiplicity of measurements along the z-direction in order to obtain a proper accuracy for the intensity maximum, and, thus, the elevation of the object. For this reason, the method can require with a great expenditure of time.
- US 2004/0008515 A1 relates to an improvement of fluorescence illumination for three-dimensionally resolving microscopy. The publication Hiraoka Y. et al., Biophys. J., Vol. 57, February 1990, pp. 325-333, explains the ascertainment of the three-dimensional imaging properties in an optical microscope. McNally J. et al., SPIE, Vol. 2984, 1997, pp. 52-63, compare several 3D microscopy methods using an accurately known test object.
- It is an object of the invention to provide a system for the quick determination of a height position of an object.
- In a method for determining a height position of an object at a lateral first location of the object by using a microscope, which images the object with a point spread function along a z-direction that coincides with the height direction, the following steps are performed: imaging the object using the microscope in wide field and determining a wide field intensity, calculating a maximum intensity that is expected at the first location by multiplying the wide field intensity by a predetermined scaling factor, partially confocally imaging the object at the first location with the focus located at a measurement position in the z-direction, wherein the partially confocal image defines a depth of field range and the focus is located within the depth of field range, and determining a partially confocal intensity of the partially confocal image at the first location, calculating an intensity corresponding to the point spread function at the first location by way of subtraction between the partially confocal intensity and the product of wide field intensity and a predetermined linking factor, computing a z-coordinate of the focus in the z-direction at which the point spread function is maximum using a predetermined form of the point spread function, the intensity corresponding to the calculated point spread function and the calculated expected maximum intensity, and using the z-coordinate as the height position of the object at the first location.
- The method is preferably used to determine a topography of a laterally extending object. To this end, the elevation at a plurality of first locations can be determined, wherein the first locations have a lateral distance from one another. The first locations consequently relate in particular to the lateral extent of the object. The first locations are preferably spaced apart perpendicularly to the z-direction.
- The method is preferably used in a scanning manner, wherein the claimed method or parts of the method is/are performed at each scanning point, which can correspond to the first location. The topography of the object can be determined from the height positions of the individual first locations.
- The microscope is in particular adapted to image the object in a partially confocal manner. Partially confocal in this context is intended to mean that the pinhole limits imaging in the z-direction, but not in diffraction-limited manner. Then, the size of the pinhole determines the region of the first location in the z-direction that is imaged by the partially confocal imaging. As is generally known, the resolution in the z-direction increases as the size of the pinhole decreases.
- Confocal imaging in this context is intended to mean that the pinhole has an extent that corresponds to the diffraction limit of the imaging. Partially confocal imaging in particular comprises components of a purely confocal image and of the wide field image. In partially confocal imaging the pinhole is in particular larger than in confocal imaging. Partially confocal imaging serves in particular for capturing a so-called composite image.
- The microscope, in particular the optics thereof and the size of the pinhole, determines the depth of field range in the z-direction. The depth of field range is thus a property of the microscope which can be determined in advance by known methods.
- The object in particular has a topography whose height variation is in the nanometer to millimeter range. The object can be any sample which can be optically imaged. The object can in particular be a biological sample.
- The partially confocal image defines the depth of field range. The depth of field range depends on the size of the pinhole, wherein the depth of field range in the z-direction increases as the size of the pinhole increases. Consequently, confocal imaging has a smaller depth of field range in the z-direction than partially confocal imaging. Further, the depth of field range of a partially confocal imaging is smaller than the depth of field range of a wide field imaging. If the size of a confocally imaging pinhole is decreased further, the depth of field range does not change, but only the intensity of the confocal image is reduced.
- A wide field imaging is intended to mean in particular an imaging of the object which has no, or a low, spatial resolution in the z-direction. This can be done by providing no pinhole, or a pinhole having a large diameter, in an imaging beam path which represents the beam path of the light, from emission or reflection at the first location to a detector device of the microscope. After wide field imaging, the intensity thereof is determined, which corresponds to the wide field intensity.
- Wide field imaging can be performed at the first location or at any other location of the object and preferably serves for capturing the optical properties, such as for example the reflectance of the sample. The wide field intensity is preferably determined only once, such that, when the method is repeated, the wide field intensity of a previous measurement is utilized. Determining the wide field intensity only once is suitable in particular if the object is one whose optical properties, in particular the reflectance, are constant or approximately constant or change by less than 10%, 20% or 50% over the extent of the object. Consequently, if the method illustrated here is repeated, preferably wide field imaging of the first location is performed only once.
- As a further step of the method, the maximum intensity expected at the first location is calculated, which corresponds to the maximum intensity that was obtained in partially confocal imaging or confocal imaging. This calculation is based on the finding that the maximum intensity in confocal or partially confocal imaging is proportional to the wide field intensity. This relationship corresponds to the scaling factor, which is preferably determined in advance for the microscope, and in particular on the object. The calculation of the expected maximum intensity at the first location can be performed only once if the determination of the wide field intensity is likewise performed only once. For this reason, the determination of the expected maximum intensity must not be performed again upon repetition of the method.
- The partially confocal imaging of the object at the first location is performed upon repetition of the method in particular at each of the first locations. The focus of this image is provided at a z measurement position which is situated in the depth of field range of the partially confocal imaging of the microscope. In particular, the z measurement position is situated in a depth of field range which is established by confocal imaging. The choice of the z measurement position is preferably random, or the same z measurement position is used for the partially confocal imaging at each of the first locations. Establishing the z measurement position such that it is situated in the depth of field range of the partially confocal image can be effected by trial and error or manually. In particular, other known parameters of the object or of the microscope can be used to determine the depth of field range of the partially confocal imaging. The partially confocal intensity determined from the partially confocal imaging can preferably be used to discover whether the z measurement position is situated in the depth of field range. If this is not the case, this method step can be repeated until it is certain that the z measurement position is located in the depth of field range of the partially confocal imaging.
- The partially confocal imaging is preferably performed by providing a pinhole in the imaging beam path. The partially confocal imaging consequently has a better resolution in the z-direction as compared to the wide field image. Due to the fact that the point spread function for the microscope can be determined, in particular the precise profile of the point spread function is known, for example the deviation of the point spread function from a Gaussian curve and the standard width. Purely confocal imaging preferably images the object with the point spread function.
- The maximum intensity of the point spread function is referred to in particular as the intensity corresponding to the point spread function. The z measurement position in the z-direction which coincides with the maximum intensity of the point spread function can be calculated based on the assumption that the intensity of a partially confocal imaging at the first location is equal to the sum of the intensity of a confocal imaging at the first location and the wide field intensity multiplied by a linking factor. The linking factor is determined preferably before performing the method, for example for the observed object or generally for the microscope. From this relationship it is possible to calculate from the measured variables, specifically the partially confocal intensity and the wide field intensity, the intensity corresponding to the point spread function at the first location. This method step is preferably performed during the topography ascertainment for each first location.
- The desired z-coordinate is the position of the focus in the z-direction at which the intensity of the point spread function is maximum. It is calculated using the information relating to the form of the point spread function and two points which are known of the point spread function. This is possible since the point spread function in the z-direction coincides with the intensity profile of a confocal image. The calculation of the position at which the point spread function is maximum can be performed for example by fitting the point spread function to the two known points.
- One of these two known points corresponds to the intensity of the partially confocal imaging corresponding to the point spread function and the other corresponds to the expected maximum intensity which was determined previously, as described above. The z-coordinate of the focus at which the intensity profile of the point spread function is maximum is the height positon of the object at the first location.
- In the determination of the position for which the point spread function is maximum, an ambiguity may arise, but can be resolved as described below. Optionally, this problem can be solved by way of the determined z-coordinate of the focus being compared to previously determined z-coordinates, for example by using the z-coordinate of the two determined z-coordinates that is closest to the previous measurement as the true position. It is moreover possible for further information that is known of the object to be used to resolve the ambiguity.
- Preferably only one partially confocal imaging step is performed for determining the height position of the object at the first location. Since in known methods for optically determining the topography of an object z-stacks of typically ten or more partially confocal images with foci which are spaced apart in the z-direction must be recorded and these measurements are performed sequentially, the invention reduces the duration for determining the height position at the location of the object to up to 10% as compared to known methods. It is in particular possible with the method to record a single partially confocal image for each first location such that, using known confocal imaging methods, not only a two-dimensional image but even a three-dimensional image of the object can be produced.
- It is preferred that for the partially confocal imaging a pinhole device having an opening which is greater than a diffraction limit is used, such that the intensity of the partially confocal imaging comprises components of a wide field image.
- An advantage of this embodiment is that the intensity profile of the partially confocal imaging depending on the z-position has a larger depth of field range as compared to a purely confocal image, i.e. an image in which the pinhole has an opening that corresponds to the diffraction limit. The depth of field range in this embodiment is consequently larger and is easier to set. Moreover, greater height differences can be determined.
- It is preferred that the scaling factor is determined by performing the following steps: producing a z-stack by repeatedly shifting the focus positon in the z-direction and partially confocal imaging a lateral first calibration location of a first calibration object or of the object at each focus position, and determining the partially confocal intensities for each of the partially confocal images thus obtained; calculating a z-direction intensity profile at the first calibration location, and determining a maximum calibration intensity of the intensity profile at the first calibration location; imaging the first calibration location with the microscope in wide field, and determining a first calibration wide field intensity at the first calibration location; and calculating the scaling factor as a ratio between the maximum calibration intensity and the first calibration wide field intensity.
- The scaling factor can be determined either on a first calibration object, which differs from the object, or on the object itself. The first calibration object preferably has similar optical properties as the object to be measured, especially with respect to reflection and scattering of light. In particular, the object and the first calibration object may deviate with respect to their optical properties by less than 10%, 20% or 50%. The use of a first calibration object has the advantage that the determination of the scaling factor does not need to be performed every time before the method. Ascertaining the scaling factor on the object to be measured itself, on the other hand, has the advantage that the scaling factor is determined with a particularly great accuracy. The scaling factor is determined in particular before the method is performed, as was described above.
- The first calibration location can be any desired location of the first calibration object or of the object. It is possible for example for the first calibration location to coincide with the first location, for example if the determination of the scaling factor is performed right before the described method is performed.
- To determine the scaling factor, the z-direction intensity profile of the partially confocal image is determined. Here, a pinhole is provided in the imaging beam path, the opening of which has a size as is used in the method described below. For the partially confocal imaging, a pinhole is in particular used which is used both for determining the scaling factor and also for determining the height position of the object at the first location.
- The plurality of partially confocal images having foci shifted in the z-direction form a z-stack and the partially confocal intensity is determined for each partially confocal image, such that the intensity profile in the z-direction can be determined therefrom. The maximum calibration intensity is calculated on the basis of said intensity profile and correlated to the likewise captured calibration wide field intensity, which was determined at the first calibration location. In particular, the scaling factor is the ratio of the maximum calibration intensity to the first calibration wide field intensity. The calibration wide field intensity is ascertained in particular like the wide field intensity determined above.
- It is preferred that the linking factor is determined by performing the following steps: partially confocally imaging a lateral second calibration location of a second calibration object or of the object with the focus located at a calibration measurement position in the z-direction which position is within the depth of field range, and determining a partially confocal calibration intensity of the partially confocal image at the second calibration location; confocally imaging the second calibration location with the focus located at the calibration measurement position, and determining a confocal calibration intensity of the confocal image at the second calibration location; imaging the second calibration location with the microscope in wide field, and determining a second calibration wide field intensity at the second calibration location; and calculating the linking factor as a ratio between the difference between the partially confocal calibration intensity and the confocal calibration intensity to the second calibration wide field intensity.
- The second calibration object can be identical with the first calibration object. In particular, the second calibration object is embodied similarly to the object with respect to reflection and scattering of light, for example the optical parameters differ by less than 10%, 20% or 50%. However, determination of the linking factor is preferably performed on the object itself. To this end, a partially confocal image is produced at an arbitrarily chosen second calibration location, which can coincide with the first calibration location or the first location. In particular, the size of the pinhole in the imaging beam path is the same as in the above-described method. The pinhole is preferably identical to the above-described method which is performed after the linking factor was determined.
- The confocal imaging is performed with a pinhole whose diameter corresponds to the diffraction limit of the imaging. The pinhole for the partially confocal imaging is thus larger than the pinhole for the confocal imaging. The confocal calibration intensity of the confocal image, determined at the second calibration location, corresponds to a calibration intensity corresponding to the point spread function, because the point spread function is obtained in confocal imaging.
- The second calibration wide field intensity can coincide with the first calibration wide field intensity if it is determined at the same location as the first calibration wide field intensity. In particular, the step of imaging the second calibration location of the microscope in wide field and determining a second calibration wide field intensity at the second calibration location can be omitted if the first calibration location coincides with the second calibration location.
- The calculation of the linking factor is performed inversely to the determination of the intensity corresponding to the point spread function on the basis of the above-described relationship between intensity of the partially confocal image, intensity of the confocal image and wide field intensity.
- It is preferred that the first location is imaged with the microscope in wide field and a current wide field intensity is determined at the first location.
- In particular, upon repetition of the above-described method, a wide field image is recorded at each of the first locations and consequently the wide field intensity at each first location is determined. This procedure has the advantage that, if the object exhibits strong variations with respect to optical properties along the extent of the object, specifically with respect to reflectance or scattering of light, the appropriate wide field intensity is used in each case. This, thus, increases the accuracy of the method by determining the wide field intensity at each first location.
- It is preferred for the method furthermore to have the following steps: partially confocally imaging the first location with the first focus at least at two z measurement positions which are spaced apart in the z-direction and are situated in each case within the depth of field range, and determining partially confocal intensities for each partially confocal image at the first location; calculating the intensities corresponding to the point spread function at the first location by way of a respective difference between the respective partially confocal intensity and a product of wide field intensity and the predefined linking factor; calculating the z-coordinate of the focus in the z-direction at which the point spread function is maximum using the form of the point spread function, the calculated intensities corresponding to the point spread function and the calculated expected maximum intensity.
- In this preferred embodiment, two or more partially confocal images of the location are taken, wherein the foci of the partially confocal images are spaced apart in the z-direction. The size of the pinhole is preferably identical for each partially confocal image. Two partially confocal images and the corresponding determination of the partially confocal intensities make it possible for the position at which the point spread function is maximum to be determined on the basis of three points, specifically the two intensities corresponding to the point spread function and the maximum intensity of the point spread function. It is consequently possible to reliably resolve the ambiguity of the above-described method.
- It is preferred that, for producing a live determination of a height profile of the object at a laterally second location of the object, which differs from the first one, the second location is imaged in partially confocal fashion and that, for calculating a z-coordinate of the focus in the z-direction at the second location, the z-coordinate of the focus in the z-direction at the first location is taken into consideration.
- In this embodiment, the information relating to the height position of the object at a previous point is used to resolve the ambiguity of the above-described method. Provision is made in particular to use the z-coordinate that is closest to the previous measurement as a true z-coordinate. The second location can be any desired location on the object that is distant from the first location. In particular, the second location can be selected by a user of the microscope.
- Since for this embodiment the second location is imaged in partially confocal fashion only once, this embodiment is particularly fast, with the result that it is even possible to display to the observer live images of the elevation of the object. This is helpful in particular if the user can manually select the second location. It is also possible for the manually selected second locations to be linked together to form a topography image of the object.
- It is preferred for the method furthermore to comprise the following steps: partially confocally imaging the first location with at least two different pinholes and with the focus at a z measurement position in the z-direction which is situated within the depth of field range, and determining a further partially confocal intensity for the partially confocal image with the further pinhole; and calculating the z-coordinate using the point spread function, the calculated intensity corresponding to the point spread function, the further partially confocal intensity and the calculated expected maximum intensity.
- In this embodiment, which can be used in addition or as an alternative to the previously mentioned embodiments, two partially confocal images are recorded at the first location, wherein the partially confocal imaging is performed with differently sized pinholes. An intensity corresponding to the point spread function is preferably calculated only for one of the partially confocal images, as has been explained above. The further partially confocal intensity may be used to resolve the above-described ambiguity. The further partially confocal intensity can be used to determine the slope side of the image point spread function on which the intensity corresponding to the point spread function is situated.
- The advantage of this embodiment is that for determining the height position of the object at the first location, the focus of the partially confocal image does not need to be changed, but only the size of the pinhole.
- It is preferred to use a laser scanning microscope as microscope, wherein, for partially confocal imaging and imaging in wide field, the size of the pinhole is changed by adjusting or exchanging the pinhole.
- The laser scanning microscope used can comprise a pinhole device in which the size of the pinhole can be adjusted manually or automatically. This embodiment of the method in particular uses a laser scanning microscope having only one detector, which records both the partially confocal image and the image in wide field. Moreover, it is possible in this embodiment to determine the ambiguity in the determination of the height position of the object at the first location by way of two images using pinholes of different sizes.
- It is preferred to use a laser scanning microscope as a microscope, wherein, for partially confocal imaging and imaging in wide field, two detectors are provided behind pinholes of different size.
- This embodiment can be implemented as an alternative or in addition to the previously described embodiment. For each of two detectors, which can be part of one detector device, one pinhole of individual size is provided. The two pinholes in particular form one pinhole device. The greater one of the two pinholes is specifically provided for producing an image in wide field, while the smaller pinhole can be used to produce the partially confocal image.
- It is a preferred advantage of this embodiment that the wide field image and the partially confocal image can be produced at the same time, for example by providing a beam splitter. Consequently, it is possible in this embodiment for the wide field intensity to be determined simultaneously with the partially confocal intensity in each determination of the elevation at the first location, with the result that the method in this embodiment of the microscope is particularly fast.
- It is preferred that a laser scanning microscope be used as microscope, wherein a wide field camera is used for imaging in wide field.
- In this configuration, the laser scanning microscope is provided with a wide field camera, onto which a fraction of the radiation of the imaging beam path is directed, for example using a beam splitter. Preferably, the image in wide field and the partially confocal image are laterally and vertically overlaid. In this configuration it is in particular possible for the partially confocal image and the image in wide field to be recorded at the same time.
- It is preferred that a confocal topography microscope is used as the microscope, in which a grating is used as pinhole, wherein a first camera with the grating is used for partially confocal imaging and a second camera is used for imaging in wide field.
- Confocal topography microscopes serve in particular for determining the topography of an object, wherein the object is imaged in a large wavelength range. Confocal topography microscopes are known from the prior art. In contrast therewith, this embodiment provides a beam splitter which directs the imaging radiation onto the first camera and the second camera in the imaging beam path. Using the first camera and the grating which is provided in front of it produces a partially confocal image of the object, whereas the second camera takes a wide field image of the object. It is once again advantageous here that the partially confocal image and the image in wide field can be recorded simultaneously. The grating can be part of a pinhole device, while the first camera and second camera can be part of a detector device.
- It is preferred that a confocal Airy microscope is used as the microscope, in which the object is imaged onto a detector device which comprises a plurality of pixels and resolves a diffraction structure of the partially confocal image, wherein the wide field intensity and the partially confocal intensity are determined from one recording.
- Airy microscopes make it possible to image the first location of the object such that the diffraction structure of the image can be resolved using the pixels of the detector device. Since the diffraction profile of the imaging is present on the detector, it is possible for the partially confocal intensity and the intensity in wide field imaging to be captured at the same time on the same detector device. It is thus possible for the partially confocal image and the image in wide field to be recorded at the same time.
- It is preferred that a confocal Airy microscope is used as the microscope, in which the object is imaged onto a detector device which comprises a plurality of pixels and resolves a diffraction structure of the partially confocal image, wherein a wide field camera is used for imaging in wide field.
- In this configuration, the Airy microscope is provided with a wide field camera, onto which a component of the radiation of the imaging beam path is directed, for example using a beam splitter. The image in wide field and the partially confocal image are here preferably laterally and vertically overlaid. In this configuration it is in particular possible for the partially confocal image and the image in wide field to be recorded at the same time.
- The invention furthermore relates to a microscope for producing a partially confocal image and a wide field image of an object, wherein the microscope comprises a detector device for capturing the image of the object, a pinhole device for providing partially confocal imaging and a recording in wide field, a focusing device which is embodied for setting a focus of the partially confocal image, and a control device for controlling the pinhole device, which is connected to the detector device. The microscope images the object in a z-direction, which coincides with the height direction of the object, with a point spread function. The control device is embodied to perform the method as has been explained above.
- The microscope can be a laser scanning microscope, an Airy microscope or a confocal topography microscope. The microscope can comprise a light source which provides white light or light of a specific wavelength, depending on what is necessary for imaging the object. The detector device in particular comprises detectors which can be used to convert incident radiation into electrical signals, wherein said electrical signals can be passed on to the control device.
- The pinhole device preferably comprises a pinhole, which can be used to image the object in partially confocal fashion onto the detector device. The pinhole size can be adjustable, with the result that the pinhole device can also be used to produce an image in wide field. Alternatively, the pinhole can be removed from the imaging beam path, with the result that a wide field image is taken, then. In the topography microscope, preferably a plurality of pinholes or gratings are provided, which are arranged on a mask that is movable laterally in the imaging beam path. Here, the pinholes have different diameters, or the gratings have different grating constants.
- The focusing device is adapted to adjust the focus of the partially confocal image. To this end, the focusing device can comprise optics which can be used to change the position of the focal plane at the object. Alternatively, the focusing device can move the object along the z-direction and consequently change the position of the focal plane of the imaging. Ultimately, only a relative displacement between focal plane and object is important.
- The microscope can further comprise a positioning device which is adapted to displace the first location with respect to the detector device laterally. The positioning device can be embodied as a scanner, which can be used to move the object with respect to the detector device. Alternatively, the beam path can be modulated, for illuminating the object, such that, as a consequence, the first location can be changed with respect to the detector device; this is implemented for example in a laser scanning microscope.
- It is preferred for the detector device to comprise a first detector for the partially confocal imaging and a second detector for imaging in wide field.
- The control device in particular serves for controlling the above-mentioned devices and for performing the previously described method. In particular, the considerations, preferred embodiments and advantageous with respect to the method analogously apply with respect to the microscope.
- It goes without saying that the aforementioned features and those yet to be explained below can be used not only in the combinations specified but also in other combinations or on their own, without departing from the scope of the present invention.
- The invention is explained in more detail below for example on the basis of the accompanying drawings, which also disclose features essential to the invention. In the figures:
-
FIG. 1 shows a laser scanning microscope as a first embodiment of a microscope for performing a method for ascertaining an elevation of an object; -
FIG. 2 shows a block illustration for illustrating the method steps; -
FIG. 3 shows a diagram for illustrating the calibration of the method; -
FIG. 4 shows a diagram for illustrating the determined and measured intensities used for the method; -
FIG. 5 shows an Airy microscope as a second embodiment of the microscope for performing the method; and -
FIG. 6 shows a confocal topography microscope as a third embodiment of a microscope for performing the method. - A
laser scanning microscope 10 serves for the scanning imaging of anobject 12, wherein a method, described below, for determining height position of theobject 12 can be performed on thelaser scanning microscope 10. - The
laser scanning microscope 10 comprises alight source 14, adetector device 16, apinhole device 18, a focusingdevice 20, apositioning device 22, acontrol device 24 and abeam splitter 26. - The
light source 14 is e.g. a laser, a light-emitting diode (LED) or another monochrome light source that emits light of a wavelength that excites fluorescent dyes in theobject 12 to emit fluorescent light. Radiation of anillumination beam path 28, produced by thelight source 14, is directed onto thebeam splitter 26, which is provided in the form of a dichroic mirror. Thebeam splitter 26 reflects the light of theillumination beam path 28 onto thepositioning device 22. - The
positioning device 22 comprises a scanner, i.e. two movably mounted mirrors, which are not illustrated inFIG. 1 . Using the scanner, thepositioning device 22 can direct theillumination beam path 28 onto different locations of theobject 12. The deflection of the illumination radiation is controlled by thecontrol device 24, which is connected to thepositioning device 22. In particular, thepositioning device 22 guides theillumination beam path 28 in scanning fashion over theobject 12. Alternatively, a sample stage carrying theobject 12 is shifted. - The
illumination beam path 28 travels from thepositioning device 22 to the focusingdevice 20, which is provided in the form of an objective. The focusingdevice 20 is used to shift the focus of theillumination beam path 28 on theobject 12 in a z-direction. The z-direction coincides with the height direction of theobject 12 and is perpendicular to a lateral extension of theobject 12, which is defined by an x- and y-direction. The focusingdevice 20 comprises a plurality of lenses, which are movable with respect to one another and are actuated e.g. using an electric motor. The focusingdevice 20 is likewise connected to thecontrol device 24, with the result that thecontrol device 24 can change the focus of thelaser scanning microscope 10. The focus coincides with a z measurement position of thelaser scanning microscope 10. Theillumination beam path 28 travels from the focusingdevice 20 to theobject 12, which is attached to asample holder 30. - The focusing
device 20 can be used to shift the focus in the z-direction. Using thepositioning device 22, the focus of theillumination beam path 28 can be laterally shifted in an x-direction and a y-direction, which are both perpendicular to the z-direction. - Provided in the
object 12 are fluorescent dyes, which are excited by the illumination radiation and emit light into animaging beam path 32. If theobject 12 contains no fluorescent dyes, the illumination beam path is partially reflected by theobject 12 into theimaging beam path 32. Up to thebeam splitter 26, theimaging beam path 32 travels in the opposite direction of theillumination beam path 28. Due to the fact that the fluorescent light has a different wavelength than the illumination radiation, the radiation in theimaging beam path 32 can pass through thedichroic beam splitter 26 and pass to thepinhole device 18, which has a pinhole of adjustable size. Alternatively, a partially reflective beam splitter can be provided instead of thedichroic beam splitter 26, especially if the illumination radiation is reflected by theobject 12. In particular, the pinhole device has an electric motor that continuously or incrementally changes the size of the pinhole. Thepinhole device 18 is connected to thecontrol device 24, with the result that thecontrol device 24 sets the size of the pinhole. - The pinhole is situated in an intermediate image plane of the
imaging beam path 32. Arranged downstream thereof is thedetector device 16, which comprises a detector for converting the radiation of theimaging beam path 32 to electrical signals. The electrical signals are passed on to thecontrol device 24, which can determine therefrom the intensity of the radiation in theimaging beam path 32. - The
laser scanning microscope 10 produces a partially confocal image by setting the pinhole of thepinhole device 18 to be small, for example 50-300% greater than a pinhole that images theobject 12 in diffraction-limited fashion. To produce a wide field image, the pinhole of thepinhole device 18 is further enlarged as compared to the partially confocal imaging, with the result that the image predominantly comprises wide field components. - The method for operating the
laser scanning microscope 10 is explained using the block diagram inFIG. 2 . The method is used to determine a height position of theobject 12 at a first location of theobject 12. The first location can be any location of theobject 12. The method is in particular repeated at a plurality of first locations, such that a plurality of elevations of theobject 12 can be determined, from which the topography of theobject 12 can be derived. - The method comprises three fundamental steps: Step S1 serves for calibrating a scaling factor Sk, step S2 serves for calibrating a linking factor Vk, and step S3 serves for calculating the height position of the first location.
- First, the calibration of the scaling factor Sk in step S1 will be explained. The calibration of the scaling factor Sk can be performed on the
object 12 or on a calibration object that has in particular similar properties with respect to reflection and scattering of light as theobject 12. In the following exemplary embodiment, the calibration of the scaling factor Sk at a calibration location, which is a site on theobject 12, will be discussed. The calibration location is an example of the first calibration location. - In step S1.1, a z-stack of a plurality of partially confocal images is recorded at the calibration location. The images cover the depth of field range of the partially confocal image, wherein the
pinhole device 18 sets a first pinhole size. The intensity is determined for each partially confocal image, such that an intensity distribution IK (z) is obtained in dependence on the measurement position z of the focus in step S1.2. This is illustrated by way of example inFIG. 3 . On the basis of the intensity profile Ik (z), the maximum calibration intensity Ik,max can be determined at the position of the focus zk,max. - In step S1.3, a calibration wide field intensity IKW1 is determined by setting the
pinhole device 18 to a second size for the pinhole, which is greater than the first size. The second size of the pinhole is such that the intensity of the wide field image dominates in the image. The second size is 5 to 10 times greater than the first size, for example. In step S1.4, the scaling factor Sk satisfying the following equation is calculated: -
- In the substep S2.1 of step S2, a point spread function PSF(z) in the z-direction of the
laser scanning microscope 10 is determined. This step is not necessary if the point spread function PSF(z) for thelaser scanning microscope 10 is already known. In step S2.2, partially confocal imaging of the second calibration location is performed on a second calibration object, which can coincide with the first calibration object, or at a second calibration location on theobject 12, optionally coinciding with the first calibration location. The size of the pinhole is set to the first size. The focus of the partially confocal image is situated at a calibration measurement position zk1 which is located in the depth of field range of the partially confocal image. Next, a partially confocal calibration intensity Ik1 of the partially confocal image of the second calibration location is determined at the calibration measurement position zk1. - In step S2.3, confocal imaging of the second calibration location is performed. The focus of the confocal image is likewise situated at the calibration measurement position zk1. Next, a confocal calibration intensity Ik1* of the confocal image of the second calibration location is determined at the calibration measurement position zk1. For confocal imaging, the size of the pinhole of the
pinhole device 18 is set such that it is at imaging diffraction limit. The confocal calibration intensity Ik1* coincides with the intensity of the point spread function PSF(z) at the calibration measurement position zk1. - In step S2.4, a calibration wide field intensity IKW2 is also determined. This is done analogously to the determination of the calibration wide field intensity IKW1 in step S1.3.
- The linking factor Vk is determined in step S2.5 from the partially confocal calibration intensity Ik1, the confocal calibration intensity Ik1* and the calibration wide field intensity IkW2 in accordance with the following equation:
-
I k1 =I* k1 +Vk×I kW2 (2) - The following text explains how the topography in the calibrated
laser scanning microscope 10 is determined in step S3. To this end, in step S3.1, thelaser scanning microscope 10 partially confocally images the first location, wherein the focus is situated at a z measurement position z1 located in the depth of field range. The pinhole is here set to the first size. Next, the partially confocal intensity I1 is determined. From the partially confocal intensity I1, the partially confocal intensity I1* that corresponds to the point spread function PSF is calculated in step S3.2 using the linking factor Vk and the above-described equation (2). The intensity I1* that corresponds to the point spread function PSF corresponds to the intensity of the point spread function PSF at the z measurement position z1. - At the z measurement position z1, further partially confocal imaging is performed in step S3.3, in which the pinhole is changed with respect to the first performed partially confocal imaging. To this end, the pinhole can be embodied to be adjustable in terms of size or pinholes are exchanged. From said further partially confocal imaging using a different pinhole, a further partially confocal intensity I1,pin can be calculated.
- By enlarging the pinhole of the
pinhole device 18, a wide field image of the first location is produced in step S3.4, as described above, and the wide field intensity Iw is determined therefrom. Using equation (1) and the scaling factor Sk, the maximum intensity Imax* of the point spread function PSF is calculated. - From the linking factor Vk, the expected maximum intensity Imax* and the intensity I1* which corresponds to the point spread function PSF, a z-coordinate zh of the focus, which coincides with the z-coordinate of the maximum intensity Imax* of the point spread function PSF, is determined in step S3.5 using equation (2). Since only two values are used for the determination, the determination is not always unique, as is indicated in
FIG. 4 by the two regions in the z-direction Δz1 and Δz1′. Using the further partially confocal intensity I1,pin it is now possible to determine the profile of the point spread function PSF, so that it is possible to establish which of the distances Δz1 and Δz1′ is the correct one, so that the z-position zh can be determined unambiguously. The z-coordinate zh of the focus corresponds to the height position of theobject 12 at the first location. Steps S3.1 to S3.5 are repeated at each of the locations to be measured, whereas steps S1 to S2 are performed only once. - A second embodiment of a microscope for performing the method is schematically illustrated in
FIG. 5 . Here, anAiry microscope 100 is used, the schematic setup of which coincides with thelaser scanning microscope 10, except for the following differences that are shown: - A
positioning device 122 of theAiry microscope 100 does not operate by way of deflecting the radiation of theillumination beam path 28, but by moving theobject 12 with respect to theillumination beam path 28 using a scanner, which forms thepositioning device 122. The scanner moves thesample holder 30. However, it is also possible to use apositioning device 122 as described in thelaser scanning microscope 10. Thepositioning device 122 is again connected to thecontrol device 24 here, such that thecontrol device 24 can set the positioning of theobject 12 with respect to theillumination beam path 28. - A
detector device 116 comprises a plurality of pixels to resolve the diffraction structure of the partially confocal image. - The mode of operation of the
Airy microscope 100 does not differ from the mode of operation of thelaser scanning microscope 10 with respect to the principle of determining the height position at the first location of theobject 12. Only the manner of producing the partially confocal image and the wide field image differs. - Since the diffraction structure can be resolved using the
detector device 116, the intensities of a single partially confocal image that are measured by the pixels comprise the partially confocal intensity I1 and the wide field intensity IW. It is thus possible with a single partially confocal image to determine both intensities I1 and IW that are required for the calculation of the elevation of the first location. - The above-described method for determining the partially confocal intensity I1 (step S3.1) and the partially confocal intensity I1* (step S3.2) which corresponds to the point spread function PSF is repeated at a second measurement position z2, from which the partially confocal intensity I2 and a partially confocal intensity I2* which corresponds to the point spread function PSF are obtained. Further partially confocal imaging with a changed size of the pinhole (step S3.3) is not performed in the
Airy microscope 100, but step S3.5 is modified as follows. - There are now three points of the point spread function PSF, specifically the intensities I1* and I2* corresponding to the point spread function PSF and the maximum value of the point spread function PSF, the intensity Imax*. The PSF, whose form is known, can be fitted on the basis of these values so that the Z coordinate zh of the maximum intensity Imax* can be ascertained. Said z-coordinate zh corresponds to the elevation of the
object 12 at the first location, such that the first location can be ascertained from the two partially confocal images at the measurement positions z1 and z2. - A further embodiment of the microscope will be explained with reference to
FIG. 6 .FIG. 6 schematically illustrates aconfocal topography microscope 200, which differs from thelaser scanning microscope 10 and theAiry microscope 100 merely in the following differences: - The
confocal typography microscope 200 has, analogously to theAiry microscope 100, apositioning device 222 which likewise comprises as a scanner for moving thesample holder 30. The embodiment of the positioning device is not relevant here either. Alight source 214 can be embodied as a laser, a light-emitting diode (LED) or any other monochrome light source, but the use of a white light source, which produces light in the entire visible range, so that theobject 12 is illuminated with illumination radiation in the visible wavelength range, is also possible. - A
detector device 216 has afirst camera 216 a, asecond camera 216 b and asecond beam splitter 216 c. Thesecond beam splitter 216 c is embodied as a partially reflective mirror, which directs 50% of the radiation in theimaging beam path 32 onto thefirst camera 216 a and 50% of the radiation in theimaging beam path 32 onto thesecond camera 216 b. Apinhole device 218 is in the form of a mask having a plurality of gratings having different grating spacings, which is movable laterally through theimaging beam path 32. - The
confocal topography microscope 200 can be used, owing to thesecond beam splitter 216 c, to simultaneously produce an image in wide field using thesecond camera 216 b, in front of which no pinhole is arranged, and a partially confocal image using thefirst camera 216 a. The mode of operation to this extent coincides with the mode of operation of theAiry microscope 100, since here a partially confocal image and an image in wide field are also produced at the same time.
Claims (14)
1. A method for determining a height position of an object at a lateral first location of the object by using a microscope, which images the object with a point spread function along a z-direction that coincides with the height direction, the method comprising the following steps:
imaging the object with the microscope in wide field and determining a wide field intensity,
calculating a maximum intensity expected at the first location by multiplying the wide field intensity with a predetermined scaling factor,
partially confocally imaging the object at the first location with the focus located at a measurement position in the z-direction, wherein a pinhole device having an opening which is greater than a diffraction limit is used for the partially confocal imaging, and wherein the partially confocal imaging defines a depth of field range and the focus is within the depth of field range, and determining a partially confocal intensity of the partially confocal image at the first location,
calculating an intensity corresponding to the point spread function at the first location by way of subtraction between the partially confocal intensity and the product of the wide field intensity and a predetermined linking factor,
computing a z-coordinate of the focus at which the point spread function is maximum using a predetermined form of the point spread function, the calculated intensity corresponding to the point spread function and the calculated expected maximum intensity, and
using the z-coordinate as the height position of the object at the first location.
2. The method as claimed in claim 1 , wherein the scaling factor is determined by performing the following steps:
producing a z-stack by repeatedly shifting the focus position in the z-direction and partially confocally imaging a lateral first calibration location of a first calibration object or of the object
at each focus position, and determining the partially confocal intensity for each of the partially confocal images thus obtained,
calculating a z-direction intensity profile at the first calibration location, and determining a maximum calibration intensity of the intensity profile at the first calibration location,
imaging the first calibration location with the microscope in wide field, and determining a first calibration wide field intensity at the first calibration location, and
calculating the scaling factor as a ratio between the maximum calibration intensity and the first calibration wide field intensity.
3. The method as claimed in claim 1 , wherein the linking factor is determined by performing the following steps:
partially confocal imaging a lateral second calibration location of a second calibration object or of the object with the focus located at a calibration measurement position in the z-direction which position is within the depth of field range, and determining a calibration intensity of the partially confocal image at the second calibration location,
confocally imaging the second calibration location with the focus located at the calibration measurement position, and determining a confocal calibration intensity of the confocal image at the second calibration location,
imaging the second calibration location with the microscope in wide field, and determining a second calibration wide field intensity at the second calibration location, and
calculating the linking factor as a ratio between the difference between the partially confocal calibration intensity and the confocal calibration intensity to the second second calibration wide field intensity.
4. The method as claimed in claim 1 , wherein the first location is imaged with the microscope in wide field and in that the wide field intensity is determined at the first location.
5. The method as claimed in claim 1 , wherein the following steps:
partially confocally imaging the first location with the focus located at least at two measurement positions which are spaced apart in the z-direction and are situated both in the depth of field range, and determining partially confocal intensities for each partially confocal image at the first location,
calculating the intensities corresponding to the point spread function at the first location as a difference between the partially confocal intensity and a product of wide field intensity and predetermined linking factor,
computing the z-coordinate using the form of the point spread function, the calculated intensities corresponding to the point spread function and the calculated expected maximum intensity.
6. The method as claimed in claim 1 , wherein for producing a live determination of a height position in the object at a laterally second location of the object, which differs from the first one, the second location is imaged in partially confocal fashion and in that, for calculating a z-coordinate at the second location, the z-coordinate of the focus at the first location is taken into consideration.
7. The method as claimed in claim 1 , further comprising the following steps:
partially confocally imaging the first location with at least two different pinholes and with the focus at a measurement position in the z-direction which is situated within the depth of field range, and determining a further partially confocal intensity for the partially confocal image with the further pinhole,
calculating the z-coordinate using the form of the point spread function, the calculated intensity corresponding to the point spread function, the further partially confocal intensity and the calculated expected maximum intensity.
8. The method as claimed in claim 1 , wherein a laser scanning microscope is used as the microscope, and wherein the size of the pinhole is changed for partially confocal imaging and for imaging in wide field.
9. The method as claimed in claim 1 , wherein a laser scanning microscope is used as the microscope, wherein, for partially confocal imaging and for imaging in wide field, two detectors are provided behind pinholes of different size or wherein a wide field camera is used for imaging in wide field.
10. The method as claimed in claim 1 , wherein a confocal topography microscope is used as the microscope, in which a grating is used as a pinhole, wherein a first camera with the grating is used for partially confocal imaging and a second camera is used for imaging in wide field.
11. The method as claimed in claim 1 , wherein a confocal Airy microscope is used as the microscope, in which the object is imaged onto a detector device which comprises a plurality of pixels and resolves a diffraction structure of the partially confocal image, wherein the wide field intensity and the partially confocal intensity are determined from one recording or wherein a wide field camera is used for imaging in wide field.
12. A microscope for producing a partially confocal image of an object and an image of the object in wide field, comprising:
a detector device,
a pinhole device for partially confocal imaging and recording in wide field,
a focusing device, which is embodied for setting a z-position of a focus of the partially confocal image, and
a control device for controlling the pinhole device, which is connected to the detector device,
wherein the microscope images the object in a z-direction, which coincides with the height direction of the object, with a point spread function,
wherein the control device is configured to carry out the method as claimed in claim 1 .
13. The microscope as claimed in claim 12 , wherein a size of a pinhole of the pinhole device is adjustable.
14. The microscope as claimed in claim 12 , wherein the detector device comprises a first detector for partially confocal imaging and a second detector for imaging in wide field.
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102016103736.2A DE102016103736A1 (en) | 2016-03-02 | 2016-03-02 | Method for determining an altitude of an object |
DE102016103736.2 | 2016-03-02 | ||
PCT/EP2016/080771 WO2017148555A1 (en) | 2016-03-02 | 2016-12-13 | Method for determining a height position of an object |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2016/080771 A-371-Of-International WO2017148555A1 (en) | 2016-03-02 | 2016-12-13 | Method for determining a height position of an object |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/236,803 Continuation US11287629B2 (en) | 2016-03-02 | 2021-04-21 | Method for determining a height position of an object |
Publications (1)
Publication Number | Publication Date |
---|---|
US20200264415A1 true US20200264415A1 (en) | 2020-08-20 |
Family
ID=57543044
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/061,639 Abandoned US20200264415A1 (en) | 2016-03-02 | 2016-12-13 | Method for determining a height position of an object |
US17/236,803 Active US11287629B2 (en) | 2016-03-02 | 2021-04-21 | Method for determining a height position of an object |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/236,803 Active US11287629B2 (en) | 2016-03-02 | 2021-04-21 | Method for determining a height position of an object |
Country Status (6)
Country | Link |
---|---|
US (2) | US20200264415A1 (en) |
EP (1) | EP3345032B1 (en) |
CN (1) | CN108700732B (en) |
DE (1) | DE102016103736A1 (en) |
ES (1) | ES2741809T3 (en) |
WO (1) | WO2017148555A1 (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102017129837A1 (en) * | 2017-12-13 | 2019-06-13 | Leibniz-Institut für Photonische Technologien e. V. | Combined examination with imaging and laser measurement |
DE102018210606A1 (en) * | 2018-06-28 | 2020-01-02 | Carl Zeiss Microscopy Gmbh | Microscopic imaging method using a correction factor |
CN111095024A (en) * | 2018-09-18 | 2020-05-01 | 深圳市大疆创新科技有限公司 | Height determination method, height determination device, electronic equipment and computer-readable storage medium |
CN110017968B (en) * | 2019-04-19 | 2020-10-02 | 北京理工大学 | High-chromatography anti-scattering laser differential confocal chromatography focusing method and device |
Citations (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5671085A (en) * | 1995-02-03 | 1997-09-23 | The Regents Of The University Of California | Method and apparatus for three-dimensional microscopy with enhanced depth resolution |
US5915048A (en) * | 1996-06-05 | 1999-06-22 | Zetetic Institute | Method and apparatus for discriminating in-focus images from out-of-focus light signals from background and foreground light sources |
US20090263002A1 (en) * | 2008-04-17 | 2009-10-22 | Ruprecht-Karls-Universitat Heidelberg | Wave field microscope with sub-wavelength resolution and methods for processing microscopic images to detect objects with sub-wavelength dimensions |
US20110267688A1 (en) * | 2009-10-28 | 2011-11-03 | Ingo Kleppe | Microscopy Method and Microscope With Enhanced Resolution |
US20120081535A1 (en) * | 2010-09-30 | 2012-04-05 | Olympus Corporation | Sample observation device for generating super resolution image |
US20120257197A1 (en) * | 2008-06-16 | 2012-10-11 | The Regents Of The University Of Colorado, A Body Corporate | Fourier domain sensing |
US20130308842A1 (en) * | 2011-01-28 | 2013-11-21 | Cornell University | Computer-implemented platform for automated fluorescence imaging and kinetic analysis |
US20140099007A1 (en) * | 2012-10-09 | 2014-04-10 | The United States Of America, As Represented By The Secretary, Department Of Health And Human Serv | Imaging methods and computer-readable media |
US20150317824A1 (en) * | 2014-05-02 | 2015-11-05 | Olympus Corporation | Sample image data generating apparatus and sample image data generating method |
US20160143517A1 (en) * | 2013-06-19 | 2016-05-26 | Optiscan Pty Ltd. | Optical scanner and scanned lens optical probe |
US20160238827A1 (en) * | 2012-02-23 | 2016-08-18 | The U.S.A. of America, as represented by The Secretary, Dept. of Health and Human Services | Multi-focal structured illumination microscopy systems and methods |
US9426455B2 (en) * | 2013-07-31 | 2016-08-23 | California Institute Of Technology | Aperture scanning fourier ptychographic imaging |
US9497379B2 (en) * | 2013-08-22 | 2016-11-15 | California Institute Of Technology | Variable-illumination fourier ptychographic imaging devices, systems, and methods |
US20170108682A1 (en) * | 2014-03-24 | 2017-04-20 | Carl Zeiss Microscopy Gmbh | Confocal microscope with aperture correlation |
US20170307440A1 (en) * | 2014-09-25 | 2017-10-26 | Northwestern University | Devices, methods, and systems relating to super resolution imaging |
US9864184B2 (en) * | 2012-10-30 | 2018-01-09 | California Institute Of Technology | Embedded pupil function recovery for fourier ptychographic imaging devices |
US10162161B2 (en) * | 2014-05-13 | 2018-12-25 | California Institute Of Technology | Ptychography imaging systems and methods with convex relaxation |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6341035B1 (en) * | 1999-07-09 | 2002-01-22 | Olympus Optical Co., Ltd. | Confocal microscope |
JP2001083432A (en) * | 1999-07-09 | 2001-03-30 | Olympus Optical Co Ltd | Confocal microscope |
TW555954B (en) * | 2001-02-28 | 2003-10-01 | Olympus Optical Co | Confocal microscope, optical height-measurement method, automatic focusing method |
US20040008515A1 (en) * | 2002-06-11 | 2004-01-15 | Applied Precision | Fluorescence illumination optimization for three-dimensional microscopy |
-
2016
- 2016-03-02 DE DE102016103736.2A patent/DE102016103736A1/en active Pending
- 2016-12-13 ES ES16809812T patent/ES2741809T3/en active Active
- 2016-12-13 CN CN201680081444.XA patent/CN108700732B/en active Active
- 2016-12-13 US US16/061,639 patent/US20200264415A1/en not_active Abandoned
- 2016-12-13 WO PCT/EP2016/080771 patent/WO2017148555A1/en active Application Filing
- 2016-12-13 EP EP16809812.7A patent/EP3345032B1/en active Active
-
2021
- 2021-04-21 US US17/236,803 patent/US11287629B2/en active Active
Patent Citations (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5671085A (en) * | 1995-02-03 | 1997-09-23 | The Regents Of The University Of California | Method and apparatus for three-dimensional microscopy with enhanced depth resolution |
US5915048A (en) * | 1996-06-05 | 1999-06-22 | Zetetic Institute | Method and apparatus for discriminating in-focus images from out-of-focus light signals from background and foreground light sources |
US20090263002A1 (en) * | 2008-04-17 | 2009-10-22 | Ruprecht-Karls-Universitat Heidelberg | Wave field microscope with sub-wavelength resolution and methods for processing microscopic images to detect objects with sub-wavelength dimensions |
US20120257197A1 (en) * | 2008-06-16 | 2012-10-11 | The Regents Of The University Of Colorado, A Body Corporate | Fourier domain sensing |
US8705172B2 (en) * | 2009-10-28 | 2014-04-22 | Carl Zeiss Microscopy Gmbh | Microscopy method and microscope with enhanced resolution |
US20110267688A1 (en) * | 2009-10-28 | 2011-11-03 | Ingo Kleppe | Microscopy Method and Microscope With Enhanced Resolution |
US20120081535A1 (en) * | 2010-09-30 | 2012-04-05 | Olympus Corporation | Sample observation device for generating super resolution image |
US20130308842A1 (en) * | 2011-01-28 | 2013-11-21 | Cornell University | Computer-implemented platform for automated fluorescence imaging and kinetic analysis |
US20160238827A1 (en) * | 2012-02-23 | 2016-08-18 | The U.S.A. of America, as represented by The Secretary, Dept. of Health and Human Services | Multi-focal structured illumination microscopy systems and methods |
US20140099007A1 (en) * | 2012-10-09 | 2014-04-10 | The United States Of America, As Represented By The Secretary, Department Of Health And Human Serv | Imaging methods and computer-readable media |
US9864184B2 (en) * | 2012-10-30 | 2018-01-09 | California Institute Of Technology | Embedded pupil function recovery for fourier ptychographic imaging devices |
US20160143517A1 (en) * | 2013-06-19 | 2016-05-26 | Optiscan Pty Ltd. | Optical scanner and scanned lens optical probe |
US9426455B2 (en) * | 2013-07-31 | 2016-08-23 | California Institute Of Technology | Aperture scanning fourier ptychographic imaging |
US9497379B2 (en) * | 2013-08-22 | 2016-11-15 | California Institute Of Technology | Variable-illumination fourier ptychographic imaging devices, systems, and methods |
US20170108682A1 (en) * | 2014-03-24 | 2017-04-20 | Carl Zeiss Microscopy Gmbh | Confocal microscope with aperture correlation |
US20150317824A1 (en) * | 2014-05-02 | 2015-11-05 | Olympus Corporation | Sample image data generating apparatus and sample image data generating method |
US10162161B2 (en) * | 2014-05-13 | 2018-12-25 | California Institute Of Technology | Ptychography imaging systems and methods with convex relaxation |
US20170307440A1 (en) * | 2014-09-25 | 2017-10-26 | Northwestern University | Devices, methods, and systems relating to super resolution imaging |
Also Published As
Publication number | Publication date |
---|---|
US20210239953A1 (en) | 2021-08-05 |
DE102016103736A1 (en) | 2017-09-07 |
EP3345032B1 (en) | 2019-06-12 |
CN108700732B (en) | 2019-10-11 |
EP3345032A1 (en) | 2018-07-11 |
CN108700732A (en) | 2018-10-23 |
ES2741809T3 (en) | 2020-02-12 |
WO2017148555A1 (en) | 2017-09-08 |
US11287629B2 (en) | 2022-03-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11287629B2 (en) | Method for determining a height position of an object | |
US10725279B2 (en) | Systems and methods for extended depth-of-field microscopy | |
JP6282706B2 (en) | High resolution microscope and method for 2D or 3D positioning of an object | |
US8773757B2 (en) | Slit-scan multi-wavelength confocal lens module and slit-scan microscopic system and method using the same | |
US7274446B2 (en) | Method and arrangement for the deep resolved optical recording of a sample | |
US7977625B2 (en) | Method and assembly for optical reproduction with depth discrimination | |
US7477401B2 (en) | Trench measurement system employing a chromatic confocal height sensor and a microscope | |
CN108254909B (en) | Optical microscope and method for recording images using an optical microscope | |
US20090021750A1 (en) | Method and Arrangement for a Rapid and Robust Chromatic Confocal 3D Measurement Technique | |
US20020104961A1 (en) | Scanning microscope and module for a scanning microscope | |
US9804029B2 (en) | Microspectroscopy device | |
JP6061958B2 (en) | Software-defined microscope | |
JP2013525838A (en) | Device for imaging the sample surface | |
JP2008268387A (en) | Confocal microscope | |
CN107430266A (en) | Method for carrying out the micro- detection of mating plate to sample | |
US20040021936A1 (en) | Microscope with and automatic focusing device | |
JP6595618B2 (en) | Method for determining spatially resolved height information of a sample using a wide field microscope and a wide field microscope | |
JP2007506955A (en) | Scanning microscope with evanescent wave illumination | |
JP6363477B2 (en) | 3D shape measuring device | |
KR20150044291A (en) | Automatic focus control apparatus and automatic focus control method using the same | |
KR20150059147A (en) | method for obtaining 3D images using chromatic aberration of lens and 3D microscope using thereof | |
CN113008134A (en) | Measuring device, optical microscope and measuring method for measuring imaging depth | |
JP2020064127A (en) | Measurement method and measurement device | |
US20220252856A1 (en) | Method and device for determining the optimal position of the focal plane for examining a specimen by microscopy | |
US20240046595A1 (en) | Method, device and non-transitory computer-readable medium for localizing individual emitters in a sample |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: CARL ZEISS MICROSCOPY GMBH, GERMANY Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LANGHOLZ, NILS;DRESCHER, VIKTOR;SIGNING DATES FROM 20180515 TO 20180516;REEL/FRAME:046121/0503 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |