EP3566089A1 - Forming images of a sample using scanning microscopy - Google Patents

Forming images of a sample using scanning microscopy

Info

Publication number
EP3566089A1
EP3566089A1 EP18700268.8A EP18700268A EP3566089A1 EP 3566089 A1 EP3566089 A1 EP 3566089A1 EP 18700268 A EP18700268 A EP 18700268A EP 3566089 A1 EP3566089 A1 EP 3566089A1
Authority
EP
European Patent Office
Prior art keywords
emission light
optical
emission
sample
light signal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP18700268.8A
Other languages
German (de)
French (fr)
Inventor
Erik MANDERS
Ronald BREEDIJK
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ConfocalNl BV
Original Assignee
ConfocalNl BV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ConfocalNl BV filed Critical ConfocalNl BV
Publication of EP3566089A1 publication Critical patent/EP3566089A1/en
Withdrawn legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/0004Microscopes specially adapted for specific applications
    • G02B21/002Scanning microscopes
    • G02B21/0024Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/0052Optical details of the image generation
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/0004Microscopes specially adapted for specific applications
    • G02B21/002Scanning microscopes
    • G02B21/0024Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/0052Optical details of the image generation
    • G02B21/0064Optical details of the image generation multi-spectral or wavelength-selective arrangements, e.g. wavelength fan-out, chromatic profiling
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/0004Microscopes specially adapted for specific applications
    • G02B21/002Scanning microscopes
    • G02B21/0024Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/0052Optical details of the image generation
    • G02B21/0072Optical details of the image generation details concerning resolution or correction, including general design of CSOM objectives
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/0004Microscopes specially adapted for specific applications
    • G02B21/002Scanning microscopes
    • G02B21/0024Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/0052Optical details of the image generation
    • G02B21/0076Optical details of the image generation arrangements using fluorescence or luminescence
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/0004Microscopes specially adapted for specific applications
    • G02B21/002Scanning microscopes
    • G02B21/0024Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
    • G02B21/008Details of detection or image processing, including general computer control

Definitions

  • the invention relates to forming images using scanning microscopy and, in particular, though not exclusively, to systems and methods for forming images of a sample using scan ⁇ ning microscopy, a computer system for controlling such systems and computer program product for executing such meth- ods .
  • Rescan confocal microscopy is a known super- resolution microscopy technique for obtaining high-resolution images using conventional refractive and reflective optical elements.
  • the principles of RCM are explained in the article by De Luca et al . , "Re-scan confocal microscopy: scanning twice for better resolution” , Biomed Opt Express; 2013 Nov 1 ; 4(11) : 2644-2656.
  • a high resolution colour image of an area of a sample can be obtained by scanning the area line by line with an illumination light beam of two different wavelengths and rescanning the light originating from the area onto an image sensor so that an image of the excited area is obtained.
  • a problem related to the colour rescan confocal microscopy technique as described by De Luca is that it is less suitable for time-critical measurements which typically require (almost) simultaneous acquisition of different images associated with different optical information.
  • the article by De Luca reported reduction of the time difference between two colour scans to the time of a single scanning line, the fact that the line images are not of the same time instance, imaged features and/or objects in different images may have moved and/or changed so that combining the images will cause artefacts in the final colour image.
  • fluorescence or luminescence lifetime imaging microscopy the luminescence lifetime properties of a sample can be obtained by exposing a sample with modulated light and by measuring multiple images of the modulated emission light.
  • the line by line scanning technique described in De Luca however is not suitable for reliably measuring lifetime properties of a sample .
  • WO 2015/164843 discloses a multifocal structured illumination microscopy system having two scanning mirrors and two cameras.
  • a sample that is positioned at a focus is scanned with excitation light using a scanning mirror.
  • the sample emits emission light back onto the scanning mirror which reflects the emission light onto a second scanning mirror, which scans the emission light onto the two cameras.
  • the emission light from the second scanning mirror passes through a dichroic mirror that splits the emission light towards the two cameras. This enables each camera to separately detect a different light color.
  • a problem related to this system is that movement of the emission light beam with respect to the dichroic mirror causes the emission light to be incident on the mirror under a varying angle. This varying angle influences the optical interactions between the dichroic mirror and the light beam, which negatively impacts the quality of the images captured by the cameras.
  • scanning microscopy that allows for increasing the amount of information that can be obtained with one scan cycle. This for example allows to reduce the number of required scans and to increase scan speed. It may also beneficially eliminate a time gap between measurements.
  • the invention may relate to a method for forming images of a sample using scanning microscopy.
  • the method may include controlling a scanning mirror of a first optical system to move (scan) one or more focused illumination light signals over (or through) an optical active sample, wherein the one or more focused
  • illumination light signal cause one or more optical
  • the first optical system forming at least one emission light signal from
  • emission light originating from the one or more optical excitations and directing the emission light signal onto the scanning mirror for reflecting the emission light signal via a second optical system to an imaging system; a light splitter in the second optical system splitting the at least one emission light signal in at least a first emission light signal comprising first optical information and second
  • emission light signal comprising second optical information, preferably the first optical information being different from the second optical information; and, the second optical system directing the first and second emission light signals onto a rescanning mirror, the rescanning mirror being controlled to move (scan) a focused emission light beam signals over the imaging plane of the imaging system for simultaneously forming a first image associated with the first optical information and second image associated with the second optical
  • the invention enables a scanning microscope to determine image data, i.e. multiple different images,
  • the invention thus enables a scanning microscope to measure and image time-critical processes, such as color images, time-dependent processes and/or life-time
  • the angle under which light falls onto the light splitter can be kept constant. Consequently, the optical interactions between the incident light and the splitter are also constant.
  • the cutoff wavelength of the dichroic mirror remains the same during an experiment, which ensures that any variation in light intensity detected by the imaging system corresponds to an intensity variation of the light emission signal and not to a variation of the cutoff wavelength of the dichroic mirror. In the latter case, a formed image would become inaccurate as it would contain intensity variations that do not correspond with variations in density of
  • fluorophores in the sample may reduce such distortions and increase the quality of the images that are generated by the scanning microscope in one scanning cycle.
  • the one or more excitations comprise at least a first type of excitation causing emission of a first wavelength or a first wavelength band and a second type of excitation causing emission of a second wavelength or second wavelength band different from the first wavelength or first wavelength band.
  • the first optical information may be associated with the emission of the first wavelength or a first wavelength band and the second optical information may be associated with the emission of the second wavelength or second wavelength band.
  • the method further comprise: the capturing first image data associated with the first emission light and a second image data associated with the second emission light; and, forming an image of at least part of the sample by comprising at least part of the first and second image data.
  • the method may further comprise: modulating the intensity of the at least one illumination light signal in accordance with a modulation frequency, wherein the modulation of the illumination light signal may cause a modulation of the intensity of the first and second emission light signal.
  • the method may further comprise phase-shifting the modulated second emission light signal with respect to the modulated first emission light signal.
  • the phase-shifting may include directing the modulated second emission light signal through an optical delay line.
  • the modulation frequency may be selected between 1 and 200 MHz.
  • the method may further comprise: the imaging system simultaneously capturing first image data associated with the modulated first emission light signal and second image data associated with the time-shifted modulated second emission light signal.
  • the method may further comprise a processor determining a life-time associated with at least one of the one or more optical excitations on the basis of at least part of the first image data and at least part of the second image data.
  • the method may comprise focusing at least part of the emission light signal upon a pinhole
  • the focusing may include using at least one first optical element for focusing the emission light signal originating from the scanning mirror onto the pinhole aperture and at least one second optical element configured to form a collimated emission light signal of the emission light that passes pinhole and to direct the
  • the one or more focused image is displayed in an embodiment.
  • illumination light signals may comprise at least a first focused illumination light spot at a first position of a focusing plane in the sample and a second focused illumination light spot at a second position in the focusing plane in the sample, preferably the first illumination light being
  • the first emission light signal comprises emission light of the first optical
  • the second emission light beam comprises emission light of the second optical excitation.
  • the one or more focused image is displayed in an embodiment.
  • illumination light signals may comprise at least a first focused illumination light spot at a first position in a first focusing plane in the sample and a second focused illumination light spot at a second position in a second focusing plane in the sample.
  • controlling the scanning and the rescanning mirror may include rotating the scanning mirror and the rescanning mirror back and forth over a predetermined first angular amplitude and a second angular amplitude
  • the second angular amplitude of the rescanning mirror may be selected larger than the first angular amplitude of the scanning mirror.
  • the second angular amplitude may be selected as two times the first angular amplitude.
  • the invention may relate to a method for forming images using scanning microscopy
  • the first optical system forming a modulated emission light signal from modulated emission light originating from the optical excitations and directing the modulated emission light signal onto the scanning mirror for reflecting the emission light signal via a second optical system to an imaging system; a light splitter in the second optical system splitting the modulated emission light signal in at least a first modulated emission light signal and second modulated emission light signal representing a phase-shifted version of the first modulated light signal; and, the light splitter directing the first and second modulated emission light signals onto a rescanning mirror, which is controlled to move (scan) focused emission light signals over the imaging plane of the imaging system in order to simultaneously form a first image of the first modulated emission light signal and second image of the second modulated emission light signal, while the one or more focused modulated illumination light signals are scanned over (or through) the sample.
  • the first optical system forming a modulated emission light signal from modulated emission light originating from the optical excitations and directing the modulated emission light signal onto the scanning mirror for reflecting the emission light signal via
  • the invention may relate to a scanning microscopy system for forming images of a sample
  • the system may comprise: a light source, preferably one or more lasers, configured for generating light for illuminating the sample; a first optical system comprising a scanning mirror, the optical system being configured: to focus illumination light of the light source as one or more
  • a second optical system comprising a
  • the light splitter being configured: to split the at least one emission light signal originating from the scanning mirror in at least a first emission light signal comprising first optical
  • second optical system being configured: to direct the first and second emission light signals towards the rescanning mirror and to focus the first and second emission light signals onto an imaging plane;
  • an imaging system configured to receive first and second emission light signals originating from the rescanning mirror; and, a computer system being adapted to: control the scanning mirror of the first optical system to move (scan) the one or more focused illumination light signals over (or through) an optical active sample; control the rescanning mirror of the second optical system to move (scan) a focused emission light beam signal over the imaging plane of the imaging system; and, control the imaging system for simultaneously capturing first image data associated with the first optical information and second image data associated with the second optical
  • the one or more excitations comprise at least a first type of excitation causing emission of a first wavelength or a first wavelength band and a second type of excitation causing emission of a second wavelength or second wavelength band different from the first wavelength or first wavelength band; and, wherein the first optical
  • the second optical information is associated with the emission of the second wavelength or second wavelength band.
  • the system may further comprise: a modulator for modulating the intensity of the at least one illumination light signal in accordance with a modulation frequency, preferably the modulation frequency being selected between 1 and 200 MHz; the modulation of the illumination light signal causing a modulation of the intensity of the first and second emission light signal; a phase-shifter, preferably an optical delay line, configured to phase-shift the modulated second emission light signal with respect to the modulated first emission light signal.
  • a modulator for modulating the intensity of the at least one illumination light signal in accordance with a modulation frequency, preferably the modulation frequency being selected between 1 and 200 MHz
  • the modulation of the illumination light signal causing a modulation of the intensity of the first and second emission light signal
  • a phase-shifter preferably an optical delay line, configured to phase-shift the modulated second emission light signal with respect to the modulated first emission light signal.
  • the system may further comprise the imaging system further being configured to simultaneously capture first image data associated with the modulated first emission light signal and second image data associated with the time-shifted modulated second emission light signal; and, the computer system further being adapted to: determine a life-time associated with at least one of the one or more optical excitations on the basis of at least part of the first image data and at least part of the second image data.
  • the invention may relate to a computer system for controlling scanning microscopy system for forming images of a sample
  • the system comprising a light source, preferably one or more lasers, configured for generating light for illuminating the sample; a first optical system comprising a scanning mirror, the optical system being configured to focus illumination light of the light source as one or more illumination light signals via the scanning mirror onto the sample, the one or more focused illumination light signals causing one or more optical excitations in the optical active sample; to form an emission light signal from emission light originating from the one or more optical excitations in the sample; and, to direct the emission light signal onto the scanning mirror; a second optical system comprising a
  • the light splitter being configured to split the at least one emission light signal originating from the scanning mirror in at least a first emission light signal comprising first optical
  • second emission light signal comprising second optical information, the second optical system being
  • the computer system comprises a computer readable storage medium having computer readable program code embodied
  • a processor preferably a microprocessor, coupled to the computer readable storage medium, wherein responsive to executing the first computer readable program code, the processor is configured to perform executable operations comprising:
  • controlling the scanning mirror of the first optical system to move (scan) the one or more focused illumination light signals over (or through) an optical active sample
  • controlling the imaging system to simultaneously first image data associated with the first optical information and second image data associated with the second optical information, while the sample is scanned with the one or more illumination light signals.
  • the processor may be further configured to perform an executable operation comprising:
  • modulation frequency being selected between 1 and 200 MHz; the modulation of the illumination light signal causing a
  • the imaging system simultaneously capturing first image data associated with the modulated first emission light signal and second image data associated with a time-shifted version of the modulated first emission light signal;
  • the invention also relates to computer program product for carrying out the methods described herein, as well as a non-transitory computer readable storage-medium storing the computer program.
  • a computer program may, for example, be downloaded (updated) to an existing control device of the scanning microscopy system or be stored upon manufac ⁇ turing of this control device.
  • Fig. 1 shows a scanning microscopy system according to an embodiment of the invention
  • FIG. 2A-2C schematically depict the scanning of an illumination light spot and emission light spots by a scanning microscopy system according to an embodiment of the invention
  • Fig. 3 depicts a flow diagram of a scanning process according to an embodiment of the invention
  • FIG. 4 depicts a scanning microscopy system according to another embodiment of the invention.
  • Fig. 5A-5C depicts a data acquisition process for a scanning microscopy system according to an embodiment of the invention . ;
  • Fig. 6 depicts a flow diagram of a scanning process according to an embodiment of the invention.
  • Fig. 7 depicts a scanning microscopy system according to another embodiment
  • Fig. 8 depicts a scanning microscopy system according to yet another embodiment
  • Fig. 9 depicts a scanning microscopy system according to a further embodiment
  • DETAILED DESCRIPTION OF THE DRAWINGS Fig. 1 shows a scanning microscopy system according to an embodiment of the invention.
  • the system may comprise a light source 102 for generating illumination light that is directed onto a sample 116 located on a sample holder (not shown) .
  • Illumination light 106 may be directed onto a sample using a first optical system 132, which may include one or more refractive and/or reflective optical elements.
  • the first optical system may include one or more collimating and/or focusing lenses
  • a lens 106 may be used in order to from a collimated beam of illumination light 106 which may be reflected via a set of mirrors 108,110,112 towards a lens 114 in order to form a focused illumination light spot 117 on the sample.
  • the focused illumination light spot may excite local optical excitations, which generate emission light 118.
  • the emission light may pass through lens 114 and may be directed via a second optical system 136 comprising one or more
  • refractive and/or reflective optical elements to an imaging system 124.
  • the second optical system may include a lens 114 for forming a collimated beam of the emission light that is reflected by mirror 112.
  • the beam of emission light may pass one or more optical elements, e.g. dichroic mirrors 110,128, for splitting the emission light beams into different emission light beams before it is reflected by a further mirror 120, which directs the emission light beams to a focussing lens 122.
  • the focussing lens may focus the emission light beams into focussed emission light spots 126a, 126b onto the imaging plane of imaging system 124.
  • the imaging system may include one or more image sensors, e.g. one or more CMOS image sensors or one or more CCDs.
  • Dichroic mirror 110 may be configured such that it functions as a reflector for
  • Mirror 112 which reflects the illumination light towards the sample and the emission light towards the further mirror 120 may be configured as a rotatable mirror which can be controlled by a computer system (not shown) .
  • This mirror may be referred to as a scanning mirror.
  • the scanning mirror may be configured to move the focused illumination light spot 117 over or through the sample 116.
  • scanning mirror 112 may be configured to rotate back and forth over a first angular amplitude Al 138 in order to move (“scan") the focused illumination light spot over or through sample causing local optical illumination along its way.
  • the rotatable scanning mirror may be controlled by a number of computer-controlled actuators. This way the rotatable scanning mirror is used in order to scan illumination light spot over the surface of the sample in accordance to a particular scanning pattern.
  • a scanning pattern may be a meandering pattern. The scanning pattern may be configured so that after scanning a particular area, the scanned area is fully exposed by the moving illumination light spot .
  • emission light originating from the moving illumination light spot is split into multiple
  • collimated beams may be reflected via the scanning mirror 112 towards the further mirror 120, for example a rotatably mounted mirror, which is controlled to reflect the light beams towards the focusing lens in order form multiple moving emission light spots 126a, 126b onto the imaging plane of the imaging system 124.
  • the further mirror 120 for example a rotatably mounted mirror, which is controlled to reflect the light beams towards the focusing lens in order form multiple moving emission light spots 126a, 126b onto the imaging plane of the imaging system 124.
  • the further mirror 120 may be configured to rotate back and forth over a second angular amplitude A2 140 in order to move (scan) the focused emission light spots over the imaging plane of the image sensor, while the focussed
  • mirror 120 may hereafter be referred to as the rescanning mirror.
  • the actuators and/or electro-motors of the scanning and rescanning mirror may be controlled by a computer system so that the mirrors can be moved in sync.
  • the frequency of the back and forth rotation of the scanning mirror is identical to the frequency of the back and forth rotation of the rescanning mirror. This way, when the scanning mirror is controlled by the computer system to scan an area of the sample by the moving
  • the rescanning mirror is controlled by the computer system to scan associated areas of pixels of the imaging plane of the image sensor so that the pixels are exposed by moving emission lights spots. This way, when an area of a sample is optically excited by the illumination beam, emission light originating from the excited area is imaged onto the imagining plane.
  • the second angular amplitude of the rescanning mirror may be larger than the first angular
  • the second angular amplitude may be twice the first angular amplitude, so that the resolution of the image is improved by a factor with respect to the resolution that may be obtained when the second angular amplitude is chosen to be equal to the first angular
  • the light source 102 may comprise a plurality of light sources, e.g. a plurality of lasers, and one or more light filters.
  • the light sources and filters may be used to control which wavelengths the illumination light comprises.
  • the light source may be configured to generate white light.
  • the light source may be configured to generate light consists of two or more
  • the light source may generate light of a first wavelength selected from the blue band of the visible spectrum and light of a second wavelength selected from the yellow band of the visible spectrum.
  • the optical elements of optical system 132 may be configured to control different wavelengths.
  • mirror 108 may reflect the illumination light onto dichroic mirror 110 that may be configured to reflect light of a first group of wavelengths, e.g. blue light and yellow light.
  • Dichroic mirror 110 may be further configured to pass light of a second group of wavelengths, e.g. red and green light. This way, the illumination light may be reflected by dichroic mirror 110 onto scanning mirror 112. Scanning mirror may reflect the illumination light onto lens 114 which may focus the illumination light onto the sample 116 creating a focused illumination light spot 117.
  • a second group of wavelengths e.g. red and green light.
  • the illumination light may comprise light of a first wavelength, e.g. yellow light, which may cause a first type of optical excitations in the sample, the first type of excitations generating first emission light.
  • a first wavelength e.g. yellow light
  • the illumination light may comprise light of a second wavelength, e.g. blue light, which may cause a second type of optical excitations in the sample, the second type of
  • first group of wavelengths which may be reflected by dichroic mirror 110, may comprise said first wavelength and said second wavelength.
  • sample may be sample, e.g. a biological material, that is imaged using a reflective
  • sample 116 may be an optically active sample.
  • the sample may be a material, e.g. a biological material, comprising one or more types of fluorescent /luminescent materials, such as
  • the optically active sample may be non-luminescent optical active material.
  • the sample may be a material that can be imaged on the basis of second harmonics generation (SHG) microscopy or third harmonics generation (THG) microscopy.
  • the sample may be a material that can be imaged using a Rahman microscopy technique, such as Coherent Anti- Stokes Rahman Scattering (CARS) microscopy.
  • the sample comprise at least a first type of fluorophores, e.g. red fluorophores, and a second type of fluorophores, e.g. green fluorophores .
  • the light of the first wavelength of the illumination light may cause the first fluorophores in the sample to emit the first emission light and the light of the second wavelength may cause the second fluorophores in the sample to emit second emission light.
  • the emission light may comprise light that is emitted as a result of photoluminescence, preferably
  • a blue light component of the illumination light may cause green fluorophores at the focused illumination light spot to emit green light.
  • yellow light component of the illumination light may cause red fluorophores at the focused illumination light spot to emit red light.
  • the emission light thus comprises green light and red light.
  • the path of the illumination light that travels from dichroic mirror to the sample may coincide with the path of the emission light that travels from the sample to the
  • the emission light may pass through dichroic mirror, since it may be configured to pass light of the second group of wavelength, (e.g. red and green light) .
  • the second group of wavelengths may thus comprise the
  • the emission light originating from the sample may include optical
  • emission light beam 118 may be split into a first emission light beam 118a and second emission light beam 118b.
  • forming the first and second emission light beams may include spitting the emission light beam in a first and second light beam of different wavelengths .
  • the first and second emission light beams may be formed on the basis of a variety of optical properties of the emission light.
  • the emission light beams may be formed on the basis of the wavelength, wherein the first emission light beam comprises a first wavelength and the second light beam a different second wavelength.
  • the emission light beams may be formed on the basis of polarization, wherein the first light beam comprises a first polarization and the second light beam a different second polarization.
  • the second optical system of the scanning microscopy system may comprise an optical splitter that is configured to split the emission light into different emission light beams on the basis of at least one optical properties, such as wavelength,
  • a light splitter such as an optical splitter 134 for splitting the emission light into different emission light beams, e.g. a first emission light 118a and second emission light 118b.
  • the light splitter 128 may include at least a dichroic mirror 128 and a mirror 130 in the path of the emission light beam 118
  • a dichroic mirror instead of a dichroic mirror other optical splitter elements may be used including but not limited to a polarization splitter, a grey splitter or an acousto-optical tunable splitter.
  • the emission light beam 118 may be incident on dichroic mirror 128, which may be configured to reflect the first emission light, e.g. green light, and to let pass the second emission light, e.g. red light.
  • dichroic mirror 128, may be configured to reflect the first emission light, e.g. green light, and to let pass the second emission light, e.g. red light.
  • dichroic mirror 128 may split the emission light into a first emission light beam 118a, which may comprise the first
  • the first emission light beam may comprise first optical information, e.g. information relating to red fluorophores in the sample
  • the second emission light beam 118b may comprise second optical information, e.g. information relating to green fluorophores in the sample.
  • the first emission light beam 118a may be incident onto rescanning mirror 120, while the second emission light beam 118b may be directed onto mirror 130, which may reflect the second emission light beam 118b onto rescanning mirror 120.
  • the light splitter may be configured so that the split emission light beams, e.g. first emission light beam 118a and second light beam 118b, may be incident on rescanning mirror 120 under different angles.
  • Rescanning mirror may reflect the emission light beams onto different areas of the image plane of the imaging system.
  • Rotatable rescanning mirror may reflect both emission light beams 118a, 118b onto the focusing lens which may focus the light beams onto an image plane of imaging system. This way, when rescanning the focused emission light spots onto the imaging plane, at least a first and a second image are formed simultaneously.
  • the first emission light beam may cause a first illumination spot at a first position 126a, while the second emission light beam may cause a second illumination spot at a second position 126b that is different from the first position.
  • filters such as color filters, e.g. a first color filter 123a and second color filter 123b, may be positioned in front of the imaging system.
  • a color filter may be configured as a bandpass filter that is configured to pass light in a predetermined band of wavelengths.
  • the first color filter 123a may be configured to pass first
  • the second color filter 123b may be configured to pass second emission light, e.g.
  • the scanning mirror may rotate over a smaller angular amplitude than the rescanning mirror. Due to the rotation of the scanning mirror, the focused illumination light spot may move over or through the sample. Due to the rotation of the rescanning mirror, the focused emission light beams, and thus the emission light spots and, may move over the image area of imaging system 124 thereby exposing pixels to the emission light.
  • Moving the scanning mirror may comprise rotating the scanning mirror over a first angular amplitude and moving the rescanning mirror may comprise rotating the rescanning mirror over a second angular amplitude.
  • the first angular amplitude may be different from the second angular amplitude, in particular the first angular amplitude may be smaller than the second angular amplitude, in particular the first angular amplitude may be approximately half the second angular amplitude.
  • Fig. 1 depicts an example wherein the emission light beam is split in two emission light beams for simultaneous imaging two images of a sample area comprising different optical information
  • this scheme may be extended to embodiments that are configured to split the emission light beam in multiple (more than two) emission light beams so that multiple images comprising different optical information.
  • scanning microscopy system depicted in Fig. 1 is capable simultaneously capturing super-resolution images of excited areas of a sample and combining these images into a super-resolution colour image.
  • the system is suitable of colour imaging time- critical processes of an area of a sample with super- resolution using conventional refractive and reflective optical elements.
  • Certain reflective elements in the scanning microscopy system such as mirrors 108,130 and dichroic mirrors 110,128 may be implemented as static optical elements. Such optical elements cannot be controlled by the computer system. Other reflective elements may be implemented as rotatable optical elements that can be controlled by the computer system.
  • the path of illumination light 106 from light source 102 towards the scanning mirror 112 may be controlled using static optical elements. During a scan, the scanning mirror 112 may rotate (i.e. rotatable moving back and forth) over an angular amplitude which causes the path of the illumination light 106 from scanning mirror 112 to the sample 110 (and thus the focused execution light spot) to move over the surface of the sample in accordance with the movements of the scanning mirror.
  • the path of the illumination light 106 from scanning mirror 112 to sample 116 is said to be dynamic in the sense that it is moving with respect to the sample and/or light detector during scanning. Consequently the point of illumination in the sample is moving during scanning and therefore also the point where emission signal comes from from the sample is moving during scanning.
  • the scanning mirror both directs the illumination light signal from the light source 102 towards the sample 116 as well as the emission light signal from the sample 116 towards the rescanning mirror 120. This way, the movement of the emission light signal originating from the sample is neutralized by the movement of the scanning mirror.
  • the path of emission light from scanning mirror to rescanning mirror (the paths of light beams 118a and 118b from dichroic mirror 128 to rescanning mirror) is static.
  • the scanning mirror 112 "descans" the emission light 118 and reflects the emission light as a static emission light beam towards the rescanning mirror.
  • the path of the emission light between rescanning mirror and light detector may be dynamic during scanning due to the movement of the rescanning mirror.
  • the optical beam splitter comprising e.g. a dichroic mirror, may be positioned in a static path of emission light.
  • FIG. 2A-2C schematically depict a top view of the scanning movements of an illumination light spot 217 and one or more associated emission light spots 226a, 226b of a
  • Fig. 2A-2C depict the movement of a focused illumination light spot 217 over or through optical active sample 216 and the movement of the associated focused first and second emission light spots 226a, 226b over the image area of the imaging system 224 at three different moments in time TO, Tl and T2.
  • the illumination light may comprise multiple wavelengths, e.g. a first and a second wavelength, for exciting different optical illumination sites in the sample.
  • Fig. 2A depicts the start of the scanning process for both the illumination light spot and the associated emission light spots.
  • Fig. 2B depicts the movement of the illumination light spot and the associated focused first and second
  • time instance Tl emission light spots over the image area of the imaging system 224 at an intermediate time instance Tl.
  • TO may mark the start of a scan wherein the illumination light spot will start exposing a predetermined area by moving
  • the scanning pattern may be a meandering pattern.
  • the light spots may be moved by a computer system controlling the scanning and descanning mirror using actuators or the like as described with reference to
  • emission light originating from the sample is split into different emission light beams which are focused as emission light spots, in this example a first and second emission light spot 226a, 226b, onto the imaging plane of the imaging system wherein each emission light beam carries a different type of optical information of the area of the sample that is excited by the illumination light spot.
  • emission light spots a first and second emission light spot 226a, 226b
  • each emission light beam carries a different type of optical information of the area of the sample that is excited by the illumination light spot.
  • Fig. 2C depicts the scanning pattern at a second time instance T2 (the end of a scan) wherein a full scan of the scanned sample area 228 may results in two scanned areas 230a, 230b of the imaging plane of the imaging system wherein each scanned area of the imaging plane may result in an image of the scanned sample, wherein each image may include
  • the imaging area of the imaging system may be any imaging area of the imaging system.
  • an image sensor may include one or more pixel arrays that are configured to simultaneously image the different illumination light beams.
  • An image sensor may be implemented as a CCD image sensor or a CMOS image sensor .
  • the speed with which the focused illumination light spot moves over or through optical active sample 216 may be lower than the speed with which the first and second emission light beams move over the image area of the imaging system.
  • said speed of the focused illumination light spot 217 may be approximately half the speed of the emission light beams, in particular of the first and second illumination spots.
  • Fig. 3 depicts a flow diagram of a scanning process according to an embodiment of the invention.
  • the figure depicts a scanning process that may be executed by a scanning microscope system as described with reference to Fig. 1.
  • the process may be executed by a computer system configured to control the light source and the actuators for controlling the movements of the scanning and rescanning mirror .
  • the computer system may control the scanning mirror of a first optical system to move (scan) one or more focused illumination light spots over (or through) an optical active sample, wherein the one or more focused illumination light spots cause local optical excitations in the optical active sample.
  • Local optical excitations may generate emission light which in a second step 304 may be picked up by the first optical system in order to form an emission light beam from the emission light originating from the optical excitations.
  • the first optical system may direct the emission light beam onto the scanning mirror which may reflect the emission light beam via a second optical system to an imaging system (step 304) .
  • the second optical system may include a light
  • the light splitter for splitting the emission light beam in at least a first emission light beam comprising first optical information and second emission light beam comprising second optical information (step 306) . Then, the light splitter may direct the first and second emission light beams onto a computer- controlled rescanning mirror, which is used to move (can) focused emission light beam spots over the imaging plane of the image system (step 308) in order to form a first image comprising the first optical information and second image comprising the second optical information, while the sample is scanned with the illumination light.
  • a computer- controlled rescanning mirror which is used to move (can) focused emission light beam spots over the imaging plane of the image system (step 308) in order to form a first image comprising the first optical information and second image comprising the second optical information, while the sample is scanned with the illumination light.
  • the computer system may be configured to
  • the system is capable of simultaneously generating multiple high-resolution images of a sample area that is excited with one or more illumination light spots. This way, the rescan microcopy system allow the imaging of time-critical processes .
  • FIG. 4 depicts a scanning microscopy system according to another embodiment of the invention.
  • Fig. 4 depicts a scanning microscopy system that is similar to the scanning microscopy system of Fig. 1, comprising a light source 402, a first optical system 432 comprising a scanning mirror 412 that is configured to guide an illumination light beam from the light source towards the scanning mirror and to expose a sample 416 with a moving illumination light beam, a second optical system 436 comprising a rescanning mirror 420
  • Fig. 4 The embodiment of Fig. 4 is configured to measure lifetime luminescence properties. As will be explained
  • the properties can be derived from a phase difference between a modulated illumination light signal and an emission light signal, because the phase difference depends on the lifetime properties of the sample as e.g.
  • the emission light signal may be integrated by the imaging system over a number of very short time intervals.
  • the Nyquist criterion stipulates that for mono-exponential life time measurements at least two time intervals or phase ranges are required per period (e.g. 0-180 and 180-360 degrees) .
  • more than two time intervals or phase ranges e.g. 0-90, 90-180, 180-270, 270-360 degrees
  • integrating the time-varying intensity of the emission light in such short phase ranges is very difficult for an image sensor, because of the very short exposure times associated with these phase ranges.
  • the second optical system 436 of the scanning microscopy system may be configured to split the modulated emission light beam into a first modulated emission light beam and into one or more further modulated emission light beams, wherein the one or more further modulated emission light beams are phase- shifted with respect to the first modulated emission light beam.
  • the phase shift may be 90 degrees. Both modulated emission light signals are then used in order to determine the emission light signal.
  • Fig. 5A-5C depicts a data acquisition process for a scanning microscopy system according to an embodiment of the invention.
  • the figures depict a data
  • Fig. 5A illustrates an illumination light signal that is modulated using a frequency in a range selected between 1 kHz and 100 MHz, typically 40 MHz. Due to the modulation of the illumination signal, the emission light signals will be modulated as well.
  • Fig. 5B depicts the first modulated emission light signal which has a certain amplitude and phase shift with respect to the illumination signal.
  • Fig. 5C depicts a second modulated emission light signal which represents a 90 degrees phase-shifted version of the first modulated emission light signal .
  • first images of the first modulated emission light signal may be captured by the imaging system at predetermined time instances TO , Tl , T2 , T3 , wherein the exposure time of the imaging system is defined by an integration window T0-T1.
  • the time instances are synchronized with the modulation frequency in the sense that at time instances T0,T2,T4 the phase angle of the signal is
  • pixel values of the first images generated during integration window in which the phase of the signal is 0-180 degrees represent an average value of the intensity of the first modulated emission signal in that window.
  • pixel values of images generated during integration window in which the phase of the signal is 180-360 degrees represent an average value of the intensity of the first modulated emission signal in that window.
  • second images are generated on the basis of the second modulated emission signal in the integration window in which the phase of the signal ranges 90- 270 degrees (region III in Fig. 5C) and 270-90 degrees (region IV in Fig. 5C) . Based on the average signal values of the four regions, the original response of the emission light signal may be obtained and subsequently the phase difference with the driving modulated illumination signal may be determined.
  • microscopy system may comprise an optical source for
  • the optical source may be configured to vary the intensity of the one or more focused illumination light spots with a modulation frequency, e.g. a sinusoidal modulation frequency, so that also the intensity of the first and second emission light spot will vary as a function of time.
  • a modulation frequency e.g. a sinusoidal modulation frequency
  • the second optical system of the microscopy system of Fig. 4 may comprise an optical splitter 427, e.g. in the form of reflective mirror, for splitting the emission light into different emission light beams, e.g. a first emission light beam and a second emission light beam.
  • the second optical system may further include an optical delay line 434 in order to introduce phase shift in the modulated second emission light signal. This way, a predetermined phase difference between the time-varying intensity of the first emission light spot 426b and the second emission light spot 426a may be introduced .
  • the optical delay line may comprise one or more mirrors, e.g. a first mirror 438 and a second mirror 440, in order to control the length of the optical path.
  • the optical delay line shown in Fig. 4 may introduce a predetermined phase shift in the modulation signal of the second emission light spot 426b with respect to that of the first emission light spot 426a.
  • the phase shift may be at least 45 degrees, preferably by at least 90 degrees, most preferably approximately 90 degrees (as e.g. depicted in Fig. 5B and 5C) .
  • the phase shifted emission light beams are directed towards the rescanning mirror, which is controlled to move (scan) focused emission light beam signals over the imaging plane of the image system in order to simultaneously form a first image associated with the first emission light beam and a second image associated with the second emission light beam, while the sample is scanned with the illumination light.
  • the optical splitter and the optical delay line enable the scanning microscopy system to simultaneously capture an image of the modulated emission light signal and an image of a 90 degrees phase-shifted version of the modulated emission light signal.
  • an image generated during an integration window in which the phase of the signal is 0-180 degrees may be captured simultaneously with an image of the signal in an integration window in which the phase of the signal is 90-270 degrees (region III in Fig. 5C) .
  • the scanning microscopy system according the invention allows lifetime measurements using a RCM super- resolution technique.
  • Fig. 4 may be configured to produce more than two phase shifted emission light beam, using e.g. a beam splitter that splits the emission light beam into multiple emission light beams and one or more optical delay lines configured to introduce different phase shifts in each of the multiple emission light beams.
  • a beam splitter that splits the emission light beam into multiple emission light beams and one or more optical delay lines configured to introduce different phase shifts in each of the multiple emission light beams.
  • the invention allows lifetime luminescence properties using a scanning microscope.
  • the invention eliminates the need for an image sensor that has on-chip phase-sensitive detection functionality.
  • the lifetime imaging may also be based on other well-known techniques such as a pulsed illumination technique.
  • the light source may be configured as a pulsed light source, e.g. one or more pulsed lasers.
  • Fig. 6 depicts a flow diagram of a scanning process according to an embodiment of the invention.
  • flow diagram depicts a scanning process that may be executed by a scanning microscope system as described with reference to Fig. 4 and 5.
  • the process may be executed by a computer system configured to control the light source and the actuators for controlling the movements of the scanning and rescanning mirror .
  • the computer system may control a scanning mirror of a first optical system to move (scan) one or more focused modulated illumination light signals over (or through) an optical active sample, wherein the one or more focused modulated illumination light spots cause optical excitations in the optical active sample.
  • Local optical excitations may generate modulated emission light which in a second step 604 may be picked up by the first optical system in order to form a modulated emission light signal from modulated emission light originating from the optical excitations and directing the modulated emission light signal onto the scanning mirror for reflecting the emission light signal via a second optical system to an imaging system.
  • the light splitter may direct the first and second modulated emission light signals onto a rescanning mirror, which is controlled to move (scan) focused emission light signals over the imaging plane of the image system in order to simultaneously form a first image of the first modulated light signal and second image of the second
  • multiple differently phase-shifted modulated emission light signals may be used so that instead of two images a plurality of images of different parts of the modulated emission light signal may be obtained.
  • the original emission light signal can be constructed and luminescent lifetimes can be determined.
  • the computer system may be configured to
  • Fig. 7 depicts a scanning microscopy system according to yet another embodiment.
  • Fig. 7 depicts a scanning microscopy system similar to the system described with reference to Fig. 1.
  • the system in Fig. 7 comprises a second optical system 736 including an optical arrangement 739 for enabling confocal measurements wherein the optical arrangement may comprise a pinhole 744.
  • the scanning microscopy may be configured to focus illumination light spot 717 in a plane of interest P.
  • the system may be configured to arranged to focus emission light from the plane of interest P onto plane P ' .
  • the system may comprise one or more optical elements, e.g.
  • Plane P' thus may be a confocal conjugate plane of plane of interest P.
  • the optical arrangement may comprise a further optical element, e.g. lens 742, arranged to form a collimated light beam of the emission light 718 that passes pinhole 744.
  • Emission light from the plane of interest in the sample may pass through the pinhole while emission light 718 that is emitted from other planes than the plane of interest is substantially blocked by the pinhole so that the signal-to- noise ratio of the emission light signal that is focused onto the imaging plane of the imaging system may be substantially improved.
  • Fig. 8 depicts a scanning microscopy system according to yet another embodiment.
  • Fig. 8 depicts a scanning microscopy system comprising a first optical system 832 comprising a number of optical elements, e.g. refractive and/or diffractive optical elements, arranged to form a first focused illumination light spot 817a at a first position in or on the sample and a second focused illumination light spot 817b at a second position in or on the sample.
  • the first and second position may have a different lateral position.
  • the first optical system further comprises at least one scanning reflector 812 for moving the light spots over or through the optical active sample 816.
  • the first illumination light spot 817a may be formed using a first illumination light beam 806a and the second illumination light spot 817b may be formed using a second illumination light beam 806b.
  • the first illumination light beam may comprise light of a first
  • the second illumination light beam may comprise light of a second wavelength, e.g. blue light.
  • the first and second illumination light beams may be formed on the basis of light of light source.
  • the first and second light spots may be arranged close together so that the spots illuminate a small area very close to each other.
  • Light of two or more wavelengths may be split using a first optical splitter 846, for example dichroic mirror.
  • the optical splitter 46 may be configured to reflect light of a second wavelength, e.g. blue light, and let pass light of a first wavelength, e.g. yellow light.
  • first illumination beam 806a may comprise the first wavelength and the second illumination beam 806b may comprise the second wavelength.
  • the illumination beams may be directed on mirror 808 that may reflect both light beams onto mirror 10.
  • Mirror 10 may reflect both the first and second illumination light beam onto scanning mirror 812.
  • the scanning mirror may reflect the two illumination light beams onto sample 16 which causes local optical excitations generating emission light at the two focused illumination light spots 817a and 817b.
  • a yellow illumination light spot 817a may cause emission of red emission light and a blue focused illumination light spot causes emission of green emission light.
  • the system may comprise a second optical system for directing the emission light via a rescanning mirror onto an imaging system that is similar to the second optical system of Fig. 1.
  • the emission light both the first and second emission light, may be incident on scanning mirror 812 and may pass through dichroic mirror 810.
  • Dichroic mirror 828 may reflect the second emission light, e.g. the green light, and let pass the first emission light, e.g. the green emission light.
  • the first emission light beam 818a comprising the first emission light and the second emission light beam 818b
  • Both emission light beams may be directed onto rescanning mirror 820 which may direct the two beams onto the image sensor.
  • a first image and second image are formed by focusing with lens 822 the first and respective second
  • the two emission light beams 18a and 18b cause first
  • this embodiment allows to simultaneously illuminate and image an area of the sample using different illumination light signals.
  • the first optical information in the first emission light beam and the second optical information in the second emission light beam may relate to the same area of the sample.
  • Fig. 9 depicts a scanning microscopy system according to another embodiment.
  • the scanning microscopy system may be configured such that the first and second illumination light spot may be at different depths into the sample.
  • an optical splitter 947 may direct part, e.g. 50%, of the incoming illumination light 906 towards mirror 948 and may let pass part, e.g. 50%, of the incoming illumination light.
  • the two illumination light beams 906a and 906b may comprise the same wavelengths of light. Again, the two illumination light beams may be directed onto scanning mirror 912.
  • the first illumination light beam may be directed via a first lens 907a onto lens 914, thereby focusing the first illumination light beam 906a onto the sample creating a first focused illumination light spot 917a at a first focus plane PI in the sample, which causes emission of first
  • the second illumination light beam may be directed via a second lens 907b onto lens 914 thereby focusing the second light beam onto the sample 916.
  • illumination light beam 906b causes emission of second
  • the first illumination light spot may cause emission of emission light from a first plane of interest PI and the second illumination light spot may cause emission of emission light from a second plane of interest P2.
  • the step of forming the first and second emission light beams on the basis of the emission light may comprise directing the first emission light through a first optical path and directing the second emission light through a second optical path, wherein the first optical path is different from the second optical path.
  • the first and second emission light beams are formed on the basis of the emission light in the sense that the second emission light is directed on mirror 930 whereas the first emission light is not.
  • the first light beam 18a which comprises the first emission light may be focused upon a pinhole 944a.
  • the pinhole 944a is placed in a conjugate focal plane PI' to plane of interest PI.
  • the second emission light beam 18b is focused upon pinhole 944b.
  • Pinhole 944b is in a conjugate focal plane P2 ' to plane of interest P2.
  • the pinholes 944a and 944b provide images of the emission light with an improved signal-to-noise ratio.
  • Various embodiments of the invention may be imple ⁇ mented as a program product for use with a computer system, where the program(s) of the program product define functions of the embodiments (including the methods described herein) .
  • the program(s) can be contained on a vari ⁇ ety of non-transitory computer-readable storage media, where, as used herein, the expression "non-transitory computer readable storage media" comprises all computer-readable media, with the sole exception being a transitory, propagating signal.
  • the program(s) can be contained on a vari ⁇ ety of transitory computer-readable storage media.
  • Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media (e.g., read- only memory devices within a computer such as CD-ROM disks readable by a CD-ROM drive, ROM chips or any type of solid- state non-volatile semiconductor memory) on which information is permanently stored; and (ii) writable storage media (e.g., flash memory, floppy disks within a diskette drive or hard- disk drive or any type of solid-state random-access semicon ⁇ ductor memory) on which alterable information is stored.
  • the computer program may be run on the processor 102 described herein .

Landscapes

  • Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Microscoopes, Condenser (AREA)

Abstract

Method and system for scanning microscopy are described. The system comprises a first optical system configured to move one or more focused illumination light spots over or through an optical active sample, the one or more focused illumination light spots causing local optical excitations in the optical active sample, the local optical excitations generating emission light. The system further comprises a second optical system configured to form at least a first emission light beam comprising first optical information and second emission light beam comprising second optical information on the basis of the emission light associated with the local optical excitations. The scanning microscopy system also comprises a lens system configured to focus the first emission light beam onto an image plane of an imaging system for forming a first image and configured to focus the second emission light beam onto the image plane of the imaging system for forming a second image. The system also comprises a moveable scanning mirror that is configured to move the focused first and second emission light beam over the image area of the imaging system when the one or more focused illumination light spots are moved over or through the sample.

Description

Forming images of a sample using scanning microscopy
FIELD OF THE INVENTION The invention relates to forming images using scanning microscopy and, in particular, though not exclusively, to systems and methods for forming images of a sample using scan¬ ning microscopy, a computer system for controlling such systems and computer program product for executing such meth- ods .
BACKGROUND OF THE INVENTION
Rescan confocal microscopy (RCM) is a known super- resolution microscopy technique for obtaining high-resolution images using conventional refractive and reflective optical elements. The principles of RCM are explained in the article by De Luca et al . , "Re-scan confocal microscopy: scanning twice for better resolution" , Biomed Opt Express; 2013 Nov 1 ; 4(11) : 2644-2656. A high resolution colour image of an area of a sample can be obtained by scanning the area line by line with an illumination light beam of two different wavelengths and rescanning the light originating from the area onto an image sensor so that an image of the excited area is obtained.
A problem related to the colour rescan confocal microscopy technique as described by De Luca is that it is less suitable for time-critical measurements which typically require (almost) simultaneous acquisition of different images associated with different optical information. Although the article by De Luca reported reduction of the time difference between two colour scans to the time of a single scanning line, the fact that the line images are not of the same time instance, imaged features and/or objects in different images may have moved and/or changed so that combining the images will cause artefacts in the final colour image. Similarly, in fluorescence or luminescence lifetime imaging microscopy the luminescence lifetime properties of a sample can be obtained by exposing a sample with modulated light and by measuring multiple images of the modulated emission light. The line by line scanning technique described in De Luca however is not suitable for reliably measuring lifetime properties of a sample .
WO 2015/164843 discloses a multifocal structured illumination microscopy system having two scanning mirrors and two cameras. In this system, a sample that is positioned at a focus is scanned with excitation light using a scanning mirror. As a result, the sample emits emission light back onto the scanning mirror which reflects the emission light onto a second scanning mirror, which scans the emission light onto the two cameras. In this system, the emission light from the second scanning mirror passes through a dichroic mirror that splits the emission light towards the two cameras. This enables each camera to separately detect a different light color. A problem related to this system is that movement of the emission light beam with respect to the dichroic mirror causes the emission light to be incident on the mirror under a varying angle. This varying angle influences the optical interactions between the dichroic mirror and the light beam, which negatively impacts the quality of the images captured by the cameras.
Hence, there is a need in the prior art for improved rescan microscopy systems that are capable of imaging and measuring time-critical processes such as colour imaging or imaging of time-dependent processes or lifetime properties of a sample.
SUMMARY OF THE INVENTION
It is an object of the present disclosure to provide an improved method and system for forming images using
scanning microscopy that allows for increasing the amount of information that can be obtained with one scan cycle. This for example allows to reduce the number of required scans and to increase scan speed. It may also beneficially eliminate a time gap between measurements.
In an aspect, the invention may relate to a method for forming images of a sample using scanning microscopy. In an embodiment, the method may include controlling a scanning mirror of a first optical system to move (scan) one or more focused illumination light signals over (or through) an optical active sample, wherein the one or more focused
illumination light signal cause one or more optical
excitations in the optical active sample; the first optical system forming at least one emission light signal from
emission light originating from the one or more optical excitations and directing the emission light signal onto the scanning mirror for reflecting the emission light signal via a second optical system to an imaging system; a light splitter in the second optical system splitting the at least one emission light signal in at least a first emission light signal comprising first optical information and second
emission light signal comprising second optical information, preferably the first optical information being different from the second optical information; and, the second optical system directing the first and second emission light signals onto a rescanning mirror, the rescanning mirror being controlled to move (scan) a focused emission light beam signals over the imaging plane of the imaging system for simultaneously forming a first image associated with the first optical information and second image associated with the second optical
information, while the sample is scanned with the one or more illumination light signals.
Hence, the invention enables a scanning microscope to determine image data, i.e. multiple different images,
associated with different types of optical information of a sample in one scanning cycle. The different optical
information may relate to different characteristics of the sample. The invention thus enables a scanning microscope to measure and image time-critical processes, such as color images, time-dependent processes and/or life-time
measurements .
Advantageously, in the methods and systems described herein, the angle under which light falls onto the light splitter can be kept constant. Consequently, the optical interactions between the incident light and the splitter are also constant. In an example, in case the light splitter is a dichroic mirror, the cutoff wavelength of the dichroic mirror remains the same during an experiment, which ensures that any variation in light intensity detected by the imaging system corresponds to an intensity variation of the light emission signal and not to a variation of the cutoff wavelength of the dichroic mirror. In the latter case, a formed image would become inaccurate as it would contain intensity variations that do not correspond with variations in density of
fluorophores in the sample. The methods and systems described herein may reduce such distortions and increase the quality of the images that are generated by the scanning microscope in one scanning cycle.
In an embodiment, the one or more excitations comprise at least a first type of excitation causing emission of a first wavelength or a first wavelength band and a second type of excitation causing emission of a second wavelength or second wavelength band different from the first wavelength or first wavelength band.
In an embodiment, the first optical information may be associated with the emission of the first wavelength or a first wavelength band and the second optical information may be associated with the emission of the second wavelength or second wavelength band.
In an embodiment, the method further comprise: the capturing first image data associated with the first emission light and a second image data associated with the second emission light; and, forming an image of at least part of the sample by comprising at least part of the first and second image data.
In an embodiment, the method may further comprise: modulating the intensity of the at least one illumination light signal in accordance with a modulation frequency, wherein the modulation of the illumination light signal may cause a modulation of the intensity of the first and second emission light signal.
In an embodiment, the method may further comprise phase-shifting the modulated second emission light signal with respect to the modulated first emission light signal.
In an embodiment, the phase-shifting may include directing the modulated second emission light signal through an optical delay line.
In an embodiment, the modulation frequency may be selected between 1 and 200 MHz.
In an embodiment, the method may further comprise: the imaging system simultaneously capturing first image data associated with the modulated first emission light signal and second image data associated with the time-shifted modulated second emission light signal.
In an embodiment, the method may further comprise a processor determining a life-time associated with at least one of the one or more optical excitations on the basis of at least part of the first image data and at least part of the second image data.
In an embodiment, the method may comprise focusing at least part of the emission light signal upon a pinhole
aperture .
In an embodiment, the focusing may include using at least one first optical element for focusing the emission light signal originating from the scanning mirror onto the pinhole aperture and at least one second optical element configured to form a collimated emission light signal of the emission light that passes pinhole and to direct the
collimated emission light signal towards the rescanning mirror .
In an embodiment, the one or more focused
illumination light signals may comprise at least a first focused illumination light spot at a first position of a focusing plane in the sample and a second focused illumination light spot at a second position in the focusing plane in the sample, preferably the first illumination light being
associated with a first wavelength or wavelength band and the second illumination light being associated with a second wavelength or wavelength band; more preferably the first position being relatively close to the second position; the first focused illumination light spot causing a first optical excitation at the first position; and, the second focused illumination light signal causing a second optical excitation at the second position; and, wherein the first emission light signal comprises emission light of the first optical
excitation and the second emission light beam comprises emission light of the second optical excitation.
In an embodiment, the one or more focused
illumination light signals may comprise at least a first focused illumination light spot at a first position in a first focusing plane in the sample and a second focused illumination light spot at a second position in a second focusing plane in the sample.
In an embodiment, controlling the scanning and the rescanning mirror may include rotating the scanning mirror and the rescanning mirror back and forth over a predetermined first angular amplitude and a second angular amplitude
respectively.
In an embodiment, the second angular amplitude of the rescanning mirror may be selected larger than the first angular amplitude of the scanning mirror.
In an embodiment, the second angular amplitude may be selected as two times the first angular amplitude.
In another aspect the invention may relate to a method for forming images using scanning microscopy
comprising: controlling a scanning mirror of a first optical system to move (scan) one or more focused modulated
illumination light signals over (or through) a sample, wherein the one or more focused modulated illumination light spots cause optical excitations in the optical active sample; the first optical system forming a modulated emission light signal from modulated emission light originating from the optical excitations and directing the modulated emission light signal onto the scanning mirror for reflecting the emission light signal via a second optical system to an imaging system; a light splitter in the second optical system splitting the modulated emission light signal in at least a first modulated emission light signal and second modulated emission light signal representing a phase-shifted version of the first modulated light signal; and, the light splitter directing the first and second modulated emission light signals onto a rescanning mirror, which is controlled to move (scan) focused emission light signals over the imaging plane of the imaging system in order to simultaneously form a first image of the first modulated emission light signal and second image of the second modulated emission light signal, while the one or more focused modulated illumination light signals are scanned over (or through) the sample.
In yet another aspect, the invention may relate to a scanning microscopy system for forming images of a sample wherein the system may comprise: a light source, preferably one or more lasers, configured for generating light for illuminating the sample; a first optical system comprising a scanning mirror, the optical system being configured: to focus illumination light of the light source as one or more
illumination light signals via the scanning mirror onto the sample, the one or more focused illumination light signals causing one or more optical excitations in the optical active sample; to form an emission light signal from emission light originating from the one or more optical excitations in the sample; and, to direct the emission light signal onto the scanning mirror; a second optical system comprising a
rescanning mirror and a light splitter, the light splitter being configured: to split the at least one emission light signal originating from the scanning mirror in at least a first emission light signal comprising first optical
information and second emission light signal comprising second optical information; the second optical system being configured: to direct the first and second emission light signals towards the rescanning mirror and to focus the first and second emission light signals onto an imaging plane;
an imaging system configured to receive first and second emission light signals originating from the rescanning mirror; and, a computer system being adapted to: control the scanning mirror of the first optical system to move (scan) the one or more focused illumination light signals over (or through) an optical active sample; control the rescanning mirror of the second optical system to move (scan) a focused emission light beam signal over the imaging plane of the imaging system; and, control the imaging system for simultaneously capturing first image data associated with the first optical information and second image data associated with the second optical
information, while the sample is scanned with the one or more illumination light signals.
In an embodiment, the one or more excitations comprise at least a first type of excitation causing emission of a first wavelength or a first wavelength band and a second type of excitation causing emission of a second wavelength or second wavelength band different from the first wavelength or first wavelength band; and, wherein the first optical
information is associated with the emission of the first wavelength or a first wavelength band and the second optical information is associated with the emission of the second wavelength or second wavelength band.
In an embodiment, the system may further comprise: a modulator for modulating the intensity of the at least one illumination light signal in accordance with a modulation frequency, preferably the modulation frequency being selected between 1 and 200 MHz; the modulation of the illumination light signal causing a modulation of the intensity of the first and second emission light signal; a phase-shifter, preferably an optical delay line, configured to phase-shift the modulated second emission light signal with respect to the modulated first emission light signal. In an embodiment, the system may further comprise the imaging system further being configured to simultaneously capture first image data associated with the modulated first emission light signal and second image data associated with the time-shifted modulated second emission light signal; and, the computer system further being adapted to: determine a life-time associated with at least one of the one or more optical excitations on the basis of at least part of the first image data and at least part of the second image data.
In further aspect, the invention may relate to a computer system for controlling scanning microscopy system for forming images of a sample comprising, the system comprising a light source, preferably one or more lasers, configured for generating light for illuminating the sample; a first optical system comprising a scanning mirror, the optical system being configured to focus illumination light of the light source as one or more illumination light signals via the scanning mirror onto the sample, the one or more focused illumination light signals causing one or more optical excitations in the optical active sample; to form an emission light signal from emission light originating from the one or more optical excitations in the sample; and, to direct the emission light signal onto the scanning mirror; a second optical system comprising a
rescanning mirror and a light splitter, the light splitter being configured to split the at least one emission light signal originating from the scanning mirror in at least a first emission light signal comprising first optical
information and second emission light signal comprising second optical information, the second optical system being
configured to direct the first and second emission light signals towards the rescanning mirror and to focus the first and second emission light signals onto an imaging plane; an imaging system configured to receive first and second emission light signals originating from the rescanning mirror; wherein the computer system comprises a computer readable storage medium having computer readable program code embodied
therewith, and a processor, preferably a microprocessor, coupled to the computer readable storage medium, wherein responsive to executing the first computer readable program code, the processor is configured to perform executable operations comprising:
controlling the scanning mirror of the first optical system to move (scan) the one or more focused illumination light signals over (or through) an optical active sample;
controlling the rescanning mirror of the second optical system to move (scan) a focused emission light beam signal over the imaging plane of the imaging system; and,
controlling the imaging system to simultaneously first image data associated with the first optical information and second image data associated with the second optical information, while the sample is scanned with the one or more illumination light signals.
In an embodiment, the processor may be further configured to perform an executable operation comprising:
modulating the intensity of the light source in accordance with a modulation frequency, preferably the
modulation frequency being selected between 1 and 200 MHz; the modulation of the illumination light signal causing a
modulation of the intensity of the first and second emission light signal;
the imaging system simultaneously capturing first image data associated with the modulated first emission light signal and second image data associated with a time-shifted version of the modulated first emission light signal; and,
determining a life-time associated with at least one of the one or more optical excitations on the basis of at least part of the first image data and at least part of the second image data.
Moreover, the invention also relates to computer program product for carrying out the methods described herein, as well as a non-transitory computer readable storage-medium storing the computer program. A computer program may, for example, be downloaded (updated) to an existing control device of the scanning microscopy system or be stored upon manufac¬ turing of this control device.
Embodiments of the present invention will be further illustrated with reference to the attached drawings, which schematically will show embodiments according to the inven¬ tion. It will be understood that the present invention is not in any way restricted to these specific embodiments.
BRIEF DESCRIPTION OF THE DRAWINGS
Aspects of the invention will be explained in greater detail by reference to exemplary embodiments shown in the drawings, in which:
Fig. 1 shows a scanning microscopy system according to an embodiment of the invention;
Fig. 2A-2C schematically depict the scanning of an illumination light spot and emission light spots by a scanning microscopy system according to an embodiment of the invention;
Fig. 3 depicts a flow diagram of a scanning process according to an embodiment of the invention;
FIG. 4 depicts a scanning microscopy system according to another embodiment of the invention;
Fig. 5A-5C depicts a data acquisition process for a scanning microscopy system according to an embodiment of the invention . ;
Fig. 6 depicts a flow diagram of a scanning process according to an embodiment of the invention;
Fig. 7 depicts a scanning microscopy system according to another embodiment;
Fig. 8 depicts a scanning microscopy system according to yet another embodiment;
Fig. 9 depicts a scanning microscopy system according to a further embodiment; DETAILED DESCRIPTION OF THE DRAWINGS Fig. 1 shows a scanning microscopy system according to an embodiment of the invention. The system may comprise a light source 102 for generating illumination light that is directed onto a sample 116 located on a sample holder (not shown) . Illumination light 106 may be directed onto a sample using a first optical system 132, which may include one or more refractive and/or reflective optical elements. For example, in an embodiment, the first optical system may include one or more collimating and/or focusing lenses
104,114, mirrors 108,112 and/or dichroic mirrors 10.
In particular, a lens 106 may be used in order to from a collimated beam of illumination light 106 which may be reflected via a set of mirrors 108,110,112 towards a lens 114 in order to form a focused illumination light spot 117 on the sample. The focused illumination light spot may excite local optical excitations, which generate emission light 118. The emission light may pass through lens 114 and may be directed via a second optical system 136 comprising one or more
refractive and/or reflective optical elements to an imaging system 124.
The second optical system may include a lens 114 for forming a collimated beam of the emission light that is reflected by mirror 112. The beam of emission light may pass one or more optical elements, e.g. dichroic mirrors 110,128, for splitting the emission light beams into different emission light beams before it is reflected by a further mirror 120, which directs the emission light beams to a focussing lens 122. The focussing lens may focus the emission light beams into focussed emission light spots 126a, 126b onto the imaging plane of imaging system 124. Here, the imaging system may include one or more image sensors, e.g. one or more CMOS image sensors or one or more CCDs. Dichroic mirror 110 may be configured such that it functions as a reflector for
illumination light 106 and such that it is transparent for emission light 118 (which typically is of a longer wavelength than illumination light 106) . Mirror 112 which reflects the illumination light towards the sample and the emission light towards the further mirror 120 may be configured as a rotatable mirror which can be controlled by a computer system (not shown) . This mirror may be referred to as a scanning mirror. The scanning mirror may be configured to move the focused illumination light spot 117 over or through the sample 116. To that end, scanning mirror 112 may be configured to rotate back and forth over a first angular amplitude Al 138 in order to move ("scan") the focused illumination light spot over or through sample causing local optical illumination along its way. In particular, in order to move the light spot in the X,Y plane of the sample, the rotatable scanning mirror may be controlled by a number of computer-controlled actuators. This way the rotatable scanning mirror is used in order to scan illumination light spot over the surface of the sample in accordance to a particular scanning pattern. For example, in an embodiment, a scanning pattern may be a meandering pattern. The scanning pattern may be configured so that after scanning a particular area, the scanned area is fully exposed by the moving illumination light spot .
During scanning, emission light originating from the moving illumination light spot is split into multiple
collimated beams 118,118a of illumination light. The
collimated beams may be reflected via the scanning mirror 112 towards the further mirror 120, for example a rotatably mounted mirror, which is controlled to reflect the light beams towards the focusing lens in order form multiple moving emission light spots 126a, 126b onto the imaging plane of the imaging system 124.
The further mirror 120 may be configured to rotate back and forth over a second angular amplitude A2 140 in order to move (scan) the focused emission light spots over the imaging plane of the image sensor, while the focussed
illumination light spot is moved (scanned) over the sample by the scanning mirror. Further mirror 120 may hereafter be referred to as the rescanning mirror. The actuators and/or electro-motors of the scanning and rescanning mirror may be controlled by a computer system so that the mirrors can be moved in sync. Here, the frequency of the back and forth rotation of the scanning mirror is identical to the frequency of the back and forth rotation of the rescanning mirror. This way, when the scanning mirror is controlled by the computer system to scan an area of the sample by the moving
illumination light spot, the rescanning mirror is controlled by the computer system to scan associated areas of pixels of the imaging plane of the image sensor so that the pixels are exposed by moving emission lights spots. This way, when an area of a sample is optically excited by the illumination beam, emission light originating from the excited area is imaged onto the imagining plane.
In an embodiment, the second angular amplitude of the rescanning mirror may be larger than the first angular
amplitude of the scanning mirror so that high-resolution images of the scanned sample area can be obtained. For
example, in an embodiment, the second angular amplitude may be twice the first angular amplitude, so that the resolution of the image is improved by a factor with respect to the resolution that may be obtained when the second angular amplitude is chosen to be equal to the first angular
amplitude .
The light source 102 may comprise a plurality of light sources, e.g. a plurality of lasers, and one or more light filters. The light sources and filters may be used to control which wavelengths the illumination light comprises. In an embodiment, the light source may be configured to generate white light. In another embodiment, the light source may be configured to generate light consists of two or more
predetermined wavelengths or wavelength bands. For example, in an embodiment, the light source may generate light of a first wavelength selected from the blue band of the visible spectrum and light of a second wavelength selected from the yellow band of the visible spectrum. The optical elements of optical system 132 may be configured to control different wavelengths. For example, mirror 108 may reflect the illumination light onto dichroic mirror 110 that may be configured to reflect light of a first group of wavelengths, e.g. blue light and yellow light.
Dichroic mirror 110 may be further configured to pass light of a second group of wavelengths, e.g. red and green light. This way, the illumination light may be reflected by dichroic mirror 110 onto scanning mirror 112. Scanning mirror may reflect the illumination light onto lens 114 which may focus the illumination light onto the sample 116 creating a focused illumination light spot 117.
In an embodiment, the illumination light may comprise light of a first wavelength, e.g. yellow light, which may cause a first type of optical excitations in the sample, the first type of excitations generating first emission light.
Further, the illumination light may comprise light of a second wavelength, e.g. blue light, which may cause a second type of optical excitations in the sample, the second type of
excitations generating second emission light. Thus the above- mentioned first group of wavelengths, which may be reflected by dichroic mirror 110, may comprise said first wavelength and said second wavelength.
In an embodiment, sample may be sample, e.g. a biological material, that is imaged using a reflective
microscopy technique. In sample 116 may be an optically active sample. For example, in an embodiment, the sample may be a material, e.g. a biological material, comprising one or more types of fluorescent /luminescent materials, such as
fluorophores . In another embodiments, the optically active sample may be non-luminescent optical active material. For example, the sample may be a material that can be imaged on the basis of second harmonics generation (SHG) microscopy or third harmonics generation (THG) microscopy. In yet a further embodiment, the sample may be a material that can be imaged using a Rahman microscopy technique, such as Coherent Anti- Stokes Rahman Scattering (CARS) microscopy. For example, the sample comprise at least a first type of fluorophores, e.g. red fluorophores, and a second type of fluorophores, e.g. green fluorophores . The light of the first wavelength of the illumination light may cause the first fluorophores in the sample to emit the first emission light and the light of the second wavelength may cause the second fluorophores in the sample to emit second emission light.
Hence, the emission light may comprise light that is emitted as a result of photoluminescence, preferably
fluorescence. In an embodiment, a blue light component of the illumination light may cause green fluorophores at the focused illumination light spot to emit green light. Similarly, yellow light component of the illumination light may cause red fluorophores at the focused illumination light spot to emit red light. In this example, the emission light thus comprises green light and red light.
The path of the illumination light that travels from dichroic mirror to the sample may coincide with the path of the emission light that travels from the sample to the
dichroic mirror. The emission light however, may pass through dichroic mirror, since it may be configured to pass light of the second group of wavelength, (e.g. red and green light) . The second group of wavelengths may thus comprise the
wavelengths of the first and second emission light.
Hence, from the above it follows that different photo-luminescent sites which are sensitive to light of different wavelengths and which - in response - emit light of different wavelengths, may be used. This way, the emission light originating from the sample may include optical
information associated with different excitation sites of the sample. In an embodiment, emission light beam 118 may be split into a first emission light beam 118a and second emission light beam 118b. In an embodiment, forming the first and second emission light beams may include spitting the emission light beam in a first and second light beam of different wavelengths . The first and second emission light beams may be formed on the basis of a variety of optical properties of the emission light. For example, in an embodiment, the emission light beams may be formed on the basis of the wavelength, wherein the first emission light beam comprises a first wavelength and the second light beam a different second wavelength. In another embodiment, the emission light beams may be formed on the basis of polarization, wherein the first light beam comprises a first polarization and the second light beam a different second polarization. The second optical system of the scanning microscopy system may comprise an optical splitter that is configured to split the emission light into different emission light beams on the basis of at least one optical properties, such as wavelength,
polarization, etc.
In an embodiment, a light splitter such as an optical splitter 134 for splitting the emission light into different emission light beams, e.g. a first emission light 118a and second emission light 118b. In an embodiment, the light splitter 128 may include at least a dichroic mirror 128 and a mirror 130 in the path of the emission light beam 118
originating from the scanner mirror. Instead of a dichroic mirror other optical splitter elements may be used including but not limited to a polarization splitter, a grey splitter or an acousto-optical tunable splitter.
In this embodiment, the emission light beam 118 may be incident on dichroic mirror 128, which may be configured to reflect the first emission light, e.g. green light, and to let pass the second emission light, e.g. red light. Hence,
dichroic mirror 128 may split the emission light into a first emission light beam 118a, which may comprise the first
emission light, e.g. red light, and a second emission light beam 118b, which may comprise the second emission light, e.g. green light. Hence, the first emission light beam may comprise first optical information, e.g. information relating to red fluorophores in the sample, and the second emission light beam 118b may comprise second optical information, e.g. information relating to green fluorophores in the sample.
The first emission light beam 118a may be incident onto rescanning mirror 120, while the second emission light beam 118b may be directed onto mirror 130, which may reflect the second emission light beam 118b onto rescanning mirror 120. The light splitter may be configured so that the split emission light beams, e.g. first emission light beam 118a and second light beam 118b, may be incident on rescanning mirror 120 under different angles. Rescanning mirror may reflect the emission light beams onto different areas of the image plane of the imaging system.
Rotatable rescanning mirror may reflect both emission light beams 118a, 118b onto the focusing lens which may focus the light beams onto an image plane of imaging system. This way, when rescanning the focused emission light spots onto the imaging plane, at least a first and a second image are formed simultaneously. The first emission light beam may cause a first illumination spot at a first position 126a, while the second emission light beam may cause a second illumination spot at a second position 126b that is different from the first position.
In an embodiment, filters such as color filters, e.g. a first color filter 123a and second color filter 123b, may be positioned in front of the imaging system. A color filter may be configured as a bandpass filter that is configured to pass light in a predetermined band of wavelengths. For example, the first color filter 123a may be configured to pass first
emission light, e.g. red light, and the second color filter 123b may be configured to pass second emission light, e.g.
green light.
During operation, the scanning mirror may rotate over a smaller angular amplitude than the rescanning mirror. Due to the rotation of the scanning mirror, the focused illumination light spot may move over or through the sample. Due to the rotation of the rescanning mirror, the focused emission light beams, and thus the emission light spots and, may move over the image area of imaging system 124 thereby exposing pixels to the emission light.
Moving the scanning mirror may comprise rotating the scanning mirror over a first angular amplitude and moving the rescanning mirror may comprise rotating the rescanning mirror over a second angular amplitude. In an embodiment, the first angular amplitude may be different from the second angular amplitude, in particular the first angular amplitude may be smaller than the second angular amplitude, in particular the first angular amplitude may be approximately half the second angular amplitude. This way, multiple (in the example two) high-resolution images may be generated by the imaging system. These high-resolution images may be combined into one color image of the area on the sample that was excited by the illumination beam.
Although Fig. 1 depicts an example wherein the emission light beam is split in two emission light beams for simultaneous imaging two images of a sample area comprising different optical information, it is apparent for a skilled person that this scheme may be extended to embodiments that are configured to split the emission light beam in multiple (more than two) emission light beams so that multiple images comprising different optical information.
Hence, scanning microscopy system depicted in Fig. 1 is capable simultaneously capturing super-resolution images of excited areas of a sample and combining these images into a super-resolution colour image. As will be described hereunder in more detail, the system is suitable of colour imaging time- critical processes of an area of a sample with super- resolution using conventional refractive and reflective optical elements.
Certain reflective elements in the scanning microscopy system such as mirrors 108,130 and dichroic mirrors 110,128 may be implemented as static optical elements. Such optical elements cannot be controlled by the computer system. Other reflective elements may be implemented as rotatable optical elements that can be controlled by the computer system. The path of illumination light 106 from light source 102 towards the scanning mirror 112 may be controlled using static optical elements. During a scan, the scanning mirror 112 may rotate (i.e. rotatable moving back and forth) over an angular amplitude which causes the path of the illumination light 106 from scanning mirror 112 to the sample 110 (and thus the focused execution light spot) to move over the surface of the sample in accordance with the movements of the scanning mirror. Hence, the path of the illumination light 106 from scanning mirror 112 to sample 116 is said to be dynamic in the sense that it is moving with respect to the sample and/or light detector during scanning. Consequently the point of illumination in the sample is moving during scanning and therefore also the point where emission signal comes from from the sample is moving during scanning.
The scanning mirror both directs the illumination light signal from the light source 102 towards the sample 116 as well as the emission light signal from the sample 116 towards the rescanning mirror 120. This way, the movement of the emission light signal originating from the sample is neutralized by the movement of the scanning mirror.
Hence, due to the movement of the scanning mirror, the path of emission light from scanning mirror to rescanning mirror (the paths of light beams 118a and 118b from dichroic mirror 128 to rescanning mirror) is static. In other words, the scanning mirror 112 "descans" the emission light 118 and reflects the emission light as a static emission light beam towards the rescanning mirror. Further, the path of the emission light between rescanning mirror and light detector may be dynamic during scanning due to the movement of the rescanning mirror. The optical beam splitter, comprising e.g. a dichroic mirror, may be positioned in a static path of emission light.
Fig. 2A-2C schematically depict a top view of the scanning movements of an illumination light spot 217 and one or more associated emission light spots 226a, 226b of a
scanning microscopy system according to an embodiment of the invention. In particular, Fig. 2A-2C depict the movement of a focused illumination light spot 217 over or through optical active sample 216 and the movement of the associated focused first and second emission light spots 226a, 226b over the image area of the imaging system 224 at three different moments in time TO, Tl and T2. In this example, the illumination light may comprise multiple wavelengths, e.g. a first and a second wavelength, for exciting different optical illumination sites in the sample.
Fig. 2A depicts the start of the scanning process for both the illumination light spot and the associated emission light spots. Fig. 2B depicts the movement of the illumination light spot and the associated focused first and second
emission light spots over the image area of the imaging system 224 at an intermediate time instance Tl. Here, time instance
TO may mark the start of a scan wherein the illumination light spot will start exposing a predetermined area by moving
("scanning") the light spot according to a predetermined scanning pattern. In an embodiment, the scanning pattern may be a meandering pattern. The light spots may be moved by a computer system controlling the scanning and descanning mirror using actuators or the like as described with reference to
Fig. 1.
When exposing the sample with an illumination light spot, emission light originating from the sample is split into different emission light beams which are focused as emission light spots, in this example a first and second emission light spot 226a, 226b, onto the imaging plane of the imaging system wherein each emission light beam carries a different type of optical information of the area of the sample that is excited by the illumination light spot. During the scanning of the illumination light spots over the imaging area, pixels are exposed and image data associated with a particular position on the sample are captured by the pixels.
Fig. 2C depicts the scanning pattern at a second time instance T2 (the end of a scan) wherein a full scan of the scanned sample area 228 may results in two scanned areas 230a, 230b of the imaging plane of the imaging system wherein each scanned area of the imaging plane may result in an image of the scanned sample, wherein each image may include
different optical information of the sample.
The imaging area of the imaging system may be
realized on the basis of one or more image sensors, e.g. image sensors that are tiled together in order to form a pixelated imaging area. In an embodiment, an image sensor may include one or more pixel arrays that are configured to simultaneously image the different illumination light beams. An image sensor may be implemented as a CCD image sensor or a CMOS image sensor .
The speed with which the focused illumination light spot moves over or through optical active sample 216 may be lower than the speed with which the first and second emission light beams move over the image area of the imaging system. In an example, said speed of the focused illumination light spot 217 may be approximately half the speed of the emission light beams, in particular of the first and second illumination spots.
Fig. 3 depicts a flow diagram of a scanning process according to an embodiment of the invention. In particular, the figure depicts a scanning process that may be executed by a scanning microscope system as described with reference to Fig. 1. The process may be executed by a computer system configured to control the light source and the actuators for controlling the movements of the scanning and rescanning mirror .
In a first step 302 the computer system may control the scanning mirror of a first optical system to move (scan) one or more focused illumination light spots over (or through) an optical active sample, wherein the one or more focused illumination light spots cause local optical excitations in the optical active sample.
Local optical excitations may generate emission light which in a second step 304 may be picked up by the first optical system in order to form an emission light beam from the emission light originating from the optical excitations. The first optical system may direct the emission light beam onto the scanning mirror which may reflect the emission light beam via a second optical system to an imaging system (step 304) .
The second optical system may include a light
splitter for splitting the emission light beam in at least a first emission light beam comprising first optical information and second emission light beam comprising second optical information (step 306) . Then, the light splitter may direct the first and second emission light beams onto a computer- controlled rescanning mirror, which is used to move (can) focused emission light beam spots over the imaging plane of the image system (step 308) in order to form a first image comprising the first optical information and second image comprising the second optical information, while the sample is scanned with the illumination light.
The computer system may be configured to
independently control the angular amplitude of the scanning and rescanning mirror so that the advantageous resolution improvement associated with the RCM technique may be achieved. This way, the system is capable of simultaneously generating multiple high-resolution images of a sample area that is excited with one or more illumination light spots. This way, the rescan microcopy system allow the imaging of time-critical processes .
FIG. 4 depicts a scanning microscopy system according to another embodiment of the invention. Fig. 4 depicts a scanning microscopy system that is similar to the scanning microscopy system of Fig. 1, comprising a light source 402, a first optical system 432 comprising a scanning mirror 412 that is configured to guide an illumination light beam from the light source towards the scanning mirror and to expose a sample 416 with a moving illumination light beam, a second optical system 436 comprising a rescanning mirror 420
configured to guide an emission light beam formed on the basis of emission light originating from the sample via the
rescanning mirror onto an imaging system 424.
The embodiment of Fig. 4 is configured to measure lifetime luminescence properties. As will be explained
hereunder in more detail, the properties can be derived from a phase difference between a modulated illumination light signal and an emission light signal, because the phase difference depends on the lifetime properties of the sample as e.g.
described in M.Ballwe et al . "An Error Analysis of the Rapid Lifetime Determination Method for the Evaluation of Single
Exponential Decays", in Anal. Chem. 1989, Vol. 61, p. 30-33.
An example of lifetime imaging on the basis such modulated light signal is described by M. Raspe et al . , in siFLIM: single-image frequency-domain FLIM provides fast and photon-efficient lifetime data, Nature Methods, Vol. 13, No.6, June 2016, p. 501-504. Typically, these measurements include the use of an imager sensor with on-chip phase-sensitive detection that simultaneously record two images shifted by n radians in phase.
In order to derive the phase difference, precise knowledge of the variation of the intensity of the emission is required. To that end, the emission light signal may be integrated by the imaging system over a number of very short time intervals. The Nyquist criterion stipulates that for mono-exponential life time measurements at least two time intervals or phase ranges are required per period (e.g. 0-180 and 180-360 degrees) . Similarly, for multi-exponential life time measurements more than two time intervals or phase ranges, (e.g. 0-90, 90-180, 180-270, 270-360 degrees) are needed per period of the emission signal in order to derive the emission signal. However, integrating the time-varying intensity of the emission light in such short phase ranges is very difficult for an image sensor, because of the very short exposure times associated with these phase ranges.
As will be described hereunder in more detail, the second optical system 436 of the scanning microscopy system may be configured to split the modulated emission light beam into a first modulated emission light beam and into one or more further modulated emission light beams, wherein the one or more further modulated emission light beams are phase- shifted with respect to the first modulated emission light beam. In an embodiment, the phase shift may be 90 degrees. Both modulated emission light signals are then used in order to determine the emission light signal.
Fig. 5A-5C depicts a data acquisition process for a scanning microscopy system according to an embodiment of the invention. In particular, the figures depict a data
acquisition process using a modulated emission light signal and a phase-shifted modulated emission light signal for lifetime measurements. Fig. 5A illustrates an illumination light signal that is modulated using a frequency in a range selected between 1 kHz and 100 MHz, typically 40 MHz. Due to the modulation of the illumination signal, the emission light signals will be modulated as well.
Fig. 5B depicts the first modulated emission light signal which has a certain amplitude and phase shift with respect to the illumination signal. Fig. 5C depicts a second modulated emission light signal which represents a 90 degrees phase-shifted version of the first modulated emission light signal .
As shown in Fig. 5B, first images of the first modulated emission light signal may be captured by the imaging system at predetermined time instances TO , Tl , T2 , T3 , wherein the exposure time of the imaging system is defined by an integration window T0-T1. The time instances are synchronized with the modulation frequency in the sense that at time instances T0,T2,T4 the phase angle of the signal is
predetermined angle phi and at time instances T1,T3,T5,... the phase angle of the signal is phi plus 180 degrees. At the same time, second images of the second modulated emission light signal (a 90-degree phase shifted version of the first
modulated emission signal) are capture at the predetermined time instances TO , Tl , T2 , T3 , .... Hence, pixel values of the first images generated during integration window in which the phase of the signal is 0-180 degrees (region I in Fig. 5B) represent an average value of the intensity of the first modulated emission signal in that window. Similarly, pixel values of images generated during integration window in which the phase of the signal is 180-360 degrees (region II in Fig. 5B) represent an average value of the intensity of the first modulated emission signal in that window. At the same time, second images are generated on the basis of the second modulated emission signal in the integration window in which the phase of the signal ranges 90- 270 degrees (region III in Fig. 5C) and 270-90 degrees (region IV in Fig. 5C) . Based on the average signal values of the four regions, the original response of the emission light signal may be obtained and subsequently the phase difference with the driving modulated illumination signal may be determined.
In order to achieve a data acquisition scheme as described with reference to Fig. 5A-5C, the scanning
microscopy system may comprise an optical source for
generating, the optical source may be configured to vary the intensity of the one or more focused illumination light spots with a modulation frequency, e.g. a sinusoidal modulation frequency, so that also the intensity of the first and second emission light spot will vary as a function of time.
In order to realize phase-shifted emission light beams, the second optical system of the microscopy system of Fig. 4 may comprise an optical splitter 427, e.g. in the form of reflective mirror, for splitting the emission light into different emission light beams, e.g. a first emission light beam and a second emission light beam. The second optical system may further include an optical delay line 434 in order to introduce phase shift in the modulated second emission light signal. This way, a predetermined phase difference between the time-varying intensity of the first emission light spot 426b and the second emission light spot 426a may be introduced . In an embodiment, the optical delay line may comprise one or more mirrors, e.g. a first mirror 438 and a second mirror 440, in order to control the length of the optical path. The optical delay line shown in Fig. 4 may introduce a predetermined phase shift in the modulation signal of the second emission light spot 426b with respect to that of the first emission light spot 426a. The phase shift may be at least 45 degrees, preferably by at least 90 degrees, most preferably approximately 90 degrees (as e.g. depicted in Fig. 5B and 5C) .
The phase shifted emission light beams are directed towards the rescanning mirror, which is controlled to move (scan) focused emission light beam signals over the imaging plane of the image system in order to simultaneously form a first image associated with the first emission light beam and a second image associated with the second emission light beam, while the sample is scanned with the illumination light.
Hence, the optical splitter and the optical delay line enable the scanning microscopy system to simultaneously capture an image of the modulated emission light signal and an image of a 90 degrees phase-shifted version of the modulated emission light signal. For example, an image generated during an integration window in which the phase of the signal is 0-180 degrees (region I in Fig. 5B) may be captured simultaneously with an image of the signal in an integration window in which the phase of the signal is 90-270 degrees (region III in Fig. 5C) . This way, the scanning microscopy system according the invention allows lifetime measurements using a RCM super- resolution technique.
In further embodiments, the second optical system in
Fig. 4 may be configured to produce more than two phase shifted emission light beam, using e.g. a beam splitter that splits the emission light beam into multiple emission light beams and one or more optical delay lines configured to introduce different phase shifts in each of the multiple emission light beams. Hence, as shown in Fig. 4 and 5 the invention allows lifetime luminescence properties using a scanning microscope. The invention eliminates the need for an image sensor that has on-chip phase-sensitive detection functionality.
Instead of a phase modulation technique as described with reference to Fig. 4 and 5, the lifetime imaging may also be based on other well-known techniques such as a pulsed illumination technique. Hence, in an embodiment, the light source may be configured as a pulsed light source, e.g. one or more pulsed lasers.
Fig. 6 depicts a flow diagram of a scanning process according to an embodiment of the invention. In particular, flow diagram depicts a scanning process that may be executed by a scanning microscope system as described with reference to Fig. 4 and 5. The process may be executed by a computer system configured to control the light source and the actuators for controlling the movements of the scanning and rescanning mirror .
In a first step 602 the computer system may control a scanning mirror of a first optical system to move (scan) one or more focused modulated illumination light signals over (or through) an optical active sample, wherein the one or more focused modulated illumination light spots cause optical excitations in the optical active sample.
Local optical excitations may generate modulated emission light which in a second step 604 may be picked up by the first optical system in order to form a modulated emission light signal from modulated emission light originating from the optical excitations and directing the modulated emission light signal onto the scanning mirror for reflecting the emission light signal via a second optical system to an imaging system.
A light splitter in the second optical system
splitting the modulated emission light signal in at least a first modulated emission light signal and second modulated emission light signal representing a phase-shifted version of the first modulated light signal (step 606) . Then, the light splitter may direct the first and second modulated emission light signals onto a rescanning mirror, which is controlled to move (scan) focused emission light signals over the imaging plane of the image system in order to simultaneously form a first image of the first modulated light signal and second image of the second
modulated light signal, while the sample is scanned with the modulated illumination light (step 608).
Instead of using a modulated emission light signals and a phase-shifted modulated emission light signal as
described with reference to Fig. 4-6, multiple differently phase-shifted modulated emission light signals may be used so that instead of two images a plurality of images of different parts of the modulated emission light signal may be obtained. By simultaneously imaging predetermined parts of the modulated emission light signal, the original emission light signal can be constructed and luminescent lifetimes can be determined.
The computer system may be configured to
independently control the angular amplitude of the scanning and rescanning mirror so that the advantageous resolution improvement associated with the RCM technique may be achieved in the life-time measurement scheme of Fig. 6.
Fig. 7 depicts a scanning microscopy system according to yet another embodiment. In particular, Fig. 7 depicts a scanning microscopy system similar to the system described with reference to Fig. 1. In contrast with the system of Fig. 1, the system in Fig. 7 comprises a second optical system 736 including an optical arrangement 739 for enabling confocal measurements wherein the optical arrangement may comprise a pinhole 744. The scanning microscopy may be configured to focus illumination light spot 717 in a plane of interest P. Additionally, the system may be configured to arranged to focus emission light from the plane of interest P onto plane P ' . For example, the system may comprise one or more optical elements, e.g. lenses 714 and 740, arranged to focus emission light from the plane of interest P onto plane P ' . Plane P' thus may be a confocal conjugate plane of plane of interest P. The optical arrangement may comprise a further optical element, e.g. lens 742, arranged to form a collimated light beam of the emission light 718 that passes pinhole 744.
Emission light from the plane of interest in the sample may pass through the pinhole while emission light 718 that is emitted from other planes than the plane of interest is substantially blocked by the pinhole so that the signal-to- noise ratio of the emission light signal that is focused onto the imaging plane of the imaging system may be substantially improved.
Fig. 8 depicts a scanning microscopy system according to yet another embodiment. In particular, Fig. 8 depicts a scanning microscopy system comprising a first optical system 832 comprising a number of optical elements, e.g. refractive and/or diffractive optical elements, arranged to form a first focused illumination light spot 817a at a first position in or on the sample and a second focused illumination light spot 817b at a second position in or on the sample. The first and second position may have a different lateral position.
The first optical system further comprises at least one scanning reflector 812 for moving the light spots over or through the optical active sample 816. The first illumination light spot 817a may be formed using a first illumination light beam 806a and the second illumination light spot 817b may be formed using a second illumination light beam 806b. The first illumination light beam may comprise light of a first
wavelength, e.g. yellow light, and the second illumination light beam may comprise light of a second wavelength, e.g. blue light. The first and second illumination light beams may be formed on the basis of light of light source. In an
embodiment, the first and second light spots may be arranged close together so that the spots illuminate a small area very close to each other. Light of two or more wavelengths may be split using a first optical splitter 846, for example dichroic mirror. The optical splitter 46 may be configured to reflect light of a second wavelength, e.g. blue light, and let pass light of a first wavelength, e.g. yellow light.
Thus the first illumination beam 806a may comprise the first wavelength and the second illumination beam 806b may comprise the second wavelength. The illumination beams may be directed on mirror 808 that may reflect both light beams onto mirror 10. Mirror 10 may reflect both the first and second illumination light beam onto scanning mirror 812. The scanning mirror may reflect the two illumination light beams onto sample 16 which causes local optical excitations generating emission light at the two focused illumination light spots 817a and 817b. In an example, a yellow illumination light spot 817a may cause emission of red emission light and a blue focused illumination light spot causes emission of green emission light.
The system may comprise a second optical system for directing the emission light via a rescanning mirror onto an imaging system that is similar to the second optical system of Fig. 1. The emission light, both the first and second emission light, may be incident on scanning mirror 812 and may pass through dichroic mirror 810. Dichroic mirror 828 may reflect the second emission light, e.g. the green light, and let pass the first emission light, e.g. the green emission light.
Hence, the first emission light beam 818a comprising the first emission light and the second emission light beam 818b
comprising the second emission light are formed.
Both emission light beams may be directed onto rescanning mirror 820 which may direct the two beams onto the image sensor. A first image and second image are formed by focusing with lens 822 the first and respective second
emission light beam onto the image plane of image sensor 824. The two emission light beams 18a and 18b cause first
illumination spot 826a and second illumination spot 826b respectively. Hence, this embodiment allows to simultaneously illuminate and image an area of the sample using different illumination light signals. This way, the first optical information in the first emission light beam and the second optical information in the second emission light beam may relate to the same area of the sample.
Fig. 9 depicts a scanning microscopy system according to another embodiment. In this embodiment, the scanning microscopy system may be configured such that the first and second illumination light spot may be at different depths into the sample. Herein an optical splitter 947 may direct part, e.g. 50%, of the incoming illumination light 906 towards mirror 948 and may let pass part, e.g. 50%, of the incoming illumination light. The two illumination light beams 906a and 906b may comprise the same wavelengths of light. Again, the two illumination light beams may be directed onto scanning mirror 912. The first illumination light beam may be directed via a first lens 907a onto lens 914, thereby focusing the first illumination light beam 906a onto the sample creating a first focused illumination light spot 917a at a first focus plane PI in the sample, which causes emission of first
emission light. The second illumination light beam may be directed via a second lens 907b onto lens 914 thereby focusing the second light beam onto the sample 916. The second
illumination light beam 906b causes emission of second
emission light at the second focused illumination light spot 917b at a second focus plane P2 in the sample wherein the first and second focus plane are at different axial positions in the sample. Two lenses associated with different focal depths PI, P2 may be used to create the two focused
illumination light spots at different depths (in the Z- direction) into the sample. The first illumination light spot may cause emission of emission light from a first plane of interest PI and the second illumination light spot may cause emission of emission light from a second plane of interest P2.
In this embodiment the step of forming the first and second emission light beams on the basis of the emission light may comprise directing the first emission light through a first optical path and directing the second emission light through a second optical path, wherein the first optical path is different from the second optical path. In the depicted example, the first and second emission light beams are formed on the basis of the emission light in the sense that the second emission light is directed on mirror 930 whereas the first emission light is not. Also note that the first light beam 18a which comprises the first emission light may be focused upon a pinhole 944a. The pinhole 944a is placed in a conjugate focal plane PI' to plane of interest PI. Similarly, the second emission light beam 18b is focused upon pinhole 944b. Pinhole 944b is in a conjugate focal plane P2 ' to plane of interest P2. The pinholes 944a and 944b provide images of the emission light with an improved signal-to-noise ratio.
Various embodiments of the invention may be imple¬ mented as a program product for use with a computer system, where the program(s) of the program product define functions of the embodiments (including the methods described herein) . In one embodiment, the program(s) can be contained on a vari¬ ety of non-transitory computer-readable storage media, where, as used herein, the expression "non-transitory computer readable storage media" comprises all computer-readable media, with the sole exception being a transitory, propagating signal. In another embodiment, the program(s) can be contained on a vari¬ ety of transitory computer-readable storage media.
Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media (e.g., read- only memory devices within a computer such as CD-ROM disks readable by a CD-ROM drive, ROM chips or any type of solid- state non-volatile semiconductor memory) on which information is permanently stored; and (ii) writable storage media (e.g., flash memory, floppy disks within a diskette drive or hard- disk drive or any type of solid-state random-access semicon¬ ductor memory) on which alterable information is stored. The computer program may be run on the processor 102 described herein .
The terminology used herein is for the purpose of de- scribing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms "a, " "an, " and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising, " when used in this specification, specify the presence of stated features, integers, steps, operations, ele¬ ments, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, oper¬ ations, elements, components, and/or groups thereof.
The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of embodiments of the present invention has been presented for purposes of illustration, but is not intended to be exhaustive or limited to the implementations in the form disclosed. Many modifications and variations will be apparent to those of or¬ dinary skill in the art without departing from the scope and spirit of the present invention. The embodiments were chosen and described in order to best explain the principles and some practical applications of the present invention, and to enable others of ordinary skill in the art to understand the present invention for various embodiments with various modifications as are suited to the particular use contemplated.

Claims

1. A method for forming images of a sample, preferably an optically active sample such as a photo- luminescent sample, using scanning microscopy comprising:
controlling a scanning mirror of a first optical system to move (scan) one or more focused illumination light signals over (or through) an optical active sample, wherein the one or more focused illumination light signal cause one or more optical excitations in the sample;
the first optical system forming at least one emission light signal from emission light originating from the one or more optical excitations and directing the emission light signal onto the scanning mirror for reflecting the emission light signal via a second optical system to an imaging system;
a light splitter in the second optical system splitting the at least one emission light signal in at least a first emission light signal comprising first optical
information and second emission light signal comprising second optical information, preferably the first optical information being different from the second optical information; and,
the second optical system directing the first and second emission light signals onto a rescanning mirror, the rescanning mirror controlling a focused emission light beam signals to move (scan) over the imaging plane of the imaging system for simultaneously forming a first image associated with the first optical information and second image associated with the second optical information, while the sample is scanned with the one or more illumination light signals.
2. The method according to claim 1 wherein the one or more excitations comprise at least a first type of excitation causing emission of a first wavelength or a first wavelength band and a second type of excitation causing emission of a second wavelength or second wavelength band different from the first wavelength or first wavelength band; and, wherein the first optical information is associated with the emission of the first wavelength or a first wavelength band and the second optical information is associated with the emission of the second wavelength or second wavelength band; preferably the method further comprising: the imaging system capturing first image data associated with the first emission light and a second image data associated with the second emission light; and, forming an image of at least part of the sample by comprising at least part of the first and second image data.
3. The method according to claims 1 or 2, further comprising :
modulating the intensity of the at least one
illumination light signal in accordance with a modulation frequency, preferably the modulation frequency being selected between 1 and 200 MHz; the modulation of the illumination light signal causing a modulation of the intensity of the first and second emission light signal;
phase-shifting the modulated second emission light signal with respect to the modulated first emission light signal, preferably the phase-shifting including directing the modulated second emission light signal through an optical delay line.
4. The method according to claim 3 further comprising :
the imaging system simultaneously capturing first image data associated with the modulated first emission light signal and second image data associated with the time-shifted modulated second emission light signal; and,
a processor determining a life-time associated with at least one of the one or more optical excitations on the basis of at least part of the first image data and at least part of the second image data.
5. The method according to any of the preceding claims, comprising
focusing at least part of the emission light signal upon a pinhole aperture, preferably the focusing including using at least one first optical element for focusing the emission light signal originating from the scanning mirror onto the pinhole aperture and at least one second optical element configured to form a collimated emission light signal of the emission light that passes pinhole and to direct the collimated emission light signal towards the rescanning mirror .
6. The method according to any of the preceding claims ,
wherein the one or more focused illumination light signals comprise at least a first focused illumination light spot at a first position of a focusing plane in the sample and a second focused illumination light spot at a second position in the focusing plane in the sample, wherein the first
illumination light is associated with a first wavelength or wavelength band and the second illumination light is
associated with a second wavelength or wavelength band;
wherein
the first focused illumination light spot causes a first optical excitation at the first position; and, the second focused illumination light signal causes a second optical excitation at the second position; and,
wherein the first emission light signal comprises emission light of the first optical excitation and the second emission light beam comprises emission light of the second optical excitation.
7. The method according to any of claims 1-6, wherein the one or more focused illumination light signals comprise at least a first focused illumination light spot at a first position in a first focusing plane in the sample and a second focused illumination light spot at a second position in a second focusing plane in the sample, wherein
the first and second focusing plane are at different axial positions in the sample.
8. Method according to any of claims 1-6 wherein controlling the scanning and the rescanning mirror includes rotating the scanning mirror and the rescanning mirror back and forth over a predetermined first angular amplitude and a second angular amplitude respectively; wherein the second angular amplitude of the rescanning mirror is larger than the first angular amplitude of the scanning mirror.
9. A scanning microscopy system for forming images of a sample, preferably an optically active sample, comprising:
a light source, preferably one or more lasers, configured for generating light for illuminating the sample;
a first optical system comprising a scanning mirror, the optical system being configured: to focus illumination light of the light source as one or more illumination light signals via the scanning mirror onto the sample, the one or more focused illumination light signals causing one or more optical excitations in the optical active sample; to form an emission light signal from emission light originating from the one or more optical excitations in the sample; and, to direct the emission light signal onto the scanning mirror;
a second optical system comprising a rescanning mirror and a light splitter, the light splitter being
configured: to split the at least one emission light signal originating from the scanning mirror in at least a first emission light signal comprising first optical information and second emission light signal comprising second optical information; the second optical system being configured: to direct the first and second emission light signals towards the rescanning mirror and to focus the first and second emission light signals onto an imaging plane; an imaging system configured to receive first and second emission light signals originating from the rescanning mirror; and,
a computer system being adapted to:
control the scanning mirror of the first optical system to move (scan) the one or more focused illumination light signals over (or through) an optical active sample;
control the rescanning mirror of the second optical system to move (scan) a focused emission light beam signal over the imaging plane of the imaging system; and,
control the imaging system for simultaneously capturing first image data associated with the first optical information and second image data associated with the second optical information, while the sample is scanned with the one or more illumination light signals.
10. The system according to claim 9, wherein the one or more excitations comprise at least a first type of
excitation causing emission of a first wavelength or a first wavelength band and a second type of excitation causing emission of a second wavelength or second wavelength band different from the first wavelength or first wavelength band; and, wherein the first optical information is associated with the emission of the first wavelength or a first wavelength band and the second optical information is associated with the emission of the second wavelength or second wavelength band;
11. The system according to claims 9 or 10, further comprising :
a modulator for modulating the intensity of the at least one illumination light signal in accordance with a modulation frequency, preferably the modulation frequency being selected between 1 and 200 MHz; the modulation of the illumination light signal causing a modulation of the
intensity of the first and second emission light signal;
a phase-shifter, preferably an optical delay line, configured to phase-shift the modulated second emission light signal with respect to the modulated first emission light signal .
12. The system according to claim 11 further
comprising:
the imaging system configured to simultaneously capture first image data associated with the modulated first emission light signal and second image data associated with the time-shifted modulated second emission light signal; and, the computer system further being adapted to:
determine a life-time associated with at least one of the one or more optical excitations on the basis of at least part of the first image data and at least part of the second image data .
13. A computer system for controlling scanning microscopy system for forming images of a sample comprising, the system comprising:
a light source, preferably one or more lasers, configured for generating light for illuminating the sample;
a first optical system comprising a scanning mirror, the optical system being configured to focus illumination light of the light source as one or more illumination light signals via the scanning mirror onto the sample, the one or more focused illumination light signals causing one or more optical excitations in the optical active sample; to form an emission light signal from emission light originating from the one or more optical excitations in the sample; and, to direct the emission light signal onto the scanning mirror;
a second optical system comprising a rescanning mirror and a light splitter, the light splitter being
configured to split the at least one emission light signal originating from the scanning mirror in at least a first emission light signal comprising first optical information and second emission light signal comprising second optical information, the second optical system being configured to direct the first and second emission light signals towards the rescanning mirror and to focus the first and second emission light signals onto an imaging plane;
an imaging system configured to receive first and second emission light signals originating from the rescanning mirror;
wherein the computer system comprises a computer readable storage medium having computer readable program code embodied therewith, and a processor, preferably a
microprocessor, coupled to the computer readable storage medium, wherein responsive to executing the first computer readable program code, the processor is configured to perform executable operations comprising:
controlling the scanning mirror of the first optical system to move (scan) the one or more focused illumination light signals over (or through) an optical active sample;
controlling the rescanning mirror of the second optical system to move (scan) a focused emission light beam signal over the imaging plane of the imaging system; and,
controlling the imaging system to simultaneously first image data associated with the first optical information and second image data associated with the second optical information, while the sample is scanned with the one or more illumination light signals.
14. A computer system according to claim 13 wherein the system further comprise the processor is further
configured to perform an executable operation comprising:
modulating the intensity of the light source in accordance with a modulation frequency, preferably the
modulation frequency being selected between 1 and 200 MHz; the modulation of the illumination light signal causing a
modulation of the intensity of the first and second emission light signal;
the imaging system simultaneously capturing first image data associated with the modulated first emission light signal and second image data associated with a time-shifted version of the modulated first emission light signal; and, determining a life-time associated with at least one of the one or more optical excitations on the basis of at least part of the first image data and at least part of the second image data.
15. A computer program or suite of computer programs comprising at least one software code portion or a computer program product storing at least one software code portion, the software code portion, when run on a computer system, being configured for executing the method according to one or more of the claims 1-8.
EP18700268.8A 2017-01-05 2018-01-05 Forming images of a sample using scanning microscopy Withdrawn EP3566089A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
NL2018134A NL2018134B1 (en) 2017-01-05 2017-01-05 Forming images of a sample using scanning microscopy
PCT/EP2018/050280 WO2018127566A1 (en) 2017-01-05 2018-01-05 Forming images of a sample using scanning microscopy

Publications (1)

Publication Number Publication Date
EP3566089A1 true EP3566089A1 (en) 2019-11-13

Family

ID=58455598

Family Applications (1)

Application Number Title Priority Date Filing Date
EP18700268.8A Withdrawn EP3566089A1 (en) 2017-01-05 2018-01-05 Forming images of a sample using scanning microscopy

Country Status (3)

Country Link
EP (1) EP3566089A1 (en)
NL (1) NL2018134B1 (en)
WO (1) WO2018127566A1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109188667B (en) * 2018-08-10 2021-03-12 国家纳米科学中心 Multi-beam array multi-photon heavy scanning microscopic imaging device
NL2028507B1 (en) 2021-06-22 2022-12-29 Confocal Nl B V Rescan optical system, microscope and method

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006086649A2 (en) * 2005-02-10 2006-08-17 California Institute Of Technology Fast spectral confocal imager
DE102005042672B4 (en) * 2005-09-08 2009-10-01 Leica Microsystems Cms Gmbh Photosensor chip, laser microscope with photosensor chip and method for reading out a photosensor chip
DE102012023024B4 (en) * 2012-11-07 2023-05-04 Carl Zeiss Microscopy Gmbh Light microscope and microscopy methods
WO2015164843A1 (en) * 2014-04-24 2015-10-29 Vutara, Inc. Galvo scanning mirror for super-resolution microscopy
WO2016049010A1 (en) * 2014-09-24 2016-03-31 The United States Of America, As Represented By The Secretary, Department Of Health And Human Services Resolution enhancement for line scanning excitation microscopy systems and methods

Also Published As

Publication number Publication date
WO2018127566A1 (en) 2018-07-12
NL2018134B1 (en) 2018-07-25

Similar Documents

Publication Publication Date Title
JP4723806B2 (en) Confocal microscope
JP5897563B2 (en) System for synchronization in line scanning microscope
US8531762B2 (en) Confocal microscope apparatus
US8094304B2 (en) Two-photon microscope with spectral resolution
JP5337483B2 (en) Luminescence microscopy with increased resolution
US9804376B2 (en) Scanner with increased dynamic range
US7619732B2 (en) Method and microscope for high spatial resolution examination of samples
JP2016212154A (en) Scan type microscope system
EP3566089A1 (en) Forming images of a sample using scanning microscopy
KR101899711B1 (en) Confocal Image Display Apparatus using Chromatic Aberration Lens
JP4963544B2 (en) Minute height measuring device
JP5609729B2 (en) Microscope device, observation method and sample mounting mechanism
JP2008051576A (en) Shape-measuring apparatus and shape-measuring method
JP2015007661A (en) Scanning optical microscope
NL2018386B1 (en) Structured illumination scanning microscopy
JP5086197B2 (en) Confocal microscope
JP2007212305A (en) Minute height measuring instrument and displacement meter unit
US7274506B2 (en) Adjustment of a microscope
JP2016524192A (en) Equipment for confocal observation of samples
WO2020087998A1 (en) Lattice light sheet microscope and method for tiling lattice light sheet in lattice light sheet microscope
JP6396137B2 (en) Microscope illumination device, microscope illumination method, and microscope
US11473975B2 (en) Low-cost, compact chromatic confocal module
JP2004177732A (en) Optical measuring device
JP6770616B2 (en) Image acquisition device, image acquisition method, and spatial optical modulation unit
JP2009025632A (en) Scanning microscope

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20190726

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20220530

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20221210