WO2018122560A1 - Photon sensor apparatus - Google Patents
Photon sensor apparatus Download PDFInfo
- Publication number
- WO2018122560A1 WO2018122560A1 PCT/GB2017/053900 GB2017053900W WO2018122560A1 WO 2018122560 A1 WO2018122560 A1 WO 2018122560A1 GB 2017053900 W GB2017053900 W GB 2017053900W WO 2018122560 A1 WO2018122560 A1 WO 2018122560A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- photon
- pixel
- detection
- time
- sensor apparatus
- Prior art date
Links
- 238000001514 detection method Methods 0.000 claims abstract description 174
- 238000012545 processing Methods 0.000 claims abstract description 93
- 230000015654 memory Effects 0.000 claims abstract description 90
- 238000000034 method Methods 0.000 claims abstract description 38
- 230000008569 process Effects 0.000 claims abstract description 34
- 230000004044 response Effects 0.000 claims abstract description 21
- 238000004891 communication Methods 0.000 claims abstract description 18
- 230000005540 biological transmission Effects 0.000 claims abstract description 12
- 238000003860 storage Methods 0.000 claims abstract description 12
- 238000009826 distribution Methods 0.000 claims description 12
- 230000005855 radiation Effects 0.000 claims description 9
- 238000004611 spectroscopical analysis Methods 0.000 claims description 6
- 230000001360 synchronised effect Effects 0.000 claims description 5
- 238000001069 Raman spectroscopy Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 claims description 3
- 238000001161 time-correlated single photon counting Methods 0.000 description 22
- 230000000875 corresponding effect Effects 0.000 description 17
- 238000010586 diagram Methods 0.000 description 15
- 238000003384 imaging method Methods 0.000 description 8
- 239000000835 fiber Substances 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 230000001276 controlling effect Effects 0.000 description 4
- 230000003111 delayed effect Effects 0.000 description 4
- 238000003491 array Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000005192 partition Methods 0.000 description 3
- 238000001228 spectrum Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 102100034112 Alkyldihydroxyacetonephosphate synthase, peroxisomal Human genes 0.000 description 1
- 101710082751 Carboxypeptidase S1 homolog A Proteins 0.000 description 1
- 101710082754 Carboxypeptidase S1 homolog B Proteins 0.000 description 1
- 102100023804 Coagulation factor VII Human genes 0.000 description 1
- 101000799143 Homo sapiens Alkyldihydroxyacetonephosphate synthase, peroxisomal Proteins 0.000 description 1
- 206010028980 Neoplasm Diseases 0.000 description 1
- 238000001530 Raman microscopy Methods 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000000848 angular dependent Auger electron spectroscopy Methods 0.000 description 1
- 239000011324 bead Substances 0.000 description 1
- 201000011510 cancer Diseases 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 239000000975 dye Substances 0.000 description 1
- 238000001839 endoscopy Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- GNBHRKFJIUUOQI-UHFFFAOYSA-N fluorescein Chemical compound O1C(=O)C2=CC=CC=C2C21C1=CC=C(O)C=C1OC1=CC(O)=CC=C21 GNBHRKFJIUUOQI-UHFFFAOYSA-N 0.000 description 1
- 238000000799 fluorescence microscopy Methods 0.000 description 1
- 238000002866 fluorescence resonance energy transfer Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000006187 pill Substances 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000010791 quenching Methods 0.000 description 1
- 230000000171 quenching effect Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000004416 surface enhanced Raman spectroscopy Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/483—Details of pulse systems
- G01S7/486—Receivers
- G01S7/487—Extracting wanted echo signals, e.g. pulse detection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/483—Details of pulse systems
- G01S7/486—Receivers
- G01S7/4865—Time delay measurement, e.g. time-of-flight measurement, time of arrival measurement or determining the exact position of a peak
- G01S7/4866—Time delay measurement, e.g. time-of-flight measurement, time of arrival measurement or determining the exact position of a peak by fitting a model or function to the received signal
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/44—Raman spectrometry; Scattering spectrometry ; Fluorescence spectrometry
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
- G01S17/06—Systems determining position data of a target
- G01S17/08—Systems determining position data of a target for measuring distance only
- G01S17/10—Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/4808—Evaluating distance, position or velocity data
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/481—Constructional features, e.g. arrangements of optical elements
- G01S7/4816—Constructional features, e.g. arrangements of optical elements of receivers alone
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/483—Details of pulse systems
- G01S7/486—Receivers
- G01S7/4861—Circuits for detection, sampling, integration or read-out
- G01S7/4863—Detector arrays, e.g. charge-transfer gates
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/483—Details of pulse systems
- G01S7/486—Receivers
- G01S7/4865—Time delay measurement, e.g. time-of-flight measurement, time of arrival measurement or determining the exact position of a peak
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/70—SSIS architectures; Circuits associated therewith
- H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
- H04N25/77—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components
- H04N25/772—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components comprising A/D, V/T, V/F, I/T or I/F converters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/70—SSIS architectures; Circuits associated therewith
- H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
- H04N25/77—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components
- H04N25/772—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components comprising A/D, V/T, V/F, I/T or I/F converters
- H04N25/773—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components comprising A/D, V/T, V/F, I/T or I/F converters comprising photon counting circuits, e.g. single photon detection [SPD] or single photon avalanche diodes [SPAD]
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
- G01S17/06—Systems determining position data of a target
- G01S17/08—Systems determining position data of a target for measuring distance only
- G01S17/10—Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves
- G01S17/18—Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves wherein range gates are used
Definitions
- the present invention relates to a photon sensor apparatus, for example an apparatus suitable for asynchronous detection and measurement of sparse photonic events.
- CMOS complementary metal-oxide semiconductors
- SPAD single photon avalanche diode
- a basic single SPAD receiver produces an electronic output when struck by a single photon.
- These individual receivers are typically arranged in an array together with additional circuitry to form a pixel device, which may be referred to as a pixel for brevity.
- the pixel devices may be arranged in a linear chain to form a line sensor, or in 2-D arrays to form an imaging sensor.
- TCSPC time correlated single photon counting
- the maximum photon rate per pixel is limited to R/(L x P) Hz.
- the I/O rate of a chip is normally limited by the number of pads, data rate of those pads and power consumption. Rates R in the range of Gb/s are typical of high resolution CMOS image sensors. This leads to maximum photon rates of around 1 MHz for pixel arrays of around 1000 pixels, two orders of magnitude below the available peak capacity dictated by those other factors. For imagers with 10-100k pixels this situation is proportionately more constraining.
- a sensor apparatus for photon sensing comprising: a plurality of pixel devices, each pixel device comprising: a plurality of photon detectors arranged to produce detection signals in response to photon detection events.
- the apparatus may comprise a processing resource, for example a processing resource dedicated to that pixel or shared with other pixels, configured to process the detection signals to produce photon detection event signals, wherein each photon detection event signal comprises time data representative of a detection time at which a respective photon detection event occurred.
- the apparatus may comprise a further processing resource configured to process the photon detection event signals to obtain detection data representative of photon detection events over a detection period.
- the apparatus may comprise a pixel memory.
- the apparatus may comprise a communication resource for transmitting the detection data from the pixel device.
- the processing of the photon detection event signals may be such that the storing and/or transmission of the detection data uses less storage capacity and/or communication capacity than would be used by storage and/or transmission of the photon detection event signals.
- the pixel memory may be configured to store the detection event signals and/or the detection data.
- the processing resource is a shared processing resource, for a plurality of the pixels there may be a single shared processing resource (e.g. a TDC) configured to process the detection signals to produce photon detection event signals.
- Each photon detector may comprise a single photon avalanche diode (SPAD).
- the processing of the photon detection event signals may comprise a histogramming process and/or the detection data may comprise histogram data.
- the processing of the detection signals produced by the photon detectors may comprise a time to digital conversion process.
- the detection data may represent a distribution of photons detected at the sensor apparatus as a function of time.
- the processing of the photon detection event signals may comprise assigning the photon detection event signals between a plurality of time bins.
- the processing of the photon detection event signals may comprise obtaining count data representing a respective count value for each time bin of the plurality of time bins.
- collected and stored photon detection data may represent a time distribution of photons arriving at the sensor apparatus.
- Each time bin may have a width representing a time interval and the further processing resource may be configured to select or vary the width of at least one of the time bins.
- the further processing resource may be configured to select or vary the width of at least one of the time bins automatically or in response to user input, and/or based on at least one configuration parameter.
- the further processing resource may be configured to select the detection period in dependence on feature of interest for example a measured or expected position of a peak in a distribution of photon detection events.
- the photon detection event signals may be obtained during the detection period and the processing of the photon detection event signals may comprise selecting a portion of the detection period and excluding photon detection event signals from outside the selected portion of the detection period, for example when processing the detection signals to obtain the detection data.
- the processing resource may be configured to assign the detection event signals for the selected portion of the detection period across the time bins.
- the further processing resource may be configured to modify and/or to select a part of the time data, for example in order to select or vary the width of the time bins and/or in order to select a portion of the detection period.
- Each item of time data may comprise a series of bits and the modifying of the time data may comprise, for at least some of the items of timestamp data, excluding at least some bits of the series of bits.
- Each pixel device may be configured to reset the further processing resource and write signals from the further processing resource automatically.
- the selecting of at least part of the time data may comprise, for at least some of the items of time data, selecting some of the bits of the series of bits.
- the apparatus may further comprise a delay generator for modifying a clock signal provided to the initial processing resource, thereby to delay collection of photon detection data and/or to shift the detection period.
- the pixel memory may comprise one or more counters.
- the pixel memory may comprise tri-stated MUX components. Counters may be used to count the number of photon events detected for each bin. At the end of an exposure time, counter contents may be stored into histogram memory (in preparation of readout) and counters may be reset for the next exposure cycle.
- the pixel memory may have a size of at least 5 bytes or 10 bytes and less than 1 kilobytes. Preferably, the pixel memory is less than 50 bytes, for example 44 bytes.
- the pixel memory size may be selected in dependence on number of histogram bins and/or number of bits per bin, or vice versa. For example, there may be 32 bins and 1 1 -bits per bin, hence a total pixel histogram memory of 44 bytes.
- the communication resource of the pixel device may be configured to transmit the detection data from the pixel device to an additional processing resource remote from the pixel device and/or a further memory remote from the pixel device.
- the additional processing resource and/or the further memory may be configured to process and/or store detection data from each of the plurality of pixel devices.
- the sensor apparatus may be provided on a chip or circuit board and the additional processing resource and/or further memory may be outside the chip or circuit board, for example remote from the chip or circuit board.
- the initial and/or further processing resources and/or the memory of the pixel device may be located adjacent to or at least partially beneath the plurality of photon detectors.
- the plurality of pixel devices may be arranged in an array, and for at least some of the pixel devices the further and/or initial processing resources and/or the memory of the pixel device may be located between the photon detectors of the pixel device and the photon detectors of an adjacent pixel device.
- the photon detection events may occur in response to application of the laser radiation to a subject.
- the apparatus may be configured such that operation of the photon sensors and/or the processing of the photon detection event signals and/or the detection period is synchronised with operation of a laser that applies the laser radiation, for example pulsed laser radiation.
- the apparatus may be configured for use in, for example, a Lidar system, a Time-of- flight system, a fluorescent spectroscopy system or a Raman spectroscopy system.
- a sensing method comprising: obtaining photon detection event signals at a pixel device in response to photon detection events, wherein each photon detection event signal comprises time data representative of a detection time at which the photon detection event occurred; processing at the pixel device the photon detection event signals to obtain detection data representative of photon detection events over a detection period; transmitting the detection data from the pixel device, wherein the processing is such that the storing and/or transmission of the detection data uses less storage capacity and/or communication capacity than would be used by storage and/or transmission of the photon detection event signals.
- a sensing system comprising a laser configured to apply laser radiation to a subject, and a sensor apparatus as claimed or described herein that is configured to sense photons emitted by the subject in response to the laser radiation.
- Figure 1 is a schematic diagram of a photon sensor apparatus
- Figure 2 is a schematic diagram showing in overview the architecture of a photon sensor chip according to an embodiment
- Figure 3 is a schematic diagram of a pixel element of the photon sensor apparatus
- Figures 4(a) and 4(b) are timing diagrams for a subset of signals generated and received by the photon sensor apparatus
- Figure 5 are illustrative plots of histograms produced by the photon sensor apparatus
- Figure 6 is a circuit diagram of an 1 1 -bit histogram bin
- Figure 7 is a circuit diagram of a histogram decoder
- Figure 8 is a circuit diagram of a 32-bin histogram memory
- Figure 9 shows example lifetime decays captured using the photon sensor
- Figure 10 shows maximal count rate comparisons captured using a line sensor
- Figure 1 1 shows time resolved 3-D spectra based on (a) TCSPC and (b) Histogram modes captured using a line sensor;
- Figure 12 shows estimated lifetimes based on (a) TCSPC and (b) Histogram modes captured using a line sensor;
- Figure 13 shows count rate comparisons between TCSPC and Histogram modes captured using a line sensor
- FIG. 14 illustrates a zooming feature of Histogram mode.
- the photon sensor apparatus has a photon sensor 10.
- An optical instrument for example, an endoscope is provided.
- the optical instrument has a fibre bundle that comprises an imaging bundle, a capillary fibre and a fluorescent bead and/or Surface Enhanced Raman Spectroscopy (SERS) fibre.
- Two pulsed lasers are in optical communication with the fibre bundle via suitable focusing means and a microscope objective.
- the suitable focusing means may be a dichroic mirror.
- Pulsed laser light from the lasers is coupled into the multimode fibre and used to illuminate the sample.
- photon detection events occur. Detection events vary depending on application. For example, the detection events may arise from fluorescence scattering and/or Raman scattering.
- Signals are received by the fibre bundle and sent to the photon sensor apparatus via a mirror or grating. Data from the line sensor is then sent to a computer and displayed on the display.
- An example embodiment of the photon sensor 10 is shown in Figure 2.
- the photon sensor 10 has a pixel array including 1 x 512 pixels.
- the sensor architecture can provide a 1 M spectra per second read out and video rate photon counting/coarse time correlated single photon counting (TCSPC) for scanned operation.
- TCSPC time correlated single photon counting
- the sensor structure provides, amongst other operating modes on-chip and in-pixel real-time histogramming capability.
- the sensor structure also provides zooming and shifting capabilities to the in-pixel histogramming.
- An illustrative pixel device 42 also referred to as a pixel for brevity, of the pixel array is shown in Figure 2.
- the pixel 42 includes two sets of eight "red" single-photon avalanche diodes (SPADs) 44 and two sets of eight "blue” SPADs 45.
- red and blue SPADs are provided in the embodiment of Figure 2, in alternative embodiments any other suitable arrangement of different SPAD device specifications including but not limited to differently colour sensitive SPADs may be provided, or all SPADS may have an identical specification for example the same wavelength sensitivity profile and/or all SPADs may be any mix of SPAD specifications.
- the red SPADs 44 are sensitive to light in the range 600 nm to 900 nm.
- the blue SPADs 46 are sensitive to light in the range 450nm to 550nm.
- the pixel 42 also has a dedicated processing resource, for example a Time to Digital Convertor (TDC) 48, and a per-pixel memory resource 50 arranged in pixel.
- TDC Time to Digital Convertor
- the TDC provides photon detection event signals, for example in the form of time stamp data, digital timing signals or other representation of the time an event occurs.
- a further processing resource associated with the per-pixel memory resource 50 is provided per pixel to process photon detection event signals to obtain detection data.
- the further processing resource may be provided as part of the per-pixel memory resource or as a separate component.
- the initial processing resource, per-pixel memory resource 50 and further processing resource are configured to perform in accordance with an operating mode of the sensor.
- the sensor also has a global processing resource in the form of a control module 52 and a delay generator 54.
- the control module 52 may include configuration registers that store the values of various configuration parameters.
- the sensor 10 also includes a communications resource in the form, in this embodiment, of a sensor output bus that includes a serialiser 56, and a readout module 60.
- the sensor output bus enables transmission of detection and other data from the sensor 10 to an external processing resource (not pictured).
- the illustrative pixel device 42 has "red” and "blue” SPADS.
- the pixel device 42 is illustrative only.
- the pixel device may also be implemented using only SPADs with identical specifications.
- a moment of arrival of a single photon is registered as a change in state of the SPAD, for example from an un-activated state to an activated state.
- each SPAD has an intrinsic dead-time period that typically lasts 10s of nanoseconds. After the dead time period has expired the SPAD resets to its original, un-activated state.
- each pixel is 23.78 ⁇ and therefore the pixel array has a width, as indicated by 62, equal to approximately 12.175mm.
- the sensor 10 has a height indicated by 64, less than or equal to 2 mm.
- the sensor has a width 66 equal to approximately 13mm.
- the above dimensions are provided for illustrative purposes only and other dimensions and shapes are possible.
- the pixel array, and the individual pixels can be of any suitable shape and size. As shown in Figure 2, the pixel array may be a one dimensional array having a width of one pixel to form a line sensor for spectrometry or line scanning. Alternatively, the pixel array may be a two dimensional array having both a length and width greater than one pixel to form an image sensor.
- the sensor 10 has some of the memory and processing components placed on-chip, alongside the SPAD array, which may be used to ensure that only selected data is processed and/or transmitted and/or stored.
- the photon sensor 10 may operate in one of a variety of operating modes, the choice of which determines how a per-pixel memory resource 50 is used. Of particular interest is an on-chip histogramming mode, and operation in this mode is described with reference to Figures 3, 4 and 5.
- Other operating modes include single photon counting (SPC) mode (time gated) where the memory has two counters, counter A and counter B, both counters having, for example, 10-bit and 20-bit modes.
- Other operating modes also include TCSPC operating mode in, for example, 1 1 -bit or 17-bit modes.
- Figure 3 is a flow chart and schematic diagram illustrating a sequence of operations performed by circuitry components and processing logic of the pixel 42 of the sensor.
- Figure 3 shows the pixel SPAD array 46 for detecting light, the TDC 48, and the per- pixel memory 50 of the pixel 42.
- Figure 3 also shows the delay generator 54.
- a controlling resource 70 is also shown.
- the controlling resource 70 may be provided as part of the on-chip controller 54 or provided off-chip.
- the pixel also has pulse combining circuitry 72, a further processing resource, for example a decoder (shown in Figure 3 as part of the per-pixel memory 50) and a readout module 74.
- a single TDC, or at least some components of a TDC is shared by a plurality of the pixels 42.
- timestamps or other timing information for a plurality of the pixels 42 may be generated in such embodiments by a common TDC or by TDCs sharing at least some common components.
- Such sharing of TDCs may enable a reduced circuit area and power consumption, but may lead to more SPAD events being missed in some cases.
- the delay generator 54 is a global component that is coupled to all pixels. As described in more detail below, the delay generator 54 provides a global modified clock signal to the pixels, indicated by STOPd. In particular, the modified clock signal is provided to the TDC 48.
- the modified clock signal generally coincides with occurrence of fluorescence or other photon emission process from a subject that causes photon detection events to occur at the sensor apparatus.
- the components of the pixel 42 are arranged such that the time of arrival of a photon at the array of SPADs 46 is measured with reference to the modified clock signal provided by the delay generator 54.
- the processing logic of the pixel 42 can provide measurements of time of arrival of a plurality of photons, which may for example be referred to as photon detection event signals, and to sort and/or otherwise process the measurements to produce and store a timing distribution, for example a histogram, across the per-pixel memory resource 50, which may for example be referred to as detection data.
- the pixel 42 can provide a readout signal, which may comprise or represent detection data, from the sensor output bus that represents the time distribution of detected photons at the SPAD array over a detection period.
- the time distributions are provided per-pixel and therefore, the sensor 10 provides time distributions for every pixel.
- each SPAD of the pixel SPAD array 46 is coupled to the pulse combiner 72 via a plurality of SPAD pulse channels, wherein each SPAD has a respective SPAD pulse channel.
- Each SPAD has corresponding processing logic (not shown) that includes a front end, a digitizer, a digital buffer, and quenching circuitry, which may be either active or passive.
- the digital buffer acts to isolate SPAD capacitance from being loaded by subsequent circuit and to transmit pulse signals to the pulse combining circuitry 72.
- the pulse combiner circuitry 72 which forms part of the pixel processing logic in some embodiments, couples the SPAD array 46 to the TDC 48.
- the pulse combiner circuit acts as a compressor and combines the SPAD outputs into a single combined channel.
- the single channel provides one short pulse per photon count.
- the relatively long SPAD pulses (tens of ns) are compressed in time by the pulse combiner circuitry 72, thereby removing any dead time of SPAD pulses.
- the relatively long SPAD pulses (tens of ns) are first compressed in time by a monostable and then ORed together through an OR tree (spatial compression).
- the SPAD pulses from the red SPADs are combined separately from the SPAD pulses from the blue SPADs.
- either the blue or the red pulse stream is selected for the next processing step.
- the pulse combiner 72 provides an efficient means of reducing hardware requirements of other stages of the sensor, for example, TDC 48 and per-pixel memory resource 50. Some spatial information relating to the precise origin of the pulse signal is lost by combining the SPAD channels. For example, on examining a combined pulse signal it may not be possible to determine from which individual SPAD a pulse has originated.
- the TDC 48 which forms part of the pixel processing logic, is configured to receive the single channel input from the pulse combiner circuitry 72 and the delayed clock signal from the delay generator 54, which acts as a global clock signal for each of the pixels of the sensor.
- the TDC 48 is configured to process these inputs and produce a photon detection event signal comprising time data representative of a detection time at which a photon detection event occurred.
- the system can be configured to operate in different modes, for example TCSPC mode and histogramming mode.
- TCSPC mode and histogramming mode.
- the TDC and other components can operate differently in TCSPC and histogramming mode. Further description of operation in TCSPC and histogramming modes is provided below in relation to Figure 4.
- the TDC 48 is configured to process these inputs and produce a photon detection event signal comprising time data representative of a detection time at which a photon detection event occurred for each pulse of the combined pulse signal.
- the TDC can be a gated ring oscillator TDC.
- the decoder may form a separate component of the pixel or form part of the per- pixel memory resource 50.
- the decoder may form part of the TDC 48.
- the decoder provides an interface between the TDC 48 and the memory resource 50.
- the decoder is configured to produce decoded signals based, at least in part, on the photon detection event signals generated by the TDC 48 and at least one control parameter.
- the control parameter may be a zooming parameter that indicates, or that can be used to determine the resolution of, a histogram. The operation of the decoder is described in further detail below.
- Each per-pixel memory resource 50 is partitioned into a number of memory portions, each memory portion acting as a time bin, for example for use in a histogramming process.
- the memory partitions may be counters.
- the decoder acts to provide decoded signals to a corresponding memory portion and the corresponding memory portion counts the received decoded signals.
- the per-pixel memory resource therefore produces and stores count data, for example count values, that represents a time distribution of photons, in the form of a histogram. Through for example the zooming parameter, or using any other suitable process, the decoder can set the histogram time resolution.
- the per-pixel memory resource 50 can be implemented in a number of alternative ways, including an array of ripple counters.
- the decoder may be implemented as a chain of logic gates, which selects one histogram bin to be incremented by one count.
- the pixel also has a pixel readout module 74 configured to receive a read signal from either another component of the sensor or externally from the sensor. In response to receiving the read signal, the readout module 74 is configured to produce a pixel output signal 78 that represents the state of the memory resource. The pixel readout module 74 provides pixel output to the sensor output bus.
- the delay generator 54 is configured to receive a delay parameter.
- the delay parameter is generated by the controlling resource 70 based either on a user input value or the pixel output 78.
- the controlling resource 70 can provide a feedback loop between the pixel 42 and the delay generator 54.
- the delay generator 54 is configured to receive a STOP signal synchronised with pulses of the laser and to adjust the STOP signal using the delay parameter to produce a delayed STOP signal.
- the STOPd signal is a copy of STOP signal delayed by a timing of D x Tdel, where D is the delay parameter and Tdel is a basic delay resolution of a Delay Locked Loop (DLL) or programmable delay chain.
- DLL Delay Locked Loop
- a photon emission event occurs and a photon is incident on a SPAD of the pixel SPAD array 46 causing the pixel SPAD array 46 to produce a SPAD pulse.
- the SPAD pulse is transmitted from the SPAD to the pulse combiner 72 via its processing logic.
- the pulse combining circuitry 72 provides a combined pulse signal to the TDC 48.
- the TDC 48 generates photon detection event signals, in the form of timestamps, for each pulse of the combined pulse signal. Timestamps are produced for substantially each pulse of the combined pulse signal that is received by the TDC 48. It is noted that the time it takes for the TDC to process a detected SPAD event (e.g. generate a time stamp) may, in some cases, lead to missed pulses. For example, if generated SPAD pulses are too close to each other the TDC may not be able to process the next SPAD pulses being finishing processing the current SPAD pulse.
- the decoder decodes the timestamps to produce a decoded signal and distributes the decoded signal to the correct portion of the per-pixel memory resource 50, e.g. representing a particular time bin, thereby increasing a count of the respective memory portion. Following a burst of photons incident on the pixel SPAD array, a count distribution is stored over the portions of the per-pixel memory 50.
- the readout module 74 transmits the readout signal 78 representing the state of the per-pixel memory 50. The transmitted signal includes the count distribution for the pixel.
- the transmitted pixel signal is received by the sensor output bus together with pixel signals from other pixels of the sensor. Together, the pixel signals can be further processed to produce a sensor array histogram. As example, sensor array histogram is shown in Figure 5. Photons can be counted by summing histograms bins off-chip.
- the TDC, decoder and memory may form part of a pixel processing logic circuit in communication with the delay generator that provides a STOP (or STOPd) signal.
- the TDC includes a gated ring oscillator, a ripple counter, and a shift register and the decoder includes a logic chain, for example as shown in Figure 7.
- the memory resource in this embodiment is provided by an array of ripple counters implementing the in-pixel histogram, see for example Figures 6 and 8.
- GRO gated ring oscillator
- the STOP pulse samples the state of the shift register to generate a thermometer code which is decoded by the XOR chain to increment a single bin of the array of ripple counters implementing the in- pixel histogram.
- the STOP signal samples how long the shift register has been shifting along a 1 which then directly activates a corresponding histogram bin.
- the shift register may be operated from a divided version of the GRO output.
- a decoder is not required if the other components of the pixel processing logic are arranged such that a single 1 can be shifted along.
- TCSPC is a favoured technique for low light time-resolved imaging due to its photon efficiency and specificity.
- Figure 4(b) shows a timing diagram for operation of the system in histogramming mode. The operation of the TDC, memory resource and decoder is dependent on the operating mode of the sensor.
- a photon detection event signal for example a timestamp, is generated by the TDC only for the first photon captured within an exposure time window.
- a photon detection event signal is generated for each pulse of the combined SPAD pulses.
- the first and uppermost waveform is that produced by a pulsed laser.
- the pulsed laser generates a stimulus for example, for a time of flight system or a fluorescence/Raman microscopy or spectroscopy system.
- a laser capable of generating short pulses typically at MHz rates may be employed to excite a sample or to generate an echo within a pulsed LIDAR system.
- a second waveform, the STOP clock is synchronised to the laser pulse.
- the STOP clock is usually derived from a SYNC output from the laser.
- the STOP clock signal provides a reference signal against which arrival of photons can be measured.
- the STOP clock signal is provided to the delay generator 54, and may be delayed by a set time as described elsewhere to provide the third waveform STOPd.
- the fourth waveform shows SPAD pulses.
- the fifth waveform is a variable exposure window (WIN) signal.
- the sixth and seventh waveforms are TDC waveforms, corresponding to a TDC counter (TDC_counter) and TDC output (TDC_OUT) waveform.
- the eighth waveform is a pixel write waveform (PIX_WRITE).
- the ninth waveform is a pixel reset waveform (PIX_RESETn).
- Pixel reset signal is generated externally (through an FPGA board) together with some other control signals, for example, PIX_WRITE.
- the time interval between the pixel reset signals define the exposure time (however, only SPAD events which are within the active window (i.e. WIN in Figure 4(a)) are considered in each exposure cycle).
- the tenth waveform is an AUTO_SEQ signal.
- the eleventh and twelfth waveforms correspond to a TDC write signal (TDC_WRITE) and a TDC reset signal (TDC_RESETn).
- the thirteenth waveform (HIST_EN) corresponds to parameter Sel, which is a control register bit that enables histogramming mode in the memory resource.
- a histogram mode parameter (HIST_MODE) control signal is the fourteenth waveform. This control signal is described in further detail with reference to Tables 1 , 2 and 3.
- the fifteenth, sixteenth and seventeenth waveforms are a histogram readout waveform (HIST_READOUT), a TDC readout (TDC_READOUT) and a pixel output signal (PIX_OUT).
- TCSPC mode Differences between TCSPC mode and histogramming mode are evident on a comparison of Figure 4(a) and 4(b). For example, in TCSPC mode a timestamp is generated only for the first photon captured within the exposure time. Furthermore, in TCSPC mode Histogram memory is by-passed and a generated timestamp is readout directly.
- the TDC can convert many more SPADs within the exposure time and each converted SPAD is used for building the histogram (by incrementing one histogram BIN).
- TDC_WRITE and TDC_RESETn control signals are generated automatically on-chip and independently for each pixel (separate from global PIX_WRITE and PIX_RESETn signals) by setting AUTO_SEQ control signal to HIGH, as shown in Figure 4(b).
- multiple photon events are converted during a single exposure cycle by these TDC reset and TDC write signals.
- TCSPC mode only one photon event is converted per exposure cycle and all TDCs (of all pixels) are reset by the global pixel reset signal (PIX_RESTn).
- a photon is detected thus producing a detection signal in the form of a SPAD pulse (as seen in the fourth waveform).
- the SPAD pulse triggers the TDC counter signal, and the subsequent STOPd rising edge stops the TDC counter.
- TDC write and TDC reset control signals are automatically generated on-chip and independently for each pixel. These two TDC control signals are generated independently from the global pixel write and pixel reset signals.
- the TDC In response to receiving the TDC write control signal, the TDC writes the TDC value to memory.
- the TDC reset control signal the TDC counter is reset to zero.
- a global pixel write control signal (PIX_WRITE) is sent to all pixels.
- the histogram memory may be reset by the global reset signals. However, in histogram mode, after detecting a photon event (e.g. generating a timestamp) the TDC data is sent to the histogram module for further processing and TDC counters are reset automatically (see for example, control signals AUTO_SEQ, TDC_WRITE and TDC_RESETn in Figure 4(b)).
- a photon event e.g. generating a timestamp
- TDC counters are reset automatically (see for example, control signals AUTO_SEQ, TDC_WRITE and TDC_RESETn in Figure 4(b)).
- the timing signals transmitted from the TDC 48 comprise a large number of timestamps (typically between 1 k and 1 M) as, in histogramming mode, a timestamp is produced for substantially every photon received by the TDC during the detection period.
- Each timestamp has a size that is determined by the TDC 48.
- a TDC may be configured to output a timestamp that is an L-bit word.
- An L-bit timestamp can store information corresponding to 2 L time intervals, in this way each TDC has an in-built maximum time resolution.
- the memory resource 50 has a defined number of partitions or counters, each counter having a predetermined size. Each counter is a suitable size to store count information. For example, a counter having an M-bit size can count up to 2 M occurrences of an event.
- per-pixel memory resource 50 is provided with a reduced number of memory portions, where each memory portion corresponds to a respective time bin. There are N time bins in total, where N ⁇ 2 L .
- N the number of bins
- the per- pixel memory area overhead required is reduced and the memory resource 50 is placed within the pixel 42.
- Memory placed within a pixel can achieve 100s MHz accumulation of timestamps due to the proximity of memory to the TDC 48.
- histogram zooming is implemented by selecting different bits of the TDC output.
- the per-pixel memory resource is sized M x N bit, where N is the number of time bins for a histogram and M is the size of each counter for each bin.
- the pixel decoder receives L-bit timestamps, selects a portion of L-bit timestamps and determines to which bin the selected portions of L-bit timestamps should be allocated.
- the pixel decoder receives input signals containing configuration parameters K and Sel.
- Parameter K may be considered as a zooming parameter and its value determines the size of the time bins.
- Sel is a control register bit that enables the histogramming mode in the memory resource.
- the input signals K and Sel are indicated as HIST_MODE and HIST_EN respectively in Figure 4(a) and Figure 4(b).
- the decoder selects a first portion of the L-bit word determined by the values of K and N, in particular, the decoder selects the portion of the L-bit word corresponding to bits K to K + log 2 N - 1 . This selection may be considered to effectively exclude at least some of the bits when performing the histogramming.
- This decoded signal therefore has a size equal to log 2 K.
- the time interval of each histogram bin is scaled by a factor dependent on zooming parameter K.
- K can be chosen to be any value from 0 to L - log 2 V. Selection of zooming parameter K increases the bin time width from the TDC resolution bin width by a factor of 2 K .
- on-chip histogram option is shown in Figure 5 and in Tables 1 , 2 and 3.
- K can be any value from 0 to 7, for example set automatically or based upon user input.
- the selection or varying of the value of K can effectively select or vary the time width of the bins, and thus effectively select or vary a level of histogram zooming.
- Options 1 to 8 are shown in Table 1 and these options correspond to values of K from 7 to 0.
- Histograms produced under options 1 , 2 and 3 are shown in Figure 5.
- the number of time bins, N 32.
- a 5-bit portion of the 12-bit TDC timestamp is selected.
- the selected 5-bit portion represents one of the 32 possible time intervals represented by the time bins and therefore determines which bin the photon should be allocated to.
- Figure 5(a) shows a laser sync signal corresponding to the STOP signal received by the TDC.
- Figure 5(b) shows an actual response curve corresponding to a fluorescence or other photon emission process measured with photon counts over time. The response curve has a first sloping segment up to a peak and a decaying segment after the peak.
- the histograms shown in Figures 5(c) to 5(e) are produced by the system represent the shape of the response curve at consecutively finer resolutions.
- Selecting option 1 corresponds to selecting K to be the largest possible value, 7.
- a plot of a resulting histogram is shown in Figure 5(c).
- This choice causes the decoder to select a 5-bit portion of the 12-bit timestamp output by the TDC, the portion corresponding to bits 7 to 1 1 of the 12-bit timestamp.
- the decoder then assigns the 5- bit portion to the histogram memory (i.e. histogram bins).
- TDC codes 128 to 255 are binned together in the second bin and the second bin counter increment by one. In this way all 32 bins counters are updated.
- the total time range of the histogram in option 1 is 204.8 ns and the bin width is 6.4ns.
- Selecting option 2 corresponds to selecting K to be the second largest possible value, 6.
- a plot of a resulting histogram is shown in Figure 5(d).
- This choice causes the decoder to select a 5-bit portion of the 12-bit timestamp output by the TDC, the portion corresponding to bits 6 to 10 of the 12-bit timestamp.
- the decoder then assigns the 5- bit portion to the histogram memory.
- TDC codes 64 to 127 are binned together in the second bin and the second bin counter incremented by one. In this way all 32 bins are provided with entries. Information held by bit 1 1 is not used. Losing the highest bit, bit 1 1 , reduces the value of the total TDC range by half, compared to option 1 . Hence the total time range of the histogram in option 2 is 102.4 ns and the bin width is 3.2ns.
- Selecting option 3 corresponds to selecting K to be the third largest possible value, 5.
- a plot of a resulting histogram is shown in Figure 5(e).
- TDC codes 32 to 63 are binned together in the second bin and the second bin counter incremented by one. In this way all 32 bins are provided with entries. Information held by bits 10 and 1 1 is not used. Losing the highest 2 bits, bits 10 and 1 1 , reduces the value of the total TDC range by half, compared to option 2. Hence the total time range of the histogram in option 3 is 51.2 ns and the bin width is 1 .6 ns. Further choices of K are shown in Table 1 , 2 and 3. Selecting option 8 corresponds to selecting K to be the smallest possible value, 0.
- This choice causes the decoder to select a 5-bit portion of the 12-bit timestamp output by the TDC, the portion corresponding to bits 0 to 4 of the 12-bit timestamp. The decoder then assigns the 5-bit portion to the histogram memory. Therefore, timing information held by bits 5 to 1 1 is not stored.
- the bin width is 0.05ns and the total TDC range is 1 .6ns. The resolution in this option corresponds to the inherent resolution of the TDC. In other words, the time bin width is equal to the time resolution of the TDC.
- Table 1 Options provided for adjusting TDC range and histogram bin width
- Table 2 Further parameters for modes presented in Table 1 .
- HIST MODE ⁇ 2:0> corresponds to value selected for K.
- Table 3 Further parameters for modes presented in Table 1 .
- Table 3 expresses the same TDC range of Tables 1 and 2 as the available number of TDC codes, and histogram bin width is expressed as the number of TDC codes.
- Tables 1 , 2 and 3 histogram bin sizes are described with and without bin-chaining. Bin-chaining allows two M-bit bins to be used as a single 2M— bit bin. As a consequence the bin size is doubled and the number of bins is halved.
- a circuit diagram of an example implementation of an 1 1 -bit histogram bin is shown in Figure 6. Each bin is implanted as an 1 1 -bit ripple adder, and a MUX-based tri-stated memory units.
- a circuit diagram of an example implementation of a 32 bin histogram is shown in Figure 7.
- each BIN is implemented as in Figure 6.
- bin chaining mode two 1 1 -bit adders are simply chained together (the most significant bit of one adder is connected to the least significant bit of the other adder), forming a 22-bit adder (effectively 20-bit since the remaining 2 bits are overflow bits).
- FIG. 7 A circuit diagram of an example implementation of a histogram decoder which is an example of a further processing resource is shown in Figure 7.
- the histogram decoder takes as input: a timestamp, from the TDC (TDC_out), a histogram mode (HIST_MODE) control signal and a histogramming mode enable (HIST_EN) control signal.
- the histogram mode control signal indicates the value of K, as described with reference to Tables 1 , 2 and 3, is used.
- Histogram decoder checks if TDC timestamp is within the histogram window using value of K. A 5-bit TDC data chunk is then selected from the TDC output using the value of K, as described with reference to Tables 1 , 2 and 3. The 5 bit selected TDC data chunk is then converted by a 5 to 32 decoder and this output is then provided to the histogramming memory to increment the corresponding histogram bin counter.
- FIG 8 A circuit diagram of 32 histogram BIN memory is shown in Figure 8.
- Figure 8 also shows two single photon counters, SPCA and SPCB, included, for operation in single photon counting mode.
- the histogram decoder as described with reference to Figure 7, outputs histogram bin information derived from TDC timestamp which increments the bin counters. Allocation may be performed sequentially.
- the delay signal also provides a delay offset of histogram prior to zooming. The delay offset shifts a feature of interest, for example the peak and exponential decay so that this feature occupies the first bins of the histogram when these bins are expanded by a factor 2 K .
- the parameter D shifts the sharp feature of a histogram corresponding to the laser pulse towards the first few bins of the histogram to allow effective zooming. This can be done automatically, or by a user of the system. Through the control resource 70, the system monitors and adjusts histogram peak position by using coarse bin sizes and progressively reducing K whilst adjusting the peak by precise choices of D. Zooming to features of an optical waveform around a sharp feature created by a pulsed laser is important for time of flight ranging systems as it gives a more precise distance estimation. Zooming and delay is also important for Raman spectrometry as it provides suppression of fluorescence component. The use of a delay signal as outlined in the preceding paragraph can effectively be used to select a portion of a detection period (e.g. the portion of the detection period starting at or just before the feature of interest, such as the peak and subsequent decay) and exclude detection signals from before and/or after the selected portion of the detection period (e.g. detection signals, if any, obtained at times significantly earlier than the peak).
- a thresholding process may be used to determine a time for which the number of counts reaches or exceeds a threshold level. The portion of a detection period before that point in time may then be excluded from use in a histogramming or other process, as it may be considered not to include sufficient useful data.
- a thresholding process may be used to determine a time at which the number of counts has fallen below a further threshold level after the peak. Data after that point in time may for example then be excluded, effectively cutting off the end of a decay curve.
- the detection period, or the portion of the detection period, to be used may be selected based on expected time period when photon detection events may be expected. For example, following application of a laser pulse to a subject there may be an expected dead time before photon detection events may be expected to begin, and there may be an expected length for a subsequent decay process during which the number of photon detection events falls from a peak to zero or to a threshold level.
- the sensor apparatus may be synchronised with operation of the laser such that the detection period is selected so that it matches a period for which useful or desired data, for example detection event signals, may be expected to be obtained.
- Figure 9 shows an example lifetime decay captured in on-chip per-pixel histogramming using the one dimensional sensor described with reference to Figure 2.
- the upper plot is a 3 dimensional plot showing a pixel array histogram.
- the x and y axes display pixel number and bin number.
- the bin number runs from 0 to 31 .
- the pixel number runs from 0 to 51 1.
- the z-axis displays photon counts. The plot therefore displays, for each pixel, photon counts for each time bin.
- the lower plot shows a corresponding heat map.
- the x-axis displays bin number and the y-axis displays pixel number.
- the colour of each pixel-bin corresponds to the height shown in the upper plot which is the photon count.
- the per- pixel processing resource and/or memory of the pixels may be located adjacent to or at least partially beneath the plurality of photon detectors, for example formed beneath the SPADs in a layered structure.
- the pixels may be arranged in an array, and for at least some of the pixels the processing resource and/or the memory of the pixel device is located between the SPADs or other photon detectors of the pixel and the SPADs or other photon detectors of an adjacent pixel.
- the sensor 10 of the embodiment of Figure 2 may be provided on a single chip or circuit board, or on a plurality of chips or circuit boards, and the communications resources of the pixel devices may be configured to transmit detection data off the chip(s) or circuit board(s), for example to a further memory, processor or other processing resource that is external to the chip(s) or circuit board(s).
- the detection data from each of the pixel devices may be transmitted to a PC or other computer, or to a server, or to dedicated processing and storage hardware for further processing and storage.
- the chip(s) or circuit board(s) may include a further communication resource, and the detection data from the communications resources of the pixel devices may be transmitted to the further memory and/or further processing resource via the further communications resource of the chip(s) or circuit board(s). The detection data may be received and re-transmitted by the further communications resource.
- the chip(s) and circuit board(s) in some embodiments may include an additional memory for storing detection or other data and/or an additional processing resource that may perform further processing of the detection data before transmission off-chip or off-circuit board.
- the additional processing resource may, for example, process the detection data to extract features of interest.
- the features of interest may comprise features of interest identified in the histogram data, for example a position or height of a peak, or one or more fitting parameters that may be used to fit a decay curve to the histogram data.
- the additional processing resource is not limited to being provided on the same chip or circuit board as the detectors and in some embodiments the additional processing resource may be provided off-chip or off-circuit board.
- a line sensor according to embodiments was tested under controlled LED illumination triggered at 100MHz to measure the maximum count rate in all modes. As shown in Figure 10, per-pixel histogramming achieves -1/4 SPC count rate demonstrating the ability to maximise TCSPC event timing. Fluorescein-Rhodamine FRET was investigated in order to demonstrate the time-resolving spectral capabilities of the sensor.
- the line sensor was placed in a spectrograph and captured fluorescence from a sample illuminated by a 483nm pulsed laser diode (20MHz repetition rate).
- Figure 1 1 shows a time-resolved spectrum of the mixture obtained in TCSPC mode ( Figure 1 1 (a)) and the on chip histogramming mode ( Figure 1 1 (b)).
- the senor may only consume power when a photon hits.
- Another advantage that may be provided by embodiments is that the sensor also reduces large data volume required to be output by SPAD based time-resolved image sensors.
- the sensor has a low power requirement scaling with number of photons incident sensor.
- a further advantage is that the sensor can be efficiently implemented in a stacked 3D CMOS with an advanced lower tier process.
- the per-pixel memory resource 50 may be part of a larger memory resource.
- a memory resource may be provided for all pixels, but partitioned into independent pixel partitions for each pixel.
- time bins are shown as equal in width
- a varying time mode can be implemented to provide logarithmically sized time bins for the histogram, for example so that each time bin width has a size T, 2T, 4T, 8T etc. This can equalise photon counts in each bin for a response curve that is exponentially decreasing waveform as found in fluorescence lifetimes.
- Further variants of the sensor may include to eliminate the need for a thermometer decoder and to reduce power by operating a TDC at a supply voltage which is a function of conversion time.
- the sensor in certain embodiments has application wherever brief (for example, few 100ns) bursts of light are generated and it is desired to capture accurately spatial position and time of arrival, as well as determining the number of photons in the event.
- One application is in the field of spectral fluorescence lifetime scanning for medical/scientific applications, e.g. endoscopy, cancer detection, ingestible pills.
- the invention is not limited to the above applications and may equally be used, for example, for time of flight distance sensing/imaging, 3D imaging, Lidar, Lidar for driverless cars (ADAS), people counting in buildings, 3D object scanning on production lines, seeing behind corners for defence, barcode scanning, ground digitisation in drones, 3D range imaging, object digitisation.
- the sensor may be used for any other suitable applications in further embodiments.
- the processing resource and the further processing resource may be provided as dedicated circuitry, for example a dedicated arrangement of electronic components to provide the desired functionality, or may be provided as software, hardware or any appropriate combination of software and hardware.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Radar, Positioning & Navigation (AREA)
- Electromagnetism (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
- Photometry And Measurement Of Optical Pulse Characteristics (AREA)
- Light Receiving Elements (AREA)
- Optical Radar Systems And Details Thereof (AREA)
Abstract
A sensor apparatus for photon sensing comprises a plurality of pixel devices, each pixel device comprising: a plurality of photon detectors arranged to produce photon detection signals in response to photon detection events; a processing resource configured to process photon detection signals to produce photon detection event signals, wherein each photon detection event signal comprises time data representative of a photon detection time at which a respective photon detection event occurred; a pixel memory; a further processing resource configured to process the photon detection event signals to obtain detection data representative of photon detection events over a detection period; a communication resource for transmitting the detection data from the pixel device, wherein the processing of the photon detection event signals is such that storing and/or transmission of the detection data uses less storage capacity and/or communication capacity than would be used by storage and/or transmission of the photon detection event signals directly.
Description
Photon sensor apparatus
Introduction
The present invention relates to a photon sensor apparatus, for example an apparatus suitable for asynchronous detection and measurement of sparse photonic events.
Background Charge-coupled devices (CCDs) and complementary metal-oxide semiconductors (CMOS) are two types of commonly used line sensor. Neither is capable of producing time-resolved outputs. Competing single photon avalanche diode (SPAD) array technology exists. SPADs in various light detectors, including point, line and 2D imaging arrays are known. A basic single SPAD receiver produces an electronic output when struck by a single photon. These individual receivers are typically arranged in an array together with additional circuitry to form a pixel device, which may be referred to as a pixel for brevity. In turn, the pixel devices may be arranged in a linear chain to form a line sensor, or in 2-D arrays to form an imaging sensor. These sensors provide a unique combination of light sensing capabilities for use in high speed sensitive optical instrumentation, and allow time-resolved information to be obtained in relation to ultra- fast optical phenomena. For time correlated single photon counting (TCSPC) histogramming, off-chip histogram memories are large. Known, solid-state SPAD sensors implement time to digital converters (TDCs) in pixel but transfer time-stamp data off-chip to perform TCSPC histogramming by using external memory resources, for example external solid-state memory or hard-drives. An input/output (I/O) data bottleneck may be reached as each photon detected by the SPAD requires data to be transferred off-chip.
Assuming a total of P pixels, a maximum I/O rate of R bits per second and L is a size of word (number of bits) generated per pixel, the maximum photon rate per pixel is limited to R/(L x P) Hz. As the number of pixels increases there is a corresponding decrease in the maximum rate of photons which can be transferred to an external memory. This rate may be significantly lower than both the rate that the SPADs can generate pulses
(100s of MHz) and the rate that the TDC can process those pulses (100s of MHz). The I/O rate of a chip is normally limited by the number of pads, data rate of those pads and power consumption. Rates R in the range of Gb/s are typical of high resolution CMOS image sensors. This leads to maximum photon rates of around 1 MHz for pixel arrays of around 1000 pixels, two orders of magnitude below the available peak capacity dictated by those other factors. For imagers with 10-100k pixels this situation is proportionately more constraining. Summary
In a first aspect of the invention there is provided a sensor apparatus for photon sensing, comprising: a plurality of pixel devices, each pixel device comprising: a plurality of photon detectors arranged to produce detection signals in response to photon detection events. The apparatus may comprise a processing resource, for example a processing resource dedicated to that pixel or shared with other pixels, configured to process the detection signals to produce photon detection event signals, wherein each photon detection event signal comprises time data representative of a detection time at which a respective photon detection event occurred. The apparatus may comprise a further processing resource configured to process the photon detection event signals to obtain detection data representative of photon detection events over a detection period. The apparatus may comprise a pixel memory. The apparatus may comprise a communication resource for transmitting the detection data from the pixel device. The processing of the photon detection event signals may be such that the storing and/or transmission of the detection data uses less storage capacity and/or communication capacity than would be used by storage and/or transmission of the photon detection event signals. The pixel memory may be configured to store the detection event signals and/or the detection data. In the case where the processing resource is a shared processing resource, for a plurality of the pixels there may be a single shared processing resource (e.g. a TDC) configured to process the detection signals to produce photon detection event signals.
Each photon detector may comprise a single photon avalanche diode (SPAD). The processing of the photon detection event signals may comprise a histogramming process and/or the detection data may comprise histogram data.
The processing of the detection signals produced by the photon detectors may comprise a time to digital conversion process. The detection data may represent a distribution of photons detected at the sensor apparatus as a function of time.
The processing of the photon detection event signals may comprise assigning the photon detection event signals between a plurality of time bins. The processing of the photon detection event signals may comprise obtaining count data representing a respective count value for each time bin of the plurality of time bins. In response to a detection event, collected and stored photon detection data may represent a time distribution of photons arriving at the sensor apparatus. Each time bin may have a width representing a time interval and the further processing resource may be configured to select or vary the width of at least one of the time bins.
The further processing resource may be configured to select or vary the width of at least one of the time bins automatically or in response to user input, and/or based on at least one configuration parameter.
The further processing resource may be configured to select the detection period in dependence on feature of interest for example a measured or expected position of a peak in a distribution of photon detection events.
The photon detection event signals may be obtained during the detection period and the processing of the photon detection event signals may comprise selecting a portion of the detection period and excluding photon detection event signals from outside the selected portion of the detection period, for example when processing the detection signals to obtain the detection data.
The processing resource may be configured to assign the detection event signals for the selected portion of the detection period across the time bins.
The further processing resource may be configured to modify and/or to select a part of the time data, for example in order to select or vary the width of the time bins and/or in order to select a portion of the detection period. Each item of time data may comprise a series of bits and the modifying of the time data may comprise, for at least some of the items of timestamp data, excluding at least some bits of the series of bits.
Each pixel device may be configured to reset the further processing resource and write signals from the further processing resource automatically.
The selecting of at least part of the time data may comprise, for at least some of the items of time data, selecting some of the bits of the series of bits. The apparatus may further comprise a delay generator for modifying a clock signal provided to the initial processing resource, thereby to delay collection of photon detection data and/or to shift the detection period.
The pixel memory may comprise one or more counters. The pixel memory may comprise tri-stated MUX components. Counters may be used to count the number of photon events detected for each bin. At the end of an exposure time, counter contents may be stored into histogram memory (in preparation of readout) and counters may be reset for the next exposure cycle. The pixel memory may have a size of at least 5 bytes or 10 bytes and less than 1 kilobytes. Preferably, the pixel memory is less than 50 bytes, for example 44 bytes. The pixel memory size may be selected in dependence on number of histogram bins and/or number of bits per bin, or vice versa. For example, there may be 32 bins and 1 1 -bits per bin, hence a total pixel histogram memory of 44 bytes.
For each of the pixel devices, the communication resource of the pixel device may be configured to transmit the detection data from the pixel device to an additional processing resource remote from the pixel device and/or a further memory remote from the pixel device.
The additional processing resource and/or the further memory may be configured to process and/or store detection data from each of the plurality of pixel devices.
The sensor apparatus may be provided on a chip or circuit board and the additional processing resource and/or further memory may be outside the chip or circuit board, for example remote from the chip or circuit board.
For each of the plurality of pixel devices, the initial and/or further processing resources and/or the memory of the pixel device may be located adjacent to or at least partially beneath the plurality of photon detectors.
The plurality of pixel devices may be arranged in an array, and for at least some of the pixel devices the further and/or initial processing resources and/or the memory of the pixel device may be located between the photon detectors of the pixel device and the photon detectors of an adjacent pixel device.
The photon detection events may occur in response to application of the laser radiation to a subject. The apparatus may be configured such that operation of the photon sensors and/or the processing of the photon detection event signals and/or the detection period is synchronised with operation of a laser that applies the laser radiation, for example pulsed laser radiation.
The apparatus may be configured for use in, for example, a Lidar system, a Time-of- flight system, a fluorescent spectroscopy system or a Raman spectroscopy system.
In a further aspect, which may be provided independently, there is provided a sensing method, comprising: obtaining photon detection event signals at a pixel device in response to photon detection events, wherein each photon detection event signal comprises time data representative of a detection time at which the photon detection event occurred; processing at the pixel device the photon detection event signals to obtain detection data representative of photon detection events over a detection period; transmitting the detection data from the pixel device, wherein the processing is such that the storing and/or transmission of the detection data uses less storage capacity and/or communication capacity than would be used by storage and/or transmission of the photon detection event signals.
In another aspect there is provided a sensing system comprising a laser configured to apply laser radiation to a subject, and a sensor apparatus as claimed or described herein that is configured to sense photons emitted by the subject in response to the laser radiation.
Features in one aspect may be provided as features in any other aspect, in any appropriate combination. For example, method features may be provided as apparatus features and vice versa.
Brief Description of the Drawings
Various aspects of the invention will now be described by way of example only, and with reference to the accompanying drawings, of which:
Figure 1 is a schematic diagram of a photon sensor apparatus;
Figure 2 is a schematic diagram showing in overview the architecture of a photon sensor chip according to an embodiment;
Figure 3 is a schematic diagram of a pixel element of the photon sensor apparatus;
Figures 4(a) and 4(b) are timing diagrams for a subset of signals generated and received by the photon sensor apparatus;
Figure 5 are illustrative plots of histograms produced by the photon sensor apparatus;
Figure 6 is a circuit diagram of an 1 1 -bit histogram bin;
Figure 7 is a circuit diagram of a histogram decoder;
Figure 8 is a circuit diagram of a 32-bin histogram memory;
Figure 9 shows example lifetime decays captured using the photon sensor;
Figure 10 shows maximal count rate comparisons captured using a line sensor;
Figure 1 1 shows time resolved 3-D spectra based on (a) TCSPC and (b) Histogram modes captured using a line sensor;
Figure 12 shows estimated lifetimes based on (a) TCSPC and (b) Histogram modes captured using a line sensor;
Figure 13 shows count rate comparisons between TCSPC and Histogram modes captured using a line sensor, and
Figure 14 illustrates a zooming feature of Histogram mode.
Detailed Description of the Drawings
An example arrangement for a photon sensor apparatus for measuring properties of a sample is shown in Figure 1 . The photon sensor apparatus has a photon sensor 10. An optical instrument, for example, an endoscope is provided. The optical instrument has a fibre bundle that comprises an imaging bundle, a capillary fibre and a fluorescent bead and/or Surface Enhanced Raman Spectroscopy (SERS) fibre. Two pulsed lasers are in optical communication with the fibre bundle via suitable focusing means and a microscope objective. The suitable focusing means may be a dichroic mirror.
Pulsed laser light from the lasers is coupled into the multimode fibre and used to illuminate the sample. In response to illumination of a sample, photon detection events occur. Detection events vary depending on application. For example, the detection events may arise from fluorescence scattering and/or Raman scattering. Signals are received by the fibre bundle and sent to the photon sensor apparatus via a mirror or grating. Data from the line sensor is then sent to a computer and displayed on the display. An example embodiment of the photon sensor 10 is shown in Figure 2. The photon sensor 10 has a pixel array including 1 x 512 pixels. The sensor architecture can provide a 1 M spectra per second read out and video rate photon counting/coarse time correlated single photon counting (TCSPC) for scanned operation. The sensor structure provides, amongst other operating modes on-chip and in-pixel real-time histogramming capability. The sensor structure also provides zooming and shifting capabilities to the in-pixel histogramming.
An illustrative pixel device 42, also referred to as a pixel for brevity, of the pixel array is shown in Figure 2. The pixel 42 includes two sets of eight "red" single-photon avalanche diodes (SPADs) 44 and two sets of eight "blue" SPADs 45. Although red and blue SPADs are provided in the embodiment of Figure 2, in alternative embodiments any other suitable arrangement of different SPAD device specifications including but not limited to differently colour sensitive SPADs may be provided, or all SPADS may have an identical specification for example the same wavelength sensitivity profile and/or all SPADs may be any mix of SPAD specifications.
The red SPADs 44 are sensitive to light in the range 600 nm to 900 nm. The blue SPADs 46 are sensitive to light in the range 450nm to 550nm. The two sets of red SPADs 44 and the two sets of blue SPADS 45, collectively form a pixel SPAD array 46. The pixel 42 also has a dedicated processing resource, for example a Time to Digital Convertor (TDC) 48, and a per-pixel memory resource 50 arranged in pixel. The TDC provides photon detection event signals, for example in the form of time stamp data, digital timing signals or other representation of the time an event occurs. A further processing resource associated with the per-pixel memory resource 50 is provided per pixel to process photon detection event signals to obtain detection data. The further processing resource may be provided as part of the per-pixel memory resource or as a separate component. The initial processing resource, per-pixel memory resource 50 and further processing resource are configured to perform in accordance with an operating mode of the sensor. The sensor also has a global processing resource in the form of a control module 52 and a delay generator 54. The control module 52 may include configuration registers that store the values of various configuration parameters. The sensor 10 also includes a communications resource in the form, in this embodiment, of a sensor output bus that includes a serialiser 56, and a readout module 60. The sensor output bus enables transmission of detection and other data from the sensor 10 to an external processing resource (not pictured).
The illustrative pixel device 42 has "red" and "blue" SPADS. However, the pixel device 42 is illustrative only. For example, the pixel device may also be implemented using only SPADs with identical specifications.
A moment of arrival of a single photon is registered as a change in state of the SPAD, for example from an un-activated state to an activated state. Following activation, each SPAD has an intrinsic dead-time period that typically lasts 10s of nanoseconds. After the dead time period has expired the SPAD resets to its original, un-activated state.
The width of each pixel is 23.78μιη and therefore the pixel array has a width, as indicated by 62, equal to approximately 12.175mm. The sensor 10 has a height indicated by 64, less than or equal to 2 mm. The sensor has a width 66 equal to approximately 13mm. The above dimensions are provided for illustrative purposes only and other dimensions and shapes are possible. The pixel array, and the individual
pixels can be of any suitable shape and size. As shown in Figure 2, the pixel array may be a one dimensional array having a width of one pixel to form a line sensor for spectrometry or line scanning. Alternatively, the pixel array may be a two dimensional array having both a length and width greater than one pixel to form an image sensor.
It is a feature of the embodiment that the sensor 10 has some of the memory and processing components placed on-chip, alongside the SPAD array, which may be used to ensure that only selected data is processed and/or transmitted and/or stored. The photon sensor 10 may operate in one of a variety of operating modes, the choice of which determines how a per-pixel memory resource 50 is used. Of particular interest is an on-chip histogramming mode, and operation in this mode is described with reference to Figures 3, 4 and 5. Other operating modes include single photon counting (SPC) mode (time gated) where the memory has two counters, counter A and counter B, both counters having, for example, 10-bit and 20-bit modes. Other operating modes also include TCSPC operating mode in, for example, 1 1 -bit or 17-bit modes.
Figure 3 is a flow chart and schematic diagram illustrating a sequence of operations performed by circuitry components and processing logic of the pixel 42 of the sensor. Figure 3 shows the pixel SPAD array 46 for detecting light, the TDC 48, and the per- pixel memory 50 of the pixel 42. Figure 3 also shows the delay generator 54. A controlling resource 70 is also shown. The controlling resource 70 may be provided as part of the on-chip controller 54 or provided off-chip. In addition to the above pixel components, the pixel also has pulse combining circuitry 72, a further processing resource, for example a decoder (shown in Figure 3 as part of the per-pixel memory 50) and a readout module 74.
In some embodiments, a single TDC, or at least some components of a TDC, is shared by a plurality of the pixels 42. Thus timestamps or other timing information for a plurality of the pixels 42 may be generated in such embodiments by a common TDC or by TDCs sharing at least some common components. Such sharing of TDCs may enable a reduced circuit area and power consumption, but may lead to more SPAD events being missed in some cases.
The delay generator 54 is a global component that is coupled to all pixels. As described in more detail below, the delay generator 54 provides a global modified clock signal to the pixels, indicated by STOPd. In particular, the modified clock signal is provided to the TDC 48. The modified clock signal generally coincides with occurrence of fluorescence or other photon emission process from a subject that causes photon detection events to occur at the sensor apparatus. The components of the pixel 42 are arranged such that the time of arrival of a photon at the array of SPADs 46 is measured with reference to the modified clock signal provided by the delay generator 54. The processing logic of the pixel 42 can provide measurements of time of arrival of a plurality of photons, which may for example be referred to as photon detection event signals, and to sort and/or otherwise process the measurements to produce and store a timing distribution, for example a histogram, across the per-pixel memory resource 50, which may for example be referred to as detection data. The pixel 42 can provide a readout signal, which may comprise or represent detection data, from the sensor output bus that represents the time distribution of detected photons at the SPAD array over a detection period. The time distributions are provided per-pixel and therefore, the sensor 10 provides time distributions for every pixel.
In further detail, each SPAD of the pixel SPAD array 46 is coupled to the pulse combiner 72 via a plurality of SPAD pulse channels, wherein each SPAD has a respective SPAD pulse channel. Each SPAD has corresponding processing logic (not shown) that includes a front end, a digitizer, a digital buffer, and quenching circuitry, which may be either active or passive. The digital buffer acts to isolate SPAD capacitance from being loaded by subsequent circuit and to transmit pulse signals to the pulse combining circuitry 72.
The pulse combiner circuitry 72, which forms part of the pixel processing logic in some embodiments, couples the SPAD array 46 to the TDC 48. The pulse combiner circuit acts as a compressor and combines the SPAD outputs into a single combined channel. The single channel provides one short pulse per photon count. The relatively long SPAD pulses (tens of ns) are compressed in time by the pulse combiner circuitry 72, thereby removing any dead time of SPAD pulses. The relatively long SPAD pulses (tens of ns) are first compressed in time by a monostable and then ORed together through an OR tree (spatial compression).
In an example implementation, where the SPAD array includes red and blue SPADs, the SPAD pulses from the red SPADs are combined separately from the SPAD pulses from the blue SPADs. In this example, either the blue or the red pulse stream is selected for the next processing step. The pulse combiner 72 provides an efficient means of reducing hardware requirements of other stages of the sensor, for example, TDC 48 and per-pixel memory resource 50. Some spatial information relating to the precise origin of the pulse signal is lost by combining the SPAD channels. For example, on examining a combined pulse signal it may not be possible to determine from which individual SPAD a pulse has originated.
The TDC 48, which forms part of the pixel processing logic, is configured to receive the single channel input from the pulse combiner circuitry 72 and the delayed clock signal from the delay generator 54, which acts as a global clock signal for each of the pixels of the sensor. The TDC 48 is configured to process these inputs and produce a photon detection event signal comprising time data representative of a detection time at which a photon detection event occurred.
The system can be configured to operate in different modes, for example TCSPC mode and histogramming mode. The TDC and other components can operate differently in TCSPC and histogramming mode. Further description of operation in TCSPC and histogramming modes is provided below in relation to Figure 4.
In histogramming mode, the TDC 48 is configured to process these inputs and produce a photon detection event signal comprising time data representative of a detection time at which a photon detection event occurred for each pulse of the combined pulse signal. The TDC can be a gated ring oscillator TDC.
The decoder, for example as shown in Figure 7, which forms part of the pixel processing logic, may form a separate component of the pixel or form part of the per- pixel memory resource 50. Alternatively, the decoder may form part of the TDC 48. The decoder provides an interface between the TDC 48 and the memory resource 50. The decoder is configured to produce decoded signals based, at least in part, on the photon detection event signals generated by the TDC 48 and at least one control parameter. The control parameter may be a zooming parameter that indicates, or that can be used
to determine the resolution of, a histogram. The operation of the decoder is described in further detail below.
Under user control, allocation of time domains to bins across the whole of the histogram can be successively improved in a "zooming" process until only the region containing useful information is spanned. Since it is not necessary to store all of the data that would normally be stored off-chip at any given time, it is possible to realise highly time resolved data with modest amounts of on-chip memory per pixel. Since this memory does not occupy too much area, it can be added to each pixel, while maintaining a reasonable fill factor for the optically sensitive area.
Each per-pixel memory resource 50 is partitioned into a number of memory portions, each memory portion acting as a time bin, for example for use in a histogramming process. The memory partitions may be counters. The decoder acts to provide decoded signals to a corresponding memory portion and the corresponding memory portion counts the received decoded signals. The per-pixel memory resource therefore produces and stores count data, for example count values, that represents a time distribution of photons, in the form of a histogram. Through for example the zooming parameter, or using any other suitable process, the decoder can set the histogram time resolution. The per-pixel memory resource 50 can be implemented in a number of alternative ways, including an array of ripple counters. The decoder may be implemented as a chain of logic gates, which selects one histogram bin to be incremented by one count. The pixel also has a pixel readout module 74 configured to receive a read signal from either another component of the sensor or externally from the sensor. In response to receiving the read signal, the readout module 74 is configured to produce a pixel output signal 78 that represents the state of the memory resource. The pixel readout module 74 provides pixel output to the sensor output bus.
The delay generator 54 is configured to receive a delay parameter. The delay parameter is generated by the controlling resource 70 based either on a user input value or the pixel output 78. The controlling resource 70 can provide a feedback loop between the pixel 42 and the delay generator 54. The delay generator 54 is configured to receive a STOP signal synchronised with pulses of the laser and to adjust the STOP
signal using the delay parameter to produce a delayed STOP signal. In a particular embodiment, the STOPd signal is a copy of STOP signal delayed by a timing of D x Tdel, where D is the delay parameter and Tdel is a basic delay resolution of a Delay Locked Loop (DLL) or programmable delay chain.
In use, in histogramming mode, a photon emission event occurs and a photon is incident on a SPAD of the pixel SPAD array 46 causing the pixel SPAD array 46 to produce a SPAD pulse. The SPAD pulse is transmitted from the SPAD to the pulse combiner 72 via its processing logic. The pulse combining circuitry 72 provides a combined pulse signal to the TDC 48. The TDC 48 generates photon detection event signals, in the form of timestamps, for each pulse of the combined pulse signal. Timestamps are produced for substantially each pulse of the combined pulse signal that is received by the TDC 48. It is noted that the time it takes for the TDC to process a detected SPAD event (e.g. generate a time stamp) may, in some cases, lead to missed pulses. For example, if generated SPAD pulses are too close to each other the TDC may not be able to process the next SPAD pulses being finishing processing the current SPAD pulse.
The decoder decodes the timestamps to produce a decoded signal and distributes the decoded signal to the correct portion of the per-pixel memory resource 50, e.g. representing a particular time bin, thereby increasing a count of the respective memory portion. Following a burst of photons incident on the pixel SPAD array, a count distribution is stored over the portions of the per-pixel memory 50. In response to a read signal the readout module 74 transmits the readout signal 78 representing the state of the per-pixel memory 50. The transmitted signal includes the count distribution for the pixel.
The transmitted pixel signal is received by the sensor output bus together with pixel signals from other pixels of the sensor. Together, the pixel signals can be further processed to produce a sensor array histogram. As example, sensor array histogram is shown in Figure 5. Photons can be counted by summing histograms bins off-chip.
While described as separate functional modules with reference to Figure 3, the TDC, decoder and memory may form part of a pixel processing logic circuit in communication with the delay generator that provides a STOP (or STOPd) signal. In one embodiment,
the TDC includes a gated ring oscillator, a ripple counter, and a shift register and the decoder includes a logic chain, for example as shown in Figure 7. The memory resource, in this embodiment is provided by an array of ripple counters implementing the in-pixel histogram, see for example Figures 6 and 8. In operation, a SPAD fires to start the gated ring oscillator (GRO) that operates the ripple counter in the TDC. This generates a clock for the shift register which clocks in a 1 . The STOP pulse samples the state of the shift register to generate a thermometer code which is decoded by the XOR chain to increment a single bin of the array of ripple counters implementing the in- pixel histogram. The STOP signal samples how long the shift register has been shifting along a 1 which then directly activates a corresponding histogram bin. The shift register may be operated from a divided version of the GRO output. In some embodiments, a decoder is not required if the other components of the pixel processing logic are arranged such that a single 1 can be shifted along. Turning to Figures 4(a) and 4(b), typical timing diagrams for the sensor in different operating modes are provided. Figure 4(a) shows a timing diagram for operation of the system in TCSPC mode. TCSPC is a favoured technique for low light time-resolved imaging due to its photon efficiency and specificity. Figure 4(b) shows a timing diagram for operation of the system in histogramming mode. The operation of the TDC, memory resource and decoder is dependent on the operating mode of the sensor.
For example, as can be seen in Figure 4(a), for operation in TCSPC mode, a photon detection event signal, for example a timestamp, is generated by the TDC only for the first photon captured within an exposure time window. For operation in histogramming mode as shown in Figure 4(b), a photon detection event signal is generated for each pulse of the combined SPAD pulses.
Both Figures 4(a) and 4(b) show seventeen waveforms. The first and uppermost waveform is that produced by a pulsed laser. The pulsed laser generates a stimulus for example, for a time of flight system or a fluorescence/Raman microscopy or spectroscopy system. In another example, a laser capable of generating short pulses typically at MHz rates may be employed to excite a sample or to generate an echo within a pulsed LIDAR system. A second waveform, the STOP clock, is synchronised to the laser pulse. The STOP clock is usually derived from a SYNC output from the laser. The STOP clock signal provides a reference signal against which arrival of
photons can be measured. The STOP clock signal is provided to the delay generator 54, and may be delayed by a set time as described elsewhere to provide the third waveform STOPd. The fourth waveform shows SPAD pulses. The fifth waveform is a variable exposure window (WIN) signal.
The sixth and seventh waveforms are TDC waveforms, corresponding to a TDC counter (TDC_counter) and TDC output (TDC_OUT) waveform. The eighth waveform is a pixel write waveform (PIX_WRITE). The ninth waveform is a pixel reset waveform (PIX_RESETn). Pixel reset signal is generated externally (through an FPGA board) together with some other control signals, for example, PIX_WRITE. The time interval between the pixel reset signals define the exposure time (however, only SPAD events which are within the active window (i.e. WIN in Figure 4(a)) are considered in each exposure cycle). The tenth waveform is an AUTO_SEQ signal. The eleventh and twelfth waveforms correspond to a TDC write signal (TDC_WRITE) and a TDC reset signal (TDC_RESETn). The thirteenth waveform (HIST_EN) corresponds to parameter Sel, which is a control register bit that enables histogramming mode in the memory resource. A histogram mode parameter (HIST_MODE) control signal is the fourteenth waveform. This control signal is described in further detail with reference to Tables 1 , 2 and 3. The fifteenth, sixteenth and seventeenth waveforms are a histogram readout waveform (HIST_READOUT), a TDC readout (TDC_READOUT) and a pixel output signal (PIX_OUT). Differences between TCSPC mode and histogramming mode are evident on a comparison of Figure 4(a) and 4(b). For example, in TCSPC mode a timestamp is generated only for the first photon captured within the exposure time. Furthermore, in TCSPC mode Histogram memory is by-passed and a generated timestamp is readout directly.
In histogramming mode, the TDC can convert many more SPADs within the exposure time and each converted SPAD is used for building the histogram (by incrementing one histogram BIN). In this mode TDC_WRITE and TDC_RESETn control signals are generated automatically on-chip and independently for each pixel (separate from global PIX_WRITE and PIX_RESETn signals) by setting AUTO_SEQ control signal to HIGH,
as shown in Figure 4(b). In histogramming mode, multiple photon events are converted during a single exposure cycle by these TDC reset and TDC write signals. In contrast, in TCSPC mode, only one photon event is converted per exposure cycle and all TDCs (of all pixels) are reset by the global pixel reset signal (PIX_RESTn).
With reference to Figure 4(b), and operation in histogramming mode, a photon is detected thus producing a detection signal in the form of a SPAD pulse (as seen in the fourth waveform). The SPAD pulse triggers the TDC counter signal, and the subsequent STOPd rising edge stops the TDC counter. TDC write and TDC reset control signals are automatically generated on-chip and independently for each pixel. These two TDC control signals are generated independently from the global pixel write and pixel reset signals. In response to receiving the TDC write control signal, the TDC writes the TDC value to memory. In response to receiving the TDC reset control signal, the TDC counter is reset to zero. As seen in Figure 4(b), a global pixel write control signal (PIX_WRITE) is sent to all pixels. On receiving the global pixel write control signal, current histogram bin counts are stored into histogram memory and the histogram bins are cleared on receiving the global pixel reset control signal (PIX_RESETn). This is then followed by reading out each histogram bin under the control of HIST_READOUT control signal.
The histogram memory may be reset by the global reset signals. However, in histogram mode, after detecting a photon event (e.g. generating a timestamp) the TDC data is sent to the histogram module for further processing and TDC counters are reset automatically (see for example, control signals AUTO_SEQ, TDC_WRITE and TDC_RESETn in Figure 4(b)).
Returning to Figure 3, zooming and delay functionality of the pixel is now described. The timing signals transmitted from the TDC 48 comprise a large number of timestamps (typically between 1 k and 1 M) as, in histogramming mode, a timestamp is produced for substantially every photon received by the TDC during the detection period. Each timestamp has a size that is determined by the TDC 48. For example, a TDC may be configured to output a timestamp that is an L-bit word. An L-bit timestamp can store information corresponding to 2L time intervals, in this way each TDC has an in-built maximum time resolution.
The memory resource 50 has a defined number of partitions or counters, each counter having a predetermined size. Each counter is a suitable size to store count information. For example, a counter having an M-bit size can count up to 2M occurrences of an event.
Example values are L=10 and M=8. As a comparative example, it is considered how large a memory resource needs to be to store timing information for events at the maximum time resolution. For the above values and at this resolution, 2L=1024 counters (a counter for each time interval) are required, where each counter is 8-bit (1 byte). To store timing information at this resolution, a memory is 1024 bytes in size for each pixel. Such a size is prohibitively large for an on-chip per-pixel memory.
Therefore, per-pixel memory resource 50 is provided with a reduced number of memory portions, where each memory portion corresponds to a respective time bin. There are N time bins in total, where N < 2L. By reducing the number of bins, the per- pixel memory area overhead required is reduced and the memory resource 50 is placed within the pixel 42. Memory placed within a pixel can achieve 100s MHz accumulation of timestamps due to the proximity of memory to the TDC 48. In the embodiment, histogram zooming is implemented by selecting different bits of the TDC output.
The per-pixel memory resource is sized M x N bit, where N is the number of time bins for a histogram and M is the size of each counter for each bin. The pixel decoder receives L-bit timestamps, selects a portion of L-bit timestamps and determines to which bin the selected portions of L-bit timestamps should be allocated.
In an example implementation, the pixel decoder receives input signals containing configuration parameters K and Sel. Parameter K may be considered as a zooming parameter and its value determines the size of the time bins. Sel is a control register bit that enables the histogramming mode in the memory resource. The input signals K and Sel are indicated as HIST_MODE and HIST_EN respectively in Figure 4(a) and Figure 4(b). The decoder selects a first portion of the L-bit word determined by the values of K and N, in particular, the decoder selects the portion of the L-bit word corresponding to bits K to K + log2N - 1 . This selection may be considered to effectively exclude at least some of the bits when performing the histogramming. This decoded signal, therefore
has a size equal to log2K. By performing the same operation on each L-bit timestamp, the time interval of each histogram bin is scaled by a factor dependent on zooming parameter K. K can be chosen to be any value from 0 to L - log2 V. Selection of zooming parameter K increases the bin time width from the TDC resolution bin width by a factor of 2K .
An example implementation of on-chip histogram option is shown in Figure 5 and in Tables 1 , 2 and 3. In this particular implementation, options are specified for a 12-bit (L=12) TDC with a time resolution of 50ps. In this case, K can be any value from 0 to 7, for example set automatically or based upon user input. The selection or varying of the value of K can effectively select or vary the time width of the bins, and thus effectively select or vary a level of histogram zooming. Options 1 to 8 are shown in Table 1 and these options correspond to values of K from 7 to 0. Histograms produced under options 1 , 2 and 3 are shown in Figure 5. In this example implementation, the number of time bins, N=32. In this implementation, a 5-bit portion of the 12-bit TDC timestamp is selected. As 25 = 32, the selected 5-bit portion represents one of the 32 possible time intervals represented by the time bins and therefore determines which bin the photon should be allocated to. Figure 5(a) shows a laser sync signal corresponding to the STOP signal received by the TDC. Figure 5(b) shows an actual response curve corresponding to a fluorescence or other photon emission process measured with photon counts over time. The response curve has a first sloping segment up to a peak and a decaying segment after the peak. The histograms shown in Figures 5(c) to 5(e) are produced by the system represent the shape of the response curve at consecutively finer resolutions.
Selecting option 1 corresponds to selecting K to be the largest possible value, 7. A plot of a resulting histogram is shown in Figure 5(c). This choice causes the decoder to select a 5-bit portion of the 12-bit timestamp output by the TDC, the portion corresponding to bits 7 to 1 1 of the 12-bit timestamp. The decoder then assigns the 5- bit portion to the histogram memory (i.e. histogram bins). Bits 0 to 6 determine the level of binning. In this case, bits 0 to 6 correspond to 2Λ7=128 consecutive TDC codes. Therefore, TDC codes 0 to 127 are binned together in the first bin and the first bin counter incremented by one. Likewise, TDC codes 128 to 255 are binned together in the second bin and the second bin counter increment by one. In this way all 32 bins
counters are updated. The total time range of the histogram in option 1 is 204.8 ns and the bin width is 6.4ns.
Selecting option 2 corresponds to selecting K to be the second largest possible value, 6. A plot of a resulting histogram is shown in Figure 5(d). This choice causes the decoder to select a 5-bit portion of the 12-bit timestamp output by the TDC, the portion corresponding to bits 6 to 10 of the 12-bit timestamp. The decoder then assigns the 5- bit portion to the histogram memory. Bits 0 to 5 determine the level of binning. In this case, bits 0 to 5 correspond to 2Λ6=64 consecutive TDC codes. Therefore, TDC codes 0 to 63 are binned together in the first bin and the first bin counter incremented by one. Likewise, TDC codes 64 to 127 are binned together in the second bin and the second bin counter incremented by one. In this way all 32 bins are provided with entries. Information held by bit 1 1 is not used. Losing the highest bit, bit 1 1 , reduces the value of the total TDC range by half, compared to option 1 . Hence the total time range of the histogram in option 2 is 102.4 ns and the bin width is 3.2ns.
Selecting option 3 corresponds to selecting K to be the third largest possible value, 5. A plot of a resulting histogram is shown in Figure 5(e). This choice causes the decoder to select a 5-bit portion of the 12-bit timestamp output by the TDC, the portion corresponding to bits 5 to 9 of the 12-bit timestamp. The decoder then assigns the 5-bit portion to the histogram memory. Bits 0 to 4 determine the level of binning. In this case, bits 0 to 4 correspond to 2Λ5=32 consecutive TDC codes. TDC codes 0 to 31 are binned together in the first bin and the first bin counter incremented by one. Likewise, TDC codes 32 to 63 are binned together in the second bin and the second bin counter incremented by one. In this way all 32 bins are provided with entries. Information held by bits 10 and 1 1 is not used. Losing the highest 2 bits, bits 10 and 1 1 , reduces the value of the total TDC range by half, compared to option 2. Hence the total time range of the histogram in option 3 is 51.2 ns and the bin width is 1 .6 ns. Further choices of K are shown in Table 1 , 2 and 3. Selecting option 8 corresponds to selecting K to be the smallest possible value, 0. This choice causes the decoder to select a 5-bit portion of the 12-bit timestamp output by the TDC, the portion corresponding to bits 0 to 4 of the 12-bit timestamp. The decoder then assigns the 5-bit portion to the histogram memory. Therefore, timing information held by bits 5 to 1 1 is not stored.
For option 8 the bin width is 0.05ns and the total TDC range is 1 .6ns. The resolution in this option corresponds to the inherent resolution of the TDC. In other words, the time bin width is equal to the time resolution of the TDC.
Table 1 : Options provided for adjusting TDC range and histogram bin width
Histogram bin width
HIST
Selected
MODE TDC Range no bin chaining with bin chaining
TDC bits
<2:0> (32 bins, (16 bins,
10-bit/bin) 20-bit/bin)
6.4ns
7 TDC<11 :7> 4096x50ps =204.8ns 12.8ns (=204.8/16)
(=204.8/32)
3.2ns
6 TDC<10:6> 2048x50ps =102.4ns 6.4ns (=102.4/16)
(=102.4/32)
5 TDC<9:5> 1024x50ps = 51.2ns 1 .6ns (=51 .2/32) 3.2ns (=51 .2/16)
4 TDC<8:4> 512x50ps = 25.6ns 0.8ns 1.6ns
3 TDC<7:3> 256x50ps = 12.8ns 0.4ns 0.8ns
2 TDC<6:2> 128x50ps = 6.4ns 0.2ns 0.4ns
1 TDC<5:1 > 64x50ps = 3.2ns 0.1 ns 0.2ns
0 TDC<4:0> 32x50ps = 1 .6ns 0.05ns 0.1 ns
Table 2: Further parameters for modes presented in Table 1 . HIST MODE <2:0> corresponds to value selected for K. Table 2 expresses the TDC range in nanoseconds for each selected mode. The total TDC range is calculated by multiplying the number of available time intervals (212 = 4096 in the first case) by the time resolution of the TDC (50ps).
Table 3: Further parameters for modes presented in Table 1 . Table 3 expresses the same TDC range of Tables 1 and 2 as the available number of TDC codes, and histogram bin width is expressed as the number of TDC codes.
In Tables 1 , 2 and 3 histogram bin sizes are described with and without bin-chaining. Bin-chaining allows two M-bit bins to be used as a single 2M— bit bin. As a consequence the bin size is doubled and the number of bins is halved. A circuit diagram of an example implementation of an 1 1 -bit histogram bin is shown in Figure 6. Each bin is implanted as an 1 1 -bit ripple adder, and a MUX-based tri-stated memory units. A circuit diagram of an example implementation of a 32 bin histogram is shown in Figure 7.
In an example implementation of bin chaining, each BIN is implemented as in Figure 6. In operation, in bin chaining mode, two 1 1 -bit adders are simply chained together (the most significant bit of one adder is connected to the least significant bit of the other adder), forming a 22-bit adder (effectively 20-bit since the remaining 2 bits are overflow bits).
A circuit diagram of an example implementation of a histogram decoder which is an example of a further processing resource is shown in Figure 7. The histogram decoder takes as input: a timestamp, from the TDC (TDC_out), a histogram mode (HIST_MODE) control signal and a histogramming mode enable (HIST_EN) control signal. The histogram mode control signal indicates the value of K, as described with reference to Tables 1 , 2 and 3, is used.
Histogram decoder checks if TDC timestamp is within the histogram window using value of K. A 5-bit TDC data chunk is then selected from the TDC output using the value of K, as described with reference to Tables 1 , 2 and 3. The 5 bit selected TDC data chunk is then converted by a 5 to 32 decoder and this output is then provided to the histogramming memory to increment the corresponding histogram bin counter.
A circuit diagram of 32 histogram BIN memory is shown in Figure 8. Figure 8 also shows two single photon counters, SPCA and SPCB, included, for operation in single photon counting mode. The histogram decoder, as described with reference to Figure 7, outputs histogram bin information derived from TDC timestamp which increments the bin counters. Allocation may be performed sequentially.
In addition to the described zooming feature, the delay signal also provides a delay offset of histogram prior to zooming. The delay offset shifts a feature of interest, for example the peak and exponential decay so that this feature occupies the first bins of the histogram when these bins are expanded by a factor 2K. The parameter D shifts the sharp feature of a histogram corresponding to the laser pulse towards the first few bins of the histogram to allow effective zooming. This can be done automatically, or by a user of the system. Through the control resource 70, the system monitors and adjusts histogram peak position by using coarse bin sizes and progressively reducing K whilst adjusting the peak by precise choices of D. Zooming to features of an optical waveform around a sharp feature created by a pulsed laser is important for time of flight ranging systems as it gives a more precise distance estimation. Zooming and delay is also important for Raman spectrometry as it provides suppression of fluorescence component. The use of a delay signal as outlined in the preceding paragraph can effectively be used to select a portion of a detection period (e.g. the portion of the detection period starting at or just before the feature of interest, such as the peak and subsequent decay) and exclude detection signals from before and/or after the selected portion of the detection period (e.g. detection signals, if any, obtained at times significantly earlier than the peak).
Any other suitable process can be used to select a detection period, or a portion of a detection period, that is to be used in a histogramming or other process. For example in some embodiments a thresholding process may be used to determine a time for which the number of counts reaches or exceeds a threshold level. The portion of a detection period before that point in time may then be excluded from use in a histogramming or other process, as it may be considered not to include sufficient useful data. Similarly, in some embodiments a thresholding process may be used to determine a time at which the number of counts has fallen below a further threshold level after the peak. Data after that point in time may for example then be excluded, effectively cutting off the end of a decay curve. In other embodiments, the detection period, or the portion of the detection period, to be used may be selected based on expected time period when photon detection events may be expected. For example, following application of a laser pulse to a subject there may be an expected dead time before photon detection events may be expected to begin, and there may be an
expected length for a subsequent decay process during which the number of photon detection events falls from a peak to zero or to a threshold level. In some embodiments the sensor apparatus may be synchronised with operation of the laser such that the detection period is selected so that it matches a period for which useful or desired data, for example detection event signals, may be expected to be obtained.
Figure 9 shows an example lifetime decay captured in on-chip per-pixel histogramming using the one dimensional sensor described with reference to Figure 2. The upper plot is a 3 dimensional plot showing a pixel array histogram. The x and y axes display pixel number and bin number. The bin number runs from 0 to 31 . The pixel number runs from 0 to 51 1. The z-axis displays photon counts. The plot therefore displays, for each pixel, photon counts for each time bin.
The lower plot shows a corresponding heat map. The x-axis displays bin number and the y-axis displays pixel number. The colour of each pixel-bin corresponds to the height shown in the upper plot which is the photon count.
It is a feature of certain embodiments that for each of the plurality of pixels, the per- pixel processing resource and/or memory of the pixels may be located adjacent to or at least partially beneath the plurality of photon detectors, for example formed beneath the SPADs in a layered structure. The pixels may be arranged in an array, and for at least some of the pixels the processing resource and/or the memory of the pixel device is located between the SPADs or other photon detectors of the pixel and the SPADs or other photon detectors of an adjacent pixel.
The sensor 10 of the embodiment of Figure 2 may be provided on a single chip or circuit board, or on a plurality of chips or circuit boards, and the communications resources of the pixel devices may be configured to transmit detection data off the chip(s) or circuit board(s), for example to a further memory, processor or other processing resource that is external to the chip(s) or circuit board(s). For example, the detection data from each of the pixel devices may be transmitted to a PC or other computer, or to a server, or to dedicated processing and storage hardware for further processing and storage. The chip(s) or circuit board(s) may include a further communication resource, and the detection data from the communications resources of the pixel devices may be transmitted to the further memory and/or further processing
resource via the further communications resource of the chip(s) or circuit board(s). The detection data may be received and re-transmitted by the further communications resource. The chip(s) and circuit board(s) in some embodiments may include an additional memory for storing detection or other data and/or an additional processing resource that may perform further processing of the detection data before transmission off-chip or off-circuit board. The additional processing resource may, for example, process the detection data to extract features of interest. The features of interest may comprise features of interest identified in the histogram data, for example a position or height of a peak, or one or more fitting parameters that may be used to fit a decay curve to the histogram data. By including the additional processing resource on chip, feature extraction or other processes may be performed more rapidly. However, the additional processing resource is not limited to being provided on the same chip or circuit board as the detectors and in some embodiments the additional processing resource may be provided off-chip or off-circuit board.
A line sensor according to embodiments was tested under controlled LED illumination triggered at 100MHz to measure the maximum count rate in all modes. As shown in Figure 10, per-pixel histogramming achieves -1/4 SPC count rate demonstrating the ability to maximise TCSPC event timing. Fluorescein-Rhodamine FRET was investigated in order to demonstrate the time-resolving spectral capabilities of the sensor. The line sensor was placed in a spectrograph and captured fluorescence from a sample illuminated by a 483nm pulsed laser diode (20MHz repetition rate). Figure 1 1 shows a time-resolved spectrum of the mixture obtained in TCSPC mode (Figure 1 1 (a)) and the on chip histogramming mode (Figure 1 1 (b)). Two clear peaks are obtained at wavelengths of 527nm and 584nm, with two distinct lifetimes. The fluorescent lifetimes are as expected for the dyes involved (Figure 12). Figure 13 shows that the on-chip histogramming data acquisition rate per wavelength is up to 15 times greater than raw TCSPC mode. This increases to 68 times with a Fluorescein sample alone. Zooming from 3.2ns/bin to 0.8ns/bin is demonstrated in Figure 14. The sensor empowers existing time-resolved spectroscopy and imaging applications and enables new ones.
In certain embodiments, the sensor may only consume power when a photon hits. Another advantage that may be provided by embodiments is that the sensor also reduces large data volume required to be output by SPAD based time-resolved image
sensors. In addition, the sensor has a low power requirement scaling with number of photons incident sensor. A further advantage is that the sensor can be efficiently implemented in a stacked 3D CMOS with an advanced lower tier process. A skilled person will appreciate that variations of the enclosed arrangement are possible without departing from the invention. For example, the per-pixel memory resource 50 may be part of a larger memory resource. For example, a memory resource may be provided for all pixels, but partitioned into independent pixel partitions for each pixel. In a further example, while time bins are shown as equal in width a varying time mode can be implemented to provide logarithmically sized time bins for the histogram, for example so that each time bin width has a size T, 2T, 4T, 8T etc. This can equalise photon counts in each bin for a response curve that is exponentially decreasing waveform as found in fluorescence lifetimes. Further variants of the sensor may include to eliminate the need for a thermometer decoder and to reduce power by operating a TDC at a supply voltage which is a function of conversion time.
The sensor in certain embodiments has application wherever brief (for example, few 100ns) bursts of light are generated and it is desired to capture accurately spatial position and time of arrival, as well as determining the number of photons in the event. One application is in the field of spectral fluorescence lifetime scanning for medical/scientific applications, e.g. endoscopy, cancer detection, ingestible pills. The invention is not limited to the above applications and may equally be used, for example, for time of flight distance sensing/imaging, 3D imaging, Lidar, Lidar for driverless cars (ADAS), people counting in buildings, 3D object scanning on production lines, seeing behind corners for defence, barcode scanning, ground digitisation in drones, 3D range imaging, object digitisation. The sensor may be used for any other suitable applications in further embodiments.
The processing resource and the further processing resource may be provided as dedicated circuitry, for example a dedicated arrangement of electronic components to provide the desired functionality, or may be provided as software, hardware or any appropriate combination of software and hardware.
The above description of specific embodiments is made by way of example only and not for the purposes of limitation. It will be clear to the skilled person that modifications of detail may be made within the scope of the invention.
Claims
1. A sensor apparatus for photon sensing, comprising:
a plurality of pixel devices, each pixel device comprising:
a plurality of photon detectors arranged to produce photon detection signals in response to photon detection events;
a processing resource configured to process photon detection signals to produce photon detection event signals, wherein each photon detection event signal comprises time data representative of a photon detection time at which a respective photon detection event occurred;
a pixel memory;
a further processing resource configured to process the photon detection event signals to obtain detection data representative of photon detection events over a detection period;
a communication resource for transmitting the detection data from the pixel device, wherein
the processing of the photon detection event signals is such that storing and/or transmission of the detection data uses less storage capacity and/or communication capacity than would be used by storage and/or transmission of the photon detection event signals directly.
2. A sensor apparatus according to claim 1 , wherein each photon detector comprises a single photon avalanche diode (SPAD).
3. A sensor apparatus according to claim 1 or 2, wherein the processing of the photon detection signals produced by the photon detectors comprises a time to digital conversion process.
4. A sensor apparatus according to claims 1 to 3, wherein the processing of the photon detection event signals comprises a histogramming process and/or the detection data comprises histogram data.
5. A sensor apparatus according to any preceding claim, wherein the further processing resource is configured to select the detection period in dependence on a measured or expected position of a peak in a distribution of photon detection events.
6. A sensor apparatus according to any preceding claim, wherein the photon detection event signals are obtained during the detection period and the processing of the photon detection event signals comprises selecting a portion of the detection period and excluding photon detection event signals from outside the selected portion of the detection period.
7. A sensor apparatus according to any preceding claim, wherein the detection data represents a distribution of photons detected at the sensor apparatus as a function of time.
8. A sensor apparatus according to any preceding claim, wherein the processing of the photon detection event signals comprises assigning the photon detection event signals between a plurality of time bins and, optionally obtaining count data representing a respective count value for each time bin of the plurality of time bins.
9. A sensor apparatus according to claim 8, wherein each time bin has a width representing a time interval and the further processing resource is configured to select or vary the width of at least one of the time bins.
10. A sensor apparatus according to claim 9, wherein the further processing resource is configured to select or vary the width of at least one of the time bins automatically or in response to user input, and/or based on at least one configuration parameter.
11 . A sensor apparatus according to any of claims 8 to 10, wherein the further processing resource is configured to assign the photon detection event signals for the detection period or the selected portion of the detection period across the series of time bins.
12. A sensor apparatus according to any preceding claim, wherein the further processing resource is configured to modify and/or to select a part of the time data in order to select or vary the width of the time bins and/or in order to select a portion of the detection period.
13. A sensor apparatus according to claim 12, wherein each item of time data comprises a series of bits and the modifying of the time data comprises, for at least some of the items of time data, excluding at least some bits of the series of bits.
14. A sensor apparatus according to any preceding claim, wherein each pixel device is configured to reset the further processing resource and write signals from the further processing resource automatically.
15. A sensor apparatus according to any preceding claim, wherein, for each of the pixel devices, the communication resource of the pixel device is configured to transmit the detection data from the pixel device to an additional processing resource remote from the pixel device and/or a further memory remote from the pixel device.
16. A sensor apparatus according to claim 15, wherein the additional processing resource and/or the further memory is configured to process and/or store detection data from each of the plurality of pixel devices.
17. A sensor apparatus according to claim 15, wherein the detection data comprises or is representative of time varying data and the additional processing resource is arranged to extract at least one feature of interest from the time varying data.
18. A sensor apparatus according to claim 15 or 16, wherein the sensor apparatus is provided on a chip or circuit board and the additional processing resource and/or further memory is outside the chip or circuit board.
19. A sensor apparatus according to any preceding claim wherein, for each of the plurality of pixel devices, the further processing resource and/or the memory of the pixel device is located adjacent to or at least partially beneath the plurality of photon detectors.
20. A sensor apparatus according to any preceding claim, wherein the plurality of pixel devices are arranged in an array, and for at least some of the pixel devices the further processing resource and/or the memory of the pixel device is located between
the photon detectors of the pixel device and the photon detectors of an adjacent pixel device.
21 . A sensor apparatus according to any preceding claim, wherein for at least some of the pixel devices the processing resource comprises a shared processing resource or component that is shared by at least some of the plurality of pixel devices.
22. An apparatus as claimed in any of the preceding claims, wherein the photon detection events occur in response to application of laser radiation to a subject, and the apparatus is configured such that operation of the photon sensors and/or the processing of the detection event signals and/or the detection period is synchronised with operation of a laser that is configured to apply the laser radiation.
23. An apparatus as claimed in any of the preceding claims configured for use in a Lidar system, a Time-of-flight system, a fluorescent spectroscopy system or a Raman spectroscopy system.
24. A sensing method, comprising:
obtaining photon detection event signals at a pixel device in response to photon detection events, wherein each photon detection event signal comprises time data representative of a detection time at which the photon detection event occurred;
processing at the pixel device the photon detection event signals to obtain detection data representative of photon detection events over a detection period;
transmitting the detection data from the pixel device, wherein
the processing is such that the storing and/or transmission of the detection data uses less storage capacity and/or communication capacity than would be used by storage and/or transmission of the photon detection event signals.
25. A sensing system comprising a laser configured to apply laser radiation to a subject, and a sensor apparatus according to any of claims 1 to 23 that is configured to sense photons emitted by the subject in response to the laser radiation.
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201780086502.2A CN110312945B (en) | 2016-12-30 | 2017-12-28 | Photon sensor device |
EP17825598.0A EP3563174A1 (en) | 2016-12-30 | 2017-12-28 | Photon sensor apparatus |
US16/474,000 US11460560B2 (en) | 2016-12-30 | 2017-12-28 | Photon sensor apparatus |
JP2019535786A JP7088935B2 (en) | 2016-12-30 | 2017-12-28 | Photon sensor device |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GBGB1622429.7A GB201622429D0 (en) | 2016-12-30 | 2016-12-30 | Photon sensor apparatus |
GB1622429.7 | 2016-12-30 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2018122560A1 true WO2018122560A1 (en) | 2018-07-05 |
Family
ID=58412337
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/GB2017/053900 WO2018122560A1 (en) | 2016-12-30 | 2017-12-28 | Photon sensor apparatus |
Country Status (6)
Country | Link |
---|---|
US (1) | US11460560B2 (en) |
EP (1) | EP3563174A1 (en) |
JP (1) | JP7088935B2 (en) |
CN (1) | CN110312945B (en) |
GB (1) | GB201622429D0 (en) |
WO (1) | WO2018122560A1 (en) |
Cited By (47)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2019007950A (en) * | 2017-06-22 | 2019-01-17 | 株式会社デンソー | Optical distance measuring device |
EP3537180A1 (en) * | 2018-03-08 | 2019-09-11 | ZF Friedrichshafen AG | Receiver for receiving light pulses, lidar module and method for receiving light pulses |
CN110596724A (en) * | 2019-09-19 | 2019-12-20 | 深圳奥锐达科技有限公司 | Method and system for measuring flight time distance during dynamic histogram drawing |
CN110596723A (en) * | 2019-09-19 | 2019-12-20 | 深圳奥锐达科技有限公司 | Method and system for measuring flight time distance during dynamic histogram drawing |
CN110596722A (en) * | 2019-09-19 | 2019-12-20 | 深圳奥锐达科技有限公司 | System and method for measuring flight time distance with adjustable histogram |
CN110609293A (en) * | 2019-09-19 | 2019-12-24 | 深圳奥锐达科技有限公司 | Distance detection system and method based on flight time |
WO2020049203A1 (en) * | 2018-09-05 | 2020-03-12 | Consejo Superior De Investigaciones Científicas | Method and device for detecting the peak of the compressed histogram of pixel values in high-resolution time-of-flight sensors |
WO2020106661A1 (en) * | 2018-11-19 | 2020-05-28 | Sense Photonics, Inc. | Digital pixel |
WO2020118017A1 (en) * | 2018-12-05 | 2020-06-11 | Sense Photonics, Inc. | Hybrid center of mass method (cmm) pixel |
WO2020150574A1 (en) * | 2019-01-18 | 2020-07-23 | Sense Photonics, Inc. | Digital pixels and operating methods thereof |
WO2020153272A1 (en) * | 2019-01-24 | 2020-07-30 | ソニーセミコンダクタソリューションズ株式会社 | Measuring device, ranging device, and method of measurement |
WO2020153275A1 (en) * | 2019-01-24 | 2020-07-30 | ソニーセミコンダクタソリューションズ株式会社 | Distance measurement device, in-vehicle system, and distance measurement method |
WO2020166419A1 (en) * | 2019-02-13 | 2020-08-20 | ソニーセミコンダクタソリューションズ株式会社 | Light reception device, histogram generation method, and distance measurement system |
WO2020184325A1 (en) * | 2019-03-12 | 2020-09-17 | ソニーセミコンダクタソリューションズ株式会社 | Measurement device, ranging device, and measurement method |
DE102019107115A1 (en) * | 2019-03-20 | 2020-09-24 | Valeo Schalter Und Sensoren Gmbh | LIDAR device for a vehicle and method for increasing the detection range of a corresponding LIDAR device |
WO2020226840A1 (en) * | 2019-05-06 | 2020-11-12 | Hi Llc | Photodetector architectures for time-correlated single photon counting |
US10868207B1 (en) | 2019-06-06 | 2020-12-15 | Hi Llc | Photodetector systems with low-power time-to-digital converter architectures to determine an arrival time of photon at a photodetector based on event detection time window |
WO2020255759A1 (en) * | 2019-06-20 | 2020-12-24 | ソニーセミコンダクタソリューションズ株式会社 | Distance measurement device, distance measurement method, and distance measurement system |
US11004998B2 (en) | 2018-05-17 | 2021-05-11 | Hi Llc | Wearable brain interface systems including a headgear and a plurality of photodetector units |
US11006876B2 (en) | 2018-12-21 | 2021-05-18 | Hi Llc | Biofeedback for awareness and modulation of mental state using a non-invasive brain interface system and method |
US11081611B2 (en) | 2019-05-21 | 2021-08-03 | Hi Llc | Photodetector architectures for efficient fast-gating comprising a control system controlling a current drawn by an array of photodetectors with a single photon avalanche diode |
US11096620B1 (en) | 2020-02-21 | 2021-08-24 | Hi Llc | Wearable module assemblies for an optical measurement system |
WO2021188487A1 (en) * | 2020-03-20 | 2021-09-23 | Hi Llc | Temporal resolution control for temporal point spread function generation in an optical measurement system |
WO2021188486A1 (en) * | 2020-03-20 | 2021-09-23 | Hi Llc | Phase lock loop circuit based adjustment of a measurement time window in an optical measurement system |
US20210302550A1 (en) * | 2020-03-30 | 2021-09-30 | Stmicroelectronics (Research & Development) Limited | Time to digital converter |
US11187575B2 (en) | 2020-03-20 | 2021-11-30 | Hi Llc | High density optical measurement systems with minimal number of light sources |
EP3936947A1 (en) * | 2020-03-30 | 2022-01-12 | STMicroelectronics (Research & Development) Limited | Latched gray code for tof applications |
US11245404B2 (en) | 2020-03-20 | 2022-02-08 | Hi Llc | Phase lock loop circuit based signal generation in an optical measurement system |
US11415679B2 (en) | 2015-08-20 | 2022-08-16 | Apple Inc. | SPAD array with gated histogram construction |
US11500094B2 (en) | 2019-06-10 | 2022-11-15 | Apple Inc. | Selection of pulse repetition intervals for sensing time of flight |
US11515014B2 (en) | 2020-02-21 | 2022-11-29 | Hi Llc | Methods and systems for initiating and conducting a customized computer-enabled brain research study |
US11555900B1 (en) | 2019-07-17 | 2023-01-17 | Apple Inc. | LiDAR system with enhanced area coverage |
US11630310B2 (en) | 2020-02-21 | 2023-04-18 | Hi Llc | Wearable devices and wearable assemblies with adjustable positioning for use in an optical measurement system |
US11681027B2 (en) | 2017-06-29 | 2023-06-20 | Apple Inc. | Time-of-flight depth mapping with parallax compensation |
US11681028B2 (en) | 2021-07-18 | 2023-06-20 | Apple Inc. | Close-range measurement of time of flight using parallax shift |
US11733359B2 (en) | 2019-12-03 | 2023-08-22 | Apple Inc. | Configurable array of single-photon detectors |
US11771362B2 (en) | 2020-02-21 | 2023-10-03 | Hi Llc | Integrated detector assemblies for a wearable module of an optical measurement system |
US11819311B2 (en) | 2020-03-20 | 2023-11-21 | Hi Llc | Maintaining consistent photodetector sensitivity in an optical measurement system |
US11852727B2 (en) | 2017-12-18 | 2023-12-26 | Apple Inc. | Time-of-flight sensing using an addressable array of emitters |
US11857348B2 (en) | 2020-03-20 | 2024-01-02 | Hi Llc | Techniques for determining a timing uncertainty of a component of an optical measurement system |
US11864867B2 (en) | 2020-03-20 | 2024-01-09 | Hi Llc | Control circuit for a light source in an optical measurement system by applying voltage with a first polarity to start an emission of a light pulse and applying voltage with a second polarity to stop the emission of the light pulse |
US11877825B2 (en) | 2020-03-20 | 2024-01-23 | Hi Llc | Device enumeration in an optical measurement system |
US11883181B2 (en) | 2020-02-21 | 2024-01-30 | Hi Llc | Multimodal wearable measurement systems and methods |
US11903676B2 (en) | 2020-03-20 | 2024-02-20 | Hi Llc | Photodetector calibration of an optical measurement system |
US11950879B2 (en) | 2020-02-21 | 2024-04-09 | Hi Llc | Estimation of source-detector separation in an optical measurement system |
US11969259B2 (en) | 2020-02-21 | 2024-04-30 | Hi Llc | Detector assemblies for a wearable module of an optical measurement system and including spring-loaded light-receiving members |
US12029558B2 (en) | 2020-02-21 | 2024-07-09 | Hi Llc | Time domain-based optical measurement systems and methods configured to measure absolute properties of tissue |
Families Citing this family (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3811113A1 (en) | 2018-06-22 | 2021-04-28 | Ams Ag | Using time-of-flight and pseudo-random bit sequences to measure distance to object |
US20210223372A1 (en) * | 2020-01-21 | 2021-07-22 | Semiconductor Components Industries, Llc | Lidar system with dynamic resolution |
CN111487637B (en) * | 2020-04-20 | 2023-12-01 | 深圳奥锐达科技有限公司 | Distance measurement system and method based on time delay |
US11941857B2 (en) | 2020-05-26 | 2024-03-26 | Hi Llc | Systems and methods for data representation in an optical measurement system |
CN111650600B (en) * | 2020-05-28 | 2023-03-31 | 北京空间机电研究所 | Double-spectrum laser imaging device for extremely weak signals |
GB2596099A (en) * | 2020-06-17 | 2021-12-22 | Ams Sensors Asia Pte Ltd | Image sensors |
US20230258783A1 (en) * | 2020-07-31 | 2023-08-17 | Ningbo ABAX Sensing Electronic Technology Co., Ltd. | Detection apparatus and method |
CN114200479A (en) * | 2020-08-31 | 2022-03-18 | 华为技术有限公司 | dTOF sensing module, terminal equipment and distance measuring method |
US11789533B2 (en) | 2020-09-22 | 2023-10-17 | Hi Llc | Synchronization between brain interface system and extended reality system |
JP2022070645A (en) * | 2020-10-27 | 2022-05-13 | 株式会社デンソーウェーブ | Laser radar |
US11452470B2 (en) | 2021-01-06 | 2022-09-27 | Hi Llc | Devices, systems, and methods using wearable time domain-based activity tracker |
JP2022126067A (en) * | 2021-02-18 | 2022-08-30 | ソニーセミコンダクタソリューションズ株式会社 | Light receiving device, distance measuring device, and signal processing method of light receiving device |
WO2022182526A1 (en) | 2021-02-26 | 2022-09-01 | Hi Llc | Brain activity tracking during electronic gaming |
CN113514842A (en) * | 2021-03-08 | 2021-10-19 | 奥诚信息科技(上海)有限公司 | Distance measuring method, system and device |
US11543885B2 (en) | 2021-05-26 | 2023-01-03 | Hi Llc | Graphical emotion symbol determination based on brain measurement data for use during an electronic messaging session |
US20230030480A1 (en) * | 2021-07-27 | 2023-02-02 | Samsung Electronics Co., Ltd. | Spad image sensor |
JP2023122390A (en) * | 2022-02-22 | 2023-09-01 | ソニーセミコンダクタソリューションズ株式会社 | Distance measuring device and distance measuring method |
WO2023218798A1 (en) * | 2022-05-10 | 2023-11-16 | ソニーセミコンダクタソリューションズ株式会社 | Distance measurement device and counter |
JP2024023058A (en) * | 2022-08-08 | 2024-02-21 | ソニーセミコンダクタソリューションズ株式会社 | Light detection device and light detection system |
JP2024075331A (en) * | 2022-11-22 | 2024-06-03 | 株式会社デンソー | Control device, optical detection system, control method, and control program |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120162632A1 (en) * | 2010-12-22 | 2012-06-28 | Stmicroelectronics (Research & Development) Limited | Real-Time Processing Method and System |
US20120280131A1 (en) * | 2011-05-04 | 2012-11-08 | Oy Ajat Ltd. | Photon/energy identifying x-ray and gamma ray imaging device ("pid") with a two dimensional array of pixels and system therefrom |
US20130300838A1 (en) * | 2010-12-23 | 2013-11-14 | Fastree3D S.A. | Methods and devices for generating a representation of a 3d scene at very high speed |
US20140021356A1 (en) * | 2011-04-05 | 2014-01-23 | Koninklijke Philips N.V. | Detector array with time-to-digital conversion having improved temporal accuracy |
US20140217264A1 (en) * | 2011-10-31 | 2014-08-07 | The Trustees Of Columbia University In The City Of New York | Systems and methods for imaging using single photon avalanche diodes |
US20160133668A1 (en) * | 2014-08-08 | 2016-05-12 | Quantum-Si Incorporated | Integrated device for temporal binning of received photons |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2382648B (en) * | 2001-12-11 | 2003-11-12 | Amersham Pharm Biotech Uk Ltd | System and method for time correlated multi-photon counting measurements |
JP6375760B2 (en) | 2014-07-31 | 2018-08-22 | 株式会社ニデック | Optical coherence tomography apparatus and fundus image processing program |
JP6020547B2 (en) * | 2014-12-26 | 2016-11-02 | トヨタ自動車株式会社 | Image acquisition apparatus and method |
JP6481405B2 (en) * | 2015-02-17 | 2019-03-13 | 株式会社デンソー | Arithmetic unit |
US10416293B2 (en) * | 2016-12-12 | 2019-09-17 | Sensl Technologies Ltd. | Histogram readout method and circuit for determining the time of flight of a photon |
US10527728B2 (en) * | 2017-01-27 | 2020-01-07 | Samsung Electronics Co., Ltd | Apparatus and method for range measurement |
KR20230169420A (en) * | 2017-03-01 | 2023-12-15 | 아우스터, 인크. | Accurate photo detector measurements for lidar |
WO2019226487A1 (en) * | 2018-05-23 | 2019-11-28 | The Charles Stark Draper Laboratory, Inc. | Parallel photon counting |
-
2016
- 2016-12-30 GB GBGB1622429.7A patent/GB201622429D0/en not_active Ceased
-
2017
- 2017-12-28 EP EP17825598.0A patent/EP3563174A1/en active Pending
- 2017-12-28 JP JP2019535786A patent/JP7088935B2/en active Active
- 2017-12-28 WO PCT/GB2017/053900 patent/WO2018122560A1/en unknown
- 2017-12-28 CN CN201780086502.2A patent/CN110312945B/en active Active
- 2017-12-28 US US16/474,000 patent/US11460560B2/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120162632A1 (en) * | 2010-12-22 | 2012-06-28 | Stmicroelectronics (Research & Development) Limited | Real-Time Processing Method and System |
US20130300838A1 (en) * | 2010-12-23 | 2013-11-14 | Fastree3D S.A. | Methods and devices for generating a representation of a 3d scene at very high speed |
US20140021356A1 (en) * | 2011-04-05 | 2014-01-23 | Koninklijke Philips N.V. | Detector array with time-to-digital conversion having improved temporal accuracy |
US20120280131A1 (en) * | 2011-05-04 | 2012-11-08 | Oy Ajat Ltd. | Photon/energy identifying x-ray and gamma ray imaging device ("pid") with a two dimensional array of pixels and system therefrom |
US20140217264A1 (en) * | 2011-10-31 | 2014-08-07 | The Trustees Of Columbia University In The City Of New York | Systems and methods for imaging using single photon avalanche diodes |
US20160133668A1 (en) * | 2014-08-08 | 2016-05-12 | Quantum-Si Incorporated | Integrated device for temporal binning of received photons |
Cited By (71)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11415679B2 (en) | 2015-08-20 | 2022-08-16 | Apple Inc. | SPAD array with gated histogram construction |
JP2019007950A (en) * | 2017-06-22 | 2019-01-17 | 株式会社デンソー | Optical distance measuring device |
US11681027B2 (en) | 2017-06-29 | 2023-06-20 | Apple Inc. | Time-of-flight depth mapping with parallax compensation |
US11852727B2 (en) | 2017-12-18 | 2023-12-26 | Apple Inc. | Time-of-flight sensing using an addressable array of emitters |
EP3537180A1 (en) * | 2018-03-08 | 2019-09-11 | ZF Friedrichshafen AG | Receiver for receiving light pulses, lidar module and method for receiving light pulses |
DE102018203534A1 (en) * | 2018-03-08 | 2019-09-12 | Ibeo Automotive Systems GmbH | Receiver arrangement for receiving light pulses, LiDAR module and method for receiving light pulses |
US11437538B2 (en) | 2018-05-17 | 2022-09-06 | Hi Llc | Wearable brain interface systems including a headgear and a plurality of photodetector units each housing a photodetector configured to be controlled by a master control unit |
US11004998B2 (en) | 2018-05-17 | 2021-05-11 | Hi Llc | Wearable brain interface systems including a headgear and a plurality of photodetector units |
WO2020049203A1 (en) * | 2018-09-05 | 2020-03-12 | Consejo Superior De Investigaciones Científicas | Method and device for detecting the peak of the compressed histogram of pixel values in high-resolution time-of-flight sensors |
US11754686B2 (en) | 2018-11-19 | 2023-09-12 | The University Court Of The University Of Edinburgh | Digital pixel |
WO2020106661A1 (en) * | 2018-11-19 | 2020-05-28 | Sense Photonics, Inc. | Digital pixel |
EP3864436A4 (en) * | 2018-11-19 | 2022-06-15 | Sense Photonics, Inc. | Digital pixel |
CN113196105A (en) * | 2018-11-19 | 2021-07-30 | 感觉光子公司 | Digital pixel |
WO2020118017A1 (en) * | 2018-12-05 | 2020-06-11 | Sense Photonics, Inc. | Hybrid center of mass method (cmm) pixel |
US11506765B2 (en) | 2018-12-05 | 2022-11-22 | Sense Photonics, Inc. | Hybrid center of mass method (CMM) pixel |
US11903713B2 (en) | 2018-12-21 | 2024-02-20 | Hi Llc | Biofeedback for awareness and modulation of mental state using a non-invasive brain interface system and method |
US11006876B2 (en) | 2018-12-21 | 2021-05-18 | Hi Llc | Biofeedback for awareness and modulation of mental state using a non-invasive brain interface system and method |
WO2020150574A1 (en) * | 2019-01-18 | 2020-07-23 | Sense Photonics, Inc. | Digital pixels and operating methods thereof |
US11639990B2 (en) | 2019-01-18 | 2023-05-02 | The University Court Of The University Of Edinburgh | Digital pixels and operating methods thereof |
CN113748356A (en) * | 2019-01-18 | 2021-12-03 | 感觉光子公司 | Digital pixel and operation method thereof |
WO2020153272A1 (en) * | 2019-01-24 | 2020-07-30 | ソニーセミコンダクタソリューションズ株式会社 | Measuring device, ranging device, and method of measurement |
US20220066038A1 (en) * | 2019-01-24 | 2022-03-03 | Sony Semiconductor Solutions Corporation | Distance measuring device, vehicle-mounted system, and distance measuring method |
WO2020153275A1 (en) * | 2019-01-24 | 2020-07-30 | ソニーセミコンダクタソリューションズ株式会社 | Distance measurement device, in-vehicle system, and distance measurement method |
WO2020166419A1 (en) * | 2019-02-13 | 2020-08-20 | ソニーセミコンダクタソリューションズ株式会社 | Light reception device, histogram generation method, and distance measurement system |
WO2020184325A1 (en) * | 2019-03-12 | 2020-09-17 | ソニーセミコンダクタソリューションズ株式会社 | Measurement device, ranging device, and measurement method |
DE102019107115A1 (en) * | 2019-03-20 | 2020-09-24 | Valeo Schalter Und Sensoren Gmbh | LIDAR device for a vehicle and method for increasing the detection range of a corresponding LIDAR device |
CN113795737A (en) * | 2019-05-06 | 2021-12-14 | Hi有限责任公司 | Photodetector architecture for time-correlated single photon counting |
US11813041B2 (en) | 2019-05-06 | 2023-11-14 | Hi Llc | Photodetector architectures for time-correlated single photon counting |
WO2020226840A1 (en) * | 2019-05-06 | 2020-11-12 | Hi Llc | Photodetector architectures for time-correlated single photon counting |
US11081611B2 (en) | 2019-05-21 | 2021-08-03 | Hi Llc | Photodetector architectures for efficient fast-gating comprising a control system controlling a current drawn by an array of photodetectors with a single photon avalanche diode |
US11398578B2 (en) | 2019-06-06 | 2022-07-26 | Hi Llc | Photodetector systems with low-power time-to-digital converter architectures to determine an arrival time of photon at a photodetector based on event detection time window |
US10868207B1 (en) | 2019-06-06 | 2020-12-15 | Hi Llc | Photodetector systems with low-power time-to-digital converter architectures to determine an arrival time of photon at a photodetector based on event detection time window |
US11500094B2 (en) | 2019-06-10 | 2022-11-15 | Apple Inc. | Selection of pulse repetition intervals for sensing time of flight |
WO2020255759A1 (en) * | 2019-06-20 | 2020-12-24 | ソニーセミコンダクタソリューションズ株式会社 | Distance measurement device, distance measurement method, and distance measurement system |
US11555900B1 (en) | 2019-07-17 | 2023-01-17 | Apple Inc. | LiDAR system with enhanced area coverage |
CN110596723B (en) * | 2019-09-19 | 2023-03-14 | 深圳奥锐达科技有限公司 | Dynamic histogram drawing flight time distance measuring method and measuring system |
WO2021051480A1 (en) * | 2019-09-19 | 2021-03-25 | 深圳奥锐达科技有限公司 | Dynamic histogram drawing-based time of flight distance measurement method and measurement system |
CN110609293A (en) * | 2019-09-19 | 2019-12-24 | 深圳奥锐达科技有限公司 | Distance detection system and method based on flight time |
WO2021051481A1 (en) * | 2019-09-19 | 2021-03-25 | 深圳奥锐达科技有限公司 | Dynamic histogram drawing time-of-flight distance measurement method and measurement system |
CN110596724B (en) * | 2019-09-19 | 2022-07-29 | 深圳奥锐达科技有限公司 | Method and system for measuring flight time distance during dynamic histogram drawing |
CN110596722A (en) * | 2019-09-19 | 2019-12-20 | 深圳奥锐达科技有限公司 | System and method for measuring flight time distance with adjustable histogram |
CN110596723A (en) * | 2019-09-19 | 2019-12-20 | 深圳奥锐达科技有限公司 | Method and system for measuring flight time distance during dynamic histogram drawing |
CN110596722B (en) * | 2019-09-19 | 2022-10-04 | 深圳奥锐达科技有限公司 | System and method for measuring flight time distance with adjustable histogram |
WO2021051477A1 (en) * | 2019-09-19 | 2021-03-25 | 深圳奥锐达科技有限公司 | Time of flight distance measurement system and method with adjustable histogram |
CN110596724A (en) * | 2019-09-19 | 2019-12-20 | 深圳奥锐达科技有限公司 | Method and system for measuring flight time distance during dynamic histogram drawing |
US11733359B2 (en) | 2019-12-03 | 2023-08-22 | Apple Inc. | Configurable array of single-photon detectors |
US11630310B2 (en) | 2020-02-21 | 2023-04-18 | Hi Llc | Wearable devices and wearable assemblies with adjustable positioning for use in an optical measurement system |
US11771362B2 (en) | 2020-02-21 | 2023-10-03 | Hi Llc | Integrated detector assemblies for a wearable module of an optical measurement system |
US12029558B2 (en) | 2020-02-21 | 2024-07-09 | Hi Llc | Time domain-based optical measurement systems and methods configured to measure absolute properties of tissue |
US11969259B2 (en) | 2020-02-21 | 2024-04-30 | Hi Llc | Detector assemblies for a wearable module of an optical measurement system and including spring-loaded light-receiving members |
US11950879B2 (en) | 2020-02-21 | 2024-04-09 | Hi Llc | Estimation of source-detector separation in an optical measurement system |
US11096620B1 (en) | 2020-02-21 | 2021-08-24 | Hi Llc | Wearable module assemblies for an optical measurement system |
US11883181B2 (en) | 2020-02-21 | 2024-01-30 | Hi Llc | Multimodal wearable measurement systems and methods |
US11515014B2 (en) | 2020-02-21 | 2022-11-29 | Hi Llc | Methods and systems for initiating and conducting a customized computer-enabled brain research study |
US11819311B2 (en) | 2020-03-20 | 2023-11-21 | Hi Llc | Maintaining consistent photodetector sensitivity in an optical measurement system |
US11877825B2 (en) | 2020-03-20 | 2024-01-23 | Hi Llc | Device enumeration in an optical measurement system |
US11607132B2 (en) | 2020-03-20 | 2023-03-21 | Hi Llc | Temporal resolution control for temporal point spread function generation in an optical measurement system |
US11245404B2 (en) | 2020-03-20 | 2022-02-08 | Hi Llc | Phase lock loop circuit based signal generation in an optical measurement system |
WO2021188486A1 (en) * | 2020-03-20 | 2021-09-23 | Hi Llc | Phase lock loop circuit based adjustment of a measurement time window in an optical measurement system |
US11187575B2 (en) | 2020-03-20 | 2021-11-30 | Hi Llc | High density optical measurement systems with minimal number of light sources |
WO2021188487A1 (en) * | 2020-03-20 | 2021-09-23 | Hi Llc | Temporal resolution control for temporal point spread function generation in an optical measurement system |
US11857348B2 (en) | 2020-03-20 | 2024-01-02 | Hi Llc | Techniques for determining a timing uncertainty of a component of an optical measurement system |
US11864867B2 (en) | 2020-03-20 | 2024-01-09 | Hi Llc | Control circuit for a light source in an optical measurement system by applying voltage with a first polarity to start an emission of a light pulse and applying voltage with a second polarity to stop the emission of the light pulse |
US11903676B2 (en) | 2020-03-20 | 2024-02-20 | Hi Llc | Photodetector calibration of an optical measurement system |
US11645483B2 (en) | 2020-03-20 | 2023-05-09 | Hi Llc | Phase lock loop circuit based adjustment of a measurement time window in an optical measurement system |
US11885915B2 (en) | 2020-03-30 | 2024-01-30 | Stmicroelectronics (Research & Development) Limited | Time to digital converter |
EP3936947A1 (en) * | 2020-03-30 | 2022-01-12 | STMicroelectronics (Research & Development) Limited | Latched gray code for tof applications |
EP3889694A1 (en) * | 2020-03-30 | 2021-10-06 | STMicroelectronics (Research & Development) Limited | Time digital converter and tof measurement |
US20210302550A1 (en) * | 2020-03-30 | 2021-09-30 | Stmicroelectronics (Research & Development) Limited | Time to digital converter |
US11353828B2 (en) | 2020-03-30 | 2022-06-07 | Stmicroelectronics (Research & Development) Limited | Latched gray code for ToF applications |
US11681028B2 (en) | 2021-07-18 | 2023-06-20 | Apple Inc. | Close-range measurement of time of flight using parallax shift |
Also Published As
Publication number | Publication date |
---|---|
CN110312945A (en) | 2019-10-08 |
US11460560B2 (en) | 2022-10-04 |
EP3563174A1 (en) | 2019-11-06 |
GB201622429D0 (en) | 2017-02-15 |
JP7088935B2 (en) | 2022-06-21 |
CN110312945B (en) | 2023-07-04 |
US20200116838A1 (en) | 2020-04-16 |
JP2020504299A (en) | 2020-02-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11460560B2 (en) | Photon sensor apparatus | |
Perenzoni et al. | A 64$\times $64-Pixels Digital Silicon Photomultiplier Direct TOF Sensor With 100-MPhotons/s/pixel Background Rejection and Imaging/Altimeter Mode With 0.14% Precision Up To 6 km for Spacecraft Navigation and Landing | |
US10075658B2 (en) | Data digitization and display for an imaging system | |
US11754686B2 (en) | Digital pixel | |
EP3388858A1 (en) | Solid-state imaging device, distance measuring device, and distance measurement method | |
Cuccato et al. | Complete and compact 32-channel system for time-correlated single-photon counting measurements | |
Field et al. | A 100 fps, time-correlated single-photon-counting-based fluorescence-lifetime imager in 130 nm CMOS | |
Padmanabhan et al. | 7.4 A 256× 128 3D-stacked (45nm) SPAD FLASH LiDAR with 7-level coincidence detection and progressive gating for 100m range and 10klux background light | |
CN110221272B (en) | Time flight depth camera and anti-interference distance measurement method | |
Lindner et al. | A 252× 144 SPAD pixel FLASH LiDAR with 1728 dual-clock 48.8 ps TDCs, integrated histogramming and 14.9-to-1 compression in 180nm CMOS technology | |
Erdogan et al. | A 16.5 giga events/s 1024× 8 SPAD line sensor with per-pixel zoomable 50ps-6.4 ns/bin histogramming TDC | |
US11626446B2 (en) | Pixel circuit and method of operating the same in an always-on mode | |
Conca et al. | Gated SPAD arrays for single-photon time-resolved imaging and spectroscopy | |
Cominelli et al. | Readout architectures for high efficiency in time-correlated single photon counting experiments—Analysis and review | |
EP3987305A1 (en) | Direct time-of-flight depth sensor architecture and method for operating of such a sensor | |
Huang et al. | 32× 64 SPAD imager using 2-bit in-pixel stack-based memory for low-light imaging | |
CN115209118A (en) | Image color correction system and method | |
US20230078828A1 (en) | Information processing system, sensor system, information processing method, and program | |
JP2009076012A (en) | Object identification device | |
Markovic et al. | Smart-pixel for 3D ranging imagers based on single-photon avalanche diode and time-to-digital converter | |
Ülkü | Large-format time-gated SPAD cameras for real-time phasor-based FLIM | |
Tosi et al. | Low-noise CMOS SPAD arrays with in-pixel time-to-digital converters | |
US20210072360A1 (en) | Photon detecting 3d imaging sensor device | |
US20230393241A1 (en) | Systems, methods, and media for single photon depth imaging with improved efficiency using compressive histograms | |
Mos et al. | Piccolo gated: a CMOS 32x32 SPAD camera with all-solid-state nanosecond time gating and PCIe readout for single-photon time-domain DCS and near-infrared optical tomography |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17825598 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2019535786 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2017825598 Country of ref document: EP Effective date: 20190730 |