26
|
Ngo NH, Shimonomura K, Ando T, Shimura T, Watanabe H, Takehara K, Nguyen AQ, Charbon E, Etoh TG. A Pixel Design of a Branching Ultra-Highspeed Image Sensor. SENSORS 2021; 21:s21072506. [PMID: 33916733 PMCID: PMC8038384 DOI: 10.3390/s21072506] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/08/2021] [Revised: 03/18/2021] [Accepted: 03/29/2021] [Indexed: 11/16/2022]
Abstract
A burst image sensor named Hanabi, meaning fireworks in Japanese, includes a branching CCD and multiple CMOS readout circuits. The sensor is backside-illuminated with a light/charge guide pipe to minimize the temporal resolution by suppressing the horizontal motion of signal carriers. On the front side, the pixel has a guide gate at the center, branching to six first-branching gates, each bifurcating to second-branching gates, and finally connected to 12 (=6×2) floating diffusions. The signals are either read out after an image capture operation to replay 12 to 48 consecutive images, or continuously transferred to a memory chip stacked on the front side of the sensor chip and converted to digital signals. A CCD burst image sensor enables a noiseless signal transfer from a photodiode to the in-situ storage even at very high frame rates. However, the pixel count conflicts with the frame count due to the large pixel size for the relatively large in-pixel CCD memory elements. A CMOS burst image sensor can use small trench-type capacitors for memory elements, instead of CCD channels. However, the transfer noise from a floating diffusion to the memory element increases in proportion to the square root of the frame rate. The Hanabi chip overcomes the compromise between these pros and cons.
Collapse
|
27
|
Zickus V, Wu ML, Morimoto K, Kapitany V, Fatima A, Turpin A, Insall R, Whitelaw J, Machesky L, Bruschini C, Faccio D, Charbon E. Fluorescence lifetime imaging with a megapixel SPAD camera and neural network lifetime estimation. Sci Rep 2020; 10:20986. [PMID: 33268900 PMCID: PMC7710711 DOI: 10.1038/s41598-020-77737-0] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2020] [Accepted: 11/06/2020] [Indexed: 01/07/2023] Open
Abstract
Fluorescence lifetime imaging microscopy (FLIM) is a key technology that provides direct insight into cell metabolism, cell dynamics and protein activity. However, determining the lifetimes of different fluorescent proteins requires the detection of a relatively large number of photons, hence slowing down total acquisition times. Moreover, there are many cases, for example in studies of cell collectives, where wide-field imaging is desired. We report scan-less wide-field FLIM based on a 0.5 MP resolution, time-gated Single Photon Avalanche Diode (SPAD) camera, with acquisition rates up to 1 Hz. Fluorescence lifetime estimation is performed via a pre-trained artificial neural network with 1000-fold improvement in processing times compared to standard least squares fitting techniques. We utilised our system to image HT1080-human fibrosarcoma cell line as well as Convallaria. The results show promise for real-time FLIM and a viable route towards multi-megapixel fluorescence lifetime images, with a proof-of-principle mosaic image shown with 3.6 MP.
Collapse
|
28
|
Lecoq P, Morel C, Prior JO, Visvikis D, Gundacker S, Auffray E, Križan P, Turtos RM, Thers D, Charbon E, Varela J, de La Taille C, Rivetti A, Breton D, Pratte JF, Nuyts J, Surti S, Vandenberghe S, Marsden P, Parodi K, Benlloch JM, Benoit M. Roadmap toward the 10 ps time-of-flight PET challenge. Phys Med Biol 2020; 65:21RM01. [PMID: 32434156 PMCID: PMC7721485 DOI: 10.1088/1361-6560/ab9500] [Citation(s) in RCA: 62] [Impact Index Per Article: 15.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
Since the seventies, positron emission tomography (PET) has become an invaluable medical molecular imaging modality with an unprecedented sensitivity at the picomolar level, especially for cancer diagnosis and the monitoring of its response to therapy. More recently, its combination with x-ray computed tomography (CT) or magnetic resonance (MR) has added high precision anatomic information in fused PET/CT and PET/MR images, thus compensating for the modest intrinsic spatial resolution of PET. Nevertheless, a number of medical challenges call for further improvements in PET sensitivity. These concern in particular new treatment opportunities in the context personalized (also called precision) medicine, such as the need to dynamically track a small number of cells in cancer immunotherapy or stem cells for tissue repair procedures. A better signal-to-noise ratio (SNR) in the image would allow detecting smaller size tumours together with a better staging of the patients, thus increasing the chances of putting cancer in complete remission. Moreover, there is an increasing demand for reducing the radioactive doses injected to the patients without impairing image quality. There are three ways to improve PET scanner sensitivity: improving detector efficiency, increasing geometrical acceptance of the imaging device and pushing the timing performance of the detectors. Currently, some pre-localization of the electron-positron annihilation along a line-of-response (LOR) given by the detection of a pair of annihilation photons is provided by the detection of the time difference between the two photons, also known as the time-of-flight (TOF) difference of the photons, whose accuracy is given by the coincidence time resolution (CTR). A CTR of about 10 picoseconds FWHM will ultimately allow to obtain a direct 3D volume representation of the activity distribution of a positron emitting radiopharmaceutical, at the millimetre level, thus introducing a quantum leap in PET imaging and quantification and fostering more frequent use of 11C radiopharmaceuticals. The present roadmap article toward the advent of 10 ps TOF-PET addresses the status and current/future challenges along the development of TOF-PET with the objective to reach this mythic 10 ps frontier that will open the door to real-time volume imaging virtually without tomographic inversion. The medical impact and prospects to achieve this technological revolution from the detection and image reconstruction point-of-views, together with a few perspectives beyond the TOF-PET application are discussed.
Collapse
|
29
|
Jiang J, Costanzo Mata AD, Lindner S, Charbon E, Wolf M, Kalyanov A. Dynamic time domain near-infrared optical tomography based on a SPAD camera. BIOMEDICAL OPTICS EXPRESS 2020; 11:5470-5477. [PMID: 33149964 PMCID: PMC7587269 DOI: 10.1364/boe.399387] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/30/2020] [Revised: 08/17/2020] [Accepted: 08/26/2020] [Indexed: 05/03/2023]
Abstract
In many clinical applications it is relevant to observe dynamic changes in oxygenation. Therefore the ability of dynamic imaging with time domain (TD) near-infrared optical tomography (NIROT) will be important. But fast imaging is a challenge. The data acquisition of our handheld TD NIROT system based on single photon avalanche diode (SPAD) camera and 11 light sources was consequently accelerated. We tested the system on a diffusive medium simulating tissue with a moving object embedded. With 3D image reconstruction, the moving object was correctly located using only 0.2 s exposure time per source.
Collapse
|
30
|
Jiang J, Costanzo Mata AD, Lindner S, Zhang C, Charbon E, Wolf M, Kalyanov A. Image reconstruction for novel time domain near infrared optical tomography: towards clinical applications. BIOMEDICAL OPTICS EXPRESS 2020; 11:4723-4734. [PMID: 32923074 PMCID: PMC7449738 DOI: 10.1364/boe.398885] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/05/2020] [Revised: 07/17/2020] [Accepted: 07/17/2020] [Indexed: 05/04/2023]
Abstract
Near infrared optical tomography (NIROT) is an emerging modality that enables imaging the oxygenation of tissue, which is a biomarker of tremendous clinical relevance. Measuring in reflectance is usually required when NIROT is applied in clinical scenarios. Single photon avalanche diode (SPAD) array technology provides a compact solution for time domain (TD) NIROT to gain huge temporal and spatial information. This makes it possible to image complex structures in tissue. The main aim of this paper is to validate the wavelength normalization method for our new TD NIROT experimentally by exposing it to a particularly difficult challenge: the recovery of two inclusions at different depths. The proposed reconstruction algorithm aims to tackle systematic errors and other artifacts with known wavelength-dependent relation. We validated the device and reconstruction method experimentally on a silicone phantom with two inclusions: one at depth of 10 mm and the other at 15 mm. Despite this tough challenge for reflectance NIROT, the system was able to localize both inclusions accurately.
Collapse
|
31
|
Morimoto K, Charbon E. High fill-factor miniaturized SPAD arrays with a guard-ring-sharing technique. OPTICS EXPRESS 2020; 28:13068-13080. [PMID: 32403788 DOI: 10.1364/oe.389216] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/07/2020] [Accepted: 03/30/2020] [Indexed: 06/11/2023]
Abstract
We present a novel guard-ring-sharing technique to push the limit of SPAD pixel miniaturization, and to demonstrate the operation of SPAD arrays with a 2.2 µm-pitch, the smallest ever reported. Device simulation and preliminary tests suggest that the optimized device design ensures the electrical isolation of SPADs with guard-ring sharing. 4×4 SPAD arrays with two parallel selective readout circuits are designed in 180 nm CMOS technology. SPAD characteristics for the pixel pitch of 2.2, 3, and 4 µm are systematically measured as a function of an active diameter, active-to-active distance, and excess bias. For a 4 µm-pitch, the fill factor is 42.4%, the maximum PDP 33.5%, the median DCR 2.5 cps, the timing jitter 88 ps, and the crosstalk probability is 3.57%, while the afterpulsing probability is 0.21%. Finally, we verified the feasibility of the proposed technique towards compact multi-megapixel 3D-stacked SPAD arrays.
Collapse
|
32
|
Ren W, Jiang J, Costanzo Mata AD, Kalyanov A, Ripoll J, Lindner S, Charbon E, Zhang C, Rudin M, Wolf M. Multimodal imaging combining time-domain near-infrared optical tomography and continuous-wave fluorescence molecular tomography. OPTICS EXPRESS 2020; 28:9860-9874. [PMID: 32225585 DOI: 10.1364/oe.385392] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/16/2019] [Accepted: 03/02/2020] [Indexed: 06/10/2023]
Abstract
Fluorescence molecular tomography (FMT) emerges as a powerful non-invasive imaging tool with the ability to resolve fluorescence signals from sources located deep in living tissues. Yet, the accuracy of FMT reconstruction depends on the deviation of the assumed optical properties from the actual values. In this work, we improved the accuracy of the initial optical properties required for FMT using a new-generation time-domain (TD) near-infrared optical tomography (NIROT) system, which effectively decouples scattering and absorption coefficients. We proposed a multimodal paradigm combining TD-NIROT and continuous-wave (CW) FMT. Both numerical simulation and experiments were performed on a heterogeneous phantom containing a fluorescent inclusion. The results demonstrate significant improvement in the FMT reconstruction by taking the NIROT-derived optical properties as prior information. The multimodal method is attractive for preclinical studies and tumor diagnostics since both functional and molecular information can be obtained.
Collapse
|
33
|
Ulku A, Ardelean A, Antolovic M, Weiss S, Charbon E, Bruschini C, Michalet X. Wide-field time-gated SPAD imager for phasor-based FLIM applications. Methods Appl Fluoresc 2020; 8:024002. [PMID: 31968310 PMCID: PMC8827132 DOI: 10.1088/2050-6120/ab6ed7] [Citation(s) in RCA: 26] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
We describe the performance of a new wide area time-gated single-photon
avalanche diode (SPAD) array for phasor-FLIM, exploring the effect of gate
length, gate number and signal intensity on the measured lifetime accuracy and
precision. We conclude that the detector functions essentially as an ideal shot
noise limited sensor and is capable of video rate FLIM measurement. The phasor
approach used in this work appears ideally suited to handle the large amount of
data generated by this type of very large sensor (512 × 512 pixels), even
in the case of small number of gates and limited photon budget.
Collapse
|
34
|
Bruschini C, Homulle H, Antolovic IM, Burri S, Charbon E. Erratum: Author Correction: Single-photon avalanche diode imagers in biophotonics: review and outlook. LIGHT, SCIENCE & APPLICATIONS 2020; 9:12. [PMID: 32025295 PMCID: PMC6987173 DOI: 10.1038/s41377-020-0248-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
[This corrects the article DOI: 10.1038/s41377-019-0191-5.].
Collapse
|
35
|
Ankri R, Basu A, Ulku AC, Bruschini C, Charbon E, Weiss S, Michalet X. Single-Photon, Time-Gated, Phasor-Based Fluorescence Lifetime Imaging through Highly Scattering Medium. ACS PHOTONICS 2020; 7:68-79. [PMID: 35936550 PMCID: PMC9355389 DOI: 10.1021/acsphotonics.9b00874] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/22/2023]
Abstract
Fluorescence lifetime imaging (FLI) is increasingly recognized as a powerful tool for biochemical and cellular investigations, including in vivo applications. Fluorescence lifetime is an intrinsic characteristic of any fluorescent dye which, to a large extent, does not depend on excitation intensity and signal level. In particular, it allows distinguishing dyes with similar emission spectra, offering additional multiplexing capabilities. However, in vivo FLI in the visible range is complicated by the contamination by (i) tissue autofluorescence, which decreases contrast, and by (ii) light scattering and absorption in tissues, which significantly reduce fluorescence intensity and modify the temporal profile of the signal. Here, we demonstrate how these issues can be accounted for and overcome, using a new time-gated single-photon avalanche diode array camera, SwissSPAD2, combined with phasor analysis to provide a simple and fast visual method for lifetime imaging. In particular, we show how phasor dispersion increases with increasing scattering and/or decreasing fluorescence intensity. Next, we show that as long as the fluorescence signal of interest is larger than the phantom autofluorescence, the presence of a distinct lifetime can be clearly identified with appropriate background correction. We use these results to demonstrate the detection of A459 cells expressing the fluorescent protein mCyRFP1 through highly scattering and autofluorescent phantom layers. These results showcase the possibility to perform FLI in challenging conditions, using standard, bright, visible fluorophore or fluorescence proteins.
Collapse
|
36
|
Padmanabhan P, Zhang C, Charbon E. Modeling and Analysis of a Direct Time-of-Flight Sensor Architecture for LiDAR Applications. SENSORS 2019; 19:s19245464. [PMID: 31835807 PMCID: PMC6960641 DOI: 10.3390/s19245464] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/31/2019] [Revised: 11/29/2019] [Accepted: 12/05/2019] [Indexed: 11/29/2022]
Abstract
Direct time-of-flight (DTOF) is a prominent depth sensing method in light detection and ranging (LiDAR) applications. Single-photon avalanche diode (SPAD) arrays integrated in DTOF sensors have demonstrated excellent ranging and 3D imaging capabilities, making them promising candidates for LiDARs. However, high background noise due to solar exposure limits their performance and degrades the signal-to-background noise ratio (SBR). Noise-filtering techniques based on coincidence detection and time-gating have been implemented to mitigate this challenge but 3D imaging of a wide dynamic range scene is an ongoing issue. In this paper, we propose a coincidence-based DTOF sensor architecture to address the aforementioned challenges. The architecture is analyzed using a probabilistic model and simulation. A flash LiDAR setup is simulated with typical operating conditions of a wide angle field-of-view (FOV = 40°) in a 50 klux ambient light assumption. Single-point ranging simulations are obtained for distances up to 150 m using the DTOF model. An activity-dependent coincidence is proposed as a way to improve imaging of wide dynamic range targets. An example scene with targets ranging between 8–60% reflectivity is used to simulate the proposed method. The model predicts that a single threshold cannot yield an accurate reconstruction and a higher (lower) reflective target requires a higher (lower) coincidence threshold. Further, a pixel-clustering scheme is introduced, capable of providing multiple simultaneous timing information as a means to enhance throughput and reduce timing uncertainty. Example scenes are reconstructed to distinguish up to 4 distinct target peaks simulated with a resolution of 500 ps. Alternatively, a time-gating mode is simulated where in the DTOF sensor performs target-selective ranging. Simulation results show reconstruction of a 10% reflective target at 20 m in the presence of a retro-reflective equivalent with a 60% reflectivity at 5 m within the same FOV.
Collapse
|
37
|
Lubin G, Tenne R, Michel Antolovic I, Charbon E, Bruschini C, Oron D. Quantum correlation measurement with single photon avalanche diode arrays. OPTICS EXPRESS 2019; 27:32863-32882. [PMID: 31878363 DOI: 10.1364/oe.27.032863] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/24/2019] [Accepted: 10/03/2019] [Indexed: 05/20/2023]
Abstract
Temporal photon correlation measurement, instrumental to probing the quantum properties of light, typically requires multiple single photon detectors. Progress in single photon avalanche diode (SPAD) array technology highlights their potential as high-performance detector arrays for quantum imaging and photon number-resolving (PNR) experiments. Here, we demonstrate this potential by incorporating a novel on-chip SPAD array with 42% peak photon detection efficiency, low dark count rate and crosstalk probability of 0.14% per detection in a confocal microscope. This enables reliable measurements of second and third order photon correlations from a single quantum dot emitter. Our analysis overcomes the inter-detector optical crosstalk background even though it is over an order of magnitude larger than our faint signal. To showcase the vast application space of such an approach, we implement a recently introduced super-resolution imaging method, quantum image scanning microscopy (Q-ISM).
Collapse
|
38
|
Bruschini C, Homulle H, Antolovic IM, Burri S, Charbon E. Single-photon avalanche diode imagers in biophotonics: review and outlook. LIGHT, SCIENCE & APPLICATIONS 2019; 8:87. [PMID: 31645931 PMCID: PMC6804596 DOI: 10.1038/s41377-019-0191-5] [Citation(s) in RCA: 118] [Impact Index Per Article: 23.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/28/2018] [Revised: 07/30/2019] [Accepted: 08/07/2019] [Indexed: 05/08/2023]
Abstract
Single-photon avalanche diode (SPAD) arrays are solid-state detectors that offer imaging capabilities at the level of individual photons, with unparalleled photon counting and time-resolved performance. This fascinating technology has progressed at a very fast pace in the past 15 years, since its inception in standard CMOS technology in 2003. A host of architectures have been investigated, ranging from simpler implementations, based solely on off-chip data processing, to progressively "smarter" sensors including on-chip, or even pixel level, time-stamping and processing capabilities. As the technology has matured, a range of biophotonics applications have been explored, including (endoscopic) FLIM, (multibeam multiphoton) FLIM-FRET, SPIM-FCS, super-resolution microscopy, time-resolved Raman spectroscopy, NIROT and PET. We will review some representative sensors and their corresponding applications, including the most relevant challenges faced by chip designers and end-users. Finally, we will provide an outlook on the future of this fascinating technology.
Collapse
|
39
|
Venialgo E, Lusardi N, Garzetti F, Geraci A, Brunner SE, Schaart DR, Charbon E. Toward a Full-Flexible and Fast-Prototyping TOF-PET Block Detector Based on TDC-on-FPGA. IEEE TRANSACTIONS ON RADIATION AND PLASMA MEDICAL SCIENCES 2019. [DOI: 10.1109/trpms.2018.2874358] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
40
|
Etoh TG, Okinaka T, Takano Y, Takehara K, Nakano H, Shimonomura K, Ando T, Ngo N, Kamakura Y, Dao VTS, Nguyen AQ, Charbon E, Zhang C, De Moor P, Goetschalckx P, Haspeslagh L. Light-In-Flight Imaging by a Silicon Image Sensor: Toward the Theoretical Highest Frame Rate. SENSORS 2019; 19:s19102247. [PMID: 31096653 PMCID: PMC6567881 DOI: 10.3390/s19102247] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/28/2019] [Revised: 05/06/2019] [Accepted: 05/10/2019] [Indexed: 11/25/2022]
Abstract
Light in flight was captured by a single shot of a newly developed backside-illuminated multi-collection-gate image sensor at a frame interval of 10 ns without high-speed gating devices such as a streak camera or post data processes. This paper reports the achievement and further evolution of the image sensor toward the theoretical temporal resolution limit of 11.1 ps derived by the authors. The theoretical analysis revealed the conditions to minimize the temporal resolution. Simulations show that the image sensor designed following the specified conditions and fabricated by existing technology will achieve a frame interval of 50 ps. The sensor, 200 times faster than our latest sensor will innovate advanced analytical apparatuses using time-of-flight or lifetime measurements, such as imaging TOF-MS, FLIM, pulse neutron tomography, PET, LIDAR, and more, beyond these known applications.
Collapse
|
41
|
Ulku AC, Bruschini C, Antolovic IM, Weiss S, Michalet X, Charbon E. Phasor-based widefield FLIM using a gated 512×512 single-photon SPAD imager. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2019; 10882. [PMID: 33859449 DOI: 10.1117/12.2511148] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
Single-photon avalanche diode (SPAD) imagers can perform fast time-resolved imaging in a compact form factor, by exploiting the processing capability and speed of integrated CMOS electronics. Developments in SPAD imagers have recently made them compatible with widefield microscopy, thanks to array formats approaching one megapixel and sensitivity and noise levels approaching those of established technologies. In this paper, phasor-based FLIM is demonstrated with a gated binary 512×512 SPAD imager, which can operate with a gate length as short as 5.75 ns, a minimum gate step of 17.9 ps, and up to 98 kfps readout rate (1-bit frames). Lifetimes of ATTO 550 and Rhodamine 6G (R6G) solutions were measured across a 472×256 sub-array using phasor analysis, acquiring data by shifting a 13.1 ns gate window across the 50 ns laser period. The measurement accuracy obtained when employing such a scheme based on long, overlapping gates was validated by comparison with TCSPC measurements and fitting analysis results based on a standard Levenberg-Marquardt algorithm (>90% accuracy for the lifetime of R6G and ATTO 550). This demonstrates the ability of the proposed method to measure short lifetimes without minimum gate length requirements. The FLIM frame rate of the overall system can be increased up to a few fps for phasor-based widefield FLIM (moving closer to real-time operation) by FPGA-based parallel computation with continuous acquisition at the full speed of 98 kfps.
Collapse
|
42
|
Ardelean A, Ulku AC, Michalet X, Charbon E, Bruschini C. Fluorescence lifetime imaging with a single-photon SPAD array using long overlapping gates: an experimental and theoretical study. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2019; 10882:108820Y. [PMID: 33833477 PMCID: PMC8026147 DOI: 10.1117/12.2511287] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Developing large arrays of single-photon avalanche diodes (SPADs) with on-chip time-correlated single-photon counting (TCSPC) capabilities continues to be a difficult task due to stringent silicon real estate constraints, high data rates and system complexity. As an alternative to TCSPC, time-gated architectures have been proposed, where the numbers of photons detected within different time gates are used as a replacement to the usual time-resolved luminescence decay. However, because of technological limitations, the minimum gate length implement is on the order of nanoseconds, longer than most fluorophore lifetimes of interest. However, recent FLIM measurements have shown that it is mainly the gate step and rise/fall time, rather than its length, which determine lifetime resolution. In addition, the large number of photons captured by longer gates results in higher SNR. In this paper, we study the effects of using long, overlapping gates on lifetime extraction by phasor analysis, using a recently developed 512×512 time-gated SPAD array. The experiments used Cy3B, Rhodamine 6G and Atto550 dyes as test samples. The gate window length was varied between 11.3 ns and 23 ns while the gate step was varied between 17.86 ps and 3 ns. We validated the results with a standard TCSPC setup and investigated the case of multi-exponential samples through simulations. Results indicate that lifetime extraction is not degraded by the use of longer gates, nor is the ability to resolve multi-exponential decays.
Collapse
|
43
|
Ulku AC, Bruschini C, Antolovic IM, Charbon E, Kuo Y, Ankri R, Weiss S, Michalet X. A 512×512 SPAD Image Sensor with Integrated Gating for Widefield FLIM. IEEE JOURNAL OF SELECTED TOPICS IN QUANTUM ELECTRONICS : A PUBLICATION OF THE IEEE LASERS AND ELECTRO-OPTICS SOCIETY 2019; 25:6801212. [PMID: 31156324 PMCID: PMC6541425 DOI: 10.1109/jstqe.2018.2867439] [Citation(s) in RCA: 45] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/18/2023]
Abstract
We report on SwissSPAD2, an image sensor with 512×512 photon-counting pixels, each comprising a single-photon avalanche diode (SPAD), a 1-bit memory, and a gating mechanism capable of turning the SPAD on and off, with a skew of 250ps and 344ps, respectively, for a minimum duration of 5.75ns. The sensor is designed to achieve a frame rate of up to 97,700 binary frames per second and sub-40ps gate shifts. By synchronizing it with a pulsed laser and using multiple successive overlapping gates, one can reconstruct a molecule's fluorescent response with picosecond temporal resolution. Thanks to the sensor's number of pixels (the largest to date) and the fully integrated gated operation, SwissSPAD2 enables widefield FLIM with an all-solid-state solution and at relatively high frame rates. This was demonstrated with preliminary results on organic dyes and semiconductor quantum dots using both decay fitting and phasor analysis. Furthermore, pixels with an exceptionally low dark count rate and high photon detection probability enable uniform and high quality imaging of biologically relevant fluorescent samples stained with multiple dyes. While future versions will feature the addition of microlenses and optimize firmware speed, our results open the way to low-cost alternatives to commercially available scientific time-resolved imagers.
Collapse
|
44
|
Zhang C, Lindner S, Antolovic IM, Wolf M, Charbon E. A CMOS SPAD Imager with Collision Detection and 128 Dynamically Reallocating TDCs for Single-Photon Counting and 3D Time-of-Flight Imaging. SENSORS 2018; 18:s18114016. [PMID: 30453648 PMCID: PMC6263909 DOI: 10.3390/s18114016] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/05/2018] [Revised: 11/14/2018] [Accepted: 11/15/2018] [Indexed: 11/16/2022]
Abstract
Per-pixel time-to-digital converter (TDC) architectures have been exploited by single-photon avalanche diode (SPAD) sensors to achieve high photon throughput, but at the expense of fill factor, pixel pitch and readout efficiency. In contrast, TDC sharing architecture usually features high fill factor at small pixel pitch and energy efficient event-driven readout. While the photon throughput is not necessarily lower than that of per-pixel TDC architectures, since the throughput is not only decided by the TDC number but also the readout bandwidth. In this paper, a SPAD sensor with 32 × 32 pixels fabricated with a 180 nm CMOS image sensor technology is presented, where dynamically reallocating TDCs were implemented to achieve the same photon throughput as that of per-pixel TDCs. Each 4 TDCs are shared by 32 pixels via a collision detection bus, which enables a fill factor of 28% with a pixel pitch of 28.5 μm. The TDCs were characterized, obtaining the peak-to-peak differential and integral non-linearity of -0.07/+0.08 LSB and -0.38/+0.75 LSB, respectively. The sensor was demonstrated in a scanning light-detection-and-ranging (LiDAR) system equipped with an ultra-low power laser, achieving depth imaging up to 10 m at 6 frames/s with a resolution of 64 × 64 with 50 lux background light.
Collapse
|
45
|
Bruschini C, Veerappan C, Gramuglia F, Bijwaard M, Papp Z, Charbon E. A Sensor Network Architecture for Digital SiPM-Based PET Systems. IEEE TRANSACTIONS ON RADIATION AND PLASMA MEDICAL SCIENCES 2018. [DOI: 10.1109/trpms.2018.2866953] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
46
|
Antolovic IM, Bruschini C, Charbon E. Dynamic range extension for photon counting arrays. OPTICS EXPRESS 2018; 26:22234-22248. [PMID: 30130919 DOI: 10.1364/oe.26.022234] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/13/2018] [Accepted: 06/27/2018] [Indexed: 05/20/2023]
Abstract
Confocal microscopes use photomultiplier tubes and hybrid detectors due to their large dynamic range, which typically exceeds the one of single-photon avalanche diodes (SPADs). The latter, due to their photon counting operation, are usually limited to an output count rate to 1/Tdead. In this paper, we present a thorough analysis, which can actually be applied to any photon counting detector, on how to extend the SPAD dynamic range by exploiting the nonlinear photon response at high count rates and for different recharge mechanisms. We applied passive, active event-driven and clock-driven (i.e. clocked, following quanta image sensor response) recharge directly to the SPADs. The photon response, photon count standard deviation, signal-to-noise ratio and dynamic range were measured and compared to models. Measurements were performed with a CMOS SPAD array targeted for image scanning microscopy, featuring best-in-class 11 V excess bias, 55% peak photon detection probability at 520 nm and >40% from 440 to 640 nm. The array features an extremely low median dark count rate below 0.05 cps/μm2 at 9 V of excess bias and 0°C. We show that active event-driven recharge provides ×75 dynamic range extension and offers novel ways for high dynamic range imaging. When compared to the clock-driven recharge and the quanta image sensor approach, the dynamic range is extended by a factor of ×12.7-26.4. Additionally, for the first time, we evaluate the influence of clock-driven recharge on the SPAD afterpulsing.
Collapse
|
47
|
Buchholz J, Krieger J, Bruschini C, Burri S, Ardelean A, Charbon E, Langowski J. Widefield High Frame Rate Single-Photon SPAD Imagers for SPIM-FCS. Biophys J 2018; 114:2455-2464. [PMID: 29753448 DOI: 10.1016/j.bpj.2018.04.029] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2017] [Revised: 04/06/2018] [Accepted: 04/13/2018] [Indexed: 11/24/2022] Open
Abstract
Photon-counting sensors based on standard complementary metal-oxide-semiconductor single-photon avalanche diodes (SPADs) represent an emerging class of imagers that enable the counting and/or timing of single photons at zero readout noise (better than high-speed electron-multiplying charge-coupling devices) and over large arrays. They have seen substantial progress over the last 15 years, increasing their spatial resolution, timing accuracy, and sensitivity while reducing spurious signals such as afterpulsing and dark counts. They are increasingly being applied for time-resolved applications with the added advantage of enabling real-time options such as autocorrelation. We report in this study on the use of such a state-of-the-art 512 × 128 SPAD array, capable of a time resolution of 10-5-10-6 s for full frames while retaining acceptable photosensitivity thanks to the use of dedicated microlenses, in a selective plane illumination-fluorescence correlation spectroscopy setup. The latter allows us to perform thousands of fluorescence-correlation spectroscopy measurements simultaneously in a two-dimensional slice of the sample. This high-speed SPAD imager enables the measurement of molecular motion of small fluorescent particles such as single chemical dye molecules. Inhomogeneities in the molecular detection efficiency were compensated for by means of a global fit of the auto- and cross-correlation curves, which also made a calibration-free measurement of various samples possible. The afterpulsing effect could also be mitigated, making the measurement of the diffusion of Alexa-488 possible, and the overall result quality was further improved by spatial binning. The particle concentrations in the focus tend to be overestimated by a factor of 1.7 compared to a confocal setup; a calibration is thus required if absolute concentrations need to be measured. The first high-speed selective plane illumination-fluorescence correlation spectroscopy in vivo measurements to our knowledge were also recorded: although two-component fit models could not be employed because of noise, the diffusion of eGFP oligomers in HeLa cells could be measured. Sensitivity and noise will be further improved in the next generation of SPAD-based widefield sensors, which are currently under testing.
Collapse
|
48
|
Homulle H, Visser S, Patra B, Charbon E. Design techniques for a stable operation of cryogenic field-programmable gate arrays. THE REVIEW OF SCIENTIFIC INSTRUMENTS 2018; 89:014703. [PMID: 29390695 DOI: 10.1063/1.5004484] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/13/2017] [Accepted: 12/09/2017] [Indexed: 06/07/2023]
Abstract
In this paper, we show how a deep-submicron field-programmable gate array (FPGA) can be operated more stably at extremely low temperatures through special firmware design techniques. Stability at low temperatures is limited through long power supply wires and reduced performance of various printed circuit board components commonly employed at room temperature. Extensive characterization of these components shows that the majority of decoupling capacitor types and voltage regulators are not well behaved at cryogenic temperatures, asking for an ad hoc solution to stabilize the FPGA supply voltage, especially for sensitive applications. Therefore, we have designed a firmware that enforces a constant power consumption, so as to stabilize the supply voltage in the interior of the FPGA. The FPGA is powered with a supply at several meters distance, causing significant resistive voltage drop and thus fluctuations on the local supply voltage. To achieve the stabilization, the variation in digital logic speed, which directly corresponds to changes in supply voltage, is constantly measured and corrected for through a tunable oscillator farm, implemented on the FPGA. The impact of the stabilization technique is demonstrated together with a reconfigurable analog-to-digital converter (ADC), completely implemented in the FPGA fabric and operating at 15 K. The ADC performance can be improved by at most 1.5 bits (effective number of bits) thanks to the more stable supply voltage. The method is versatile and robust, enabling seamless porting to other FPGA families and configurations.
Collapse
|
49
|
Homulle H, Visser S, Patra B, Ferrari G, Prati E, Sebastiano F, Charbon E. A reconfigurable cryogenic platform for the classical control of quantum processors. THE REVIEW OF SCIENTIFIC INSTRUMENTS 2017; 88:045103. [PMID: 28456245 DOI: 10.1063/1.4979611] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
The implementation of a classical control infrastructure for large-scale quantum computers is challenging due to the need for integration and processing time, which is constrained by coherence time. We propose a cryogenic reconfigurable platform as the heart of the control infrastructure implementing the digital error-correction control loop. The platform is implemented on a field-programmable gate array (FPGA) that supports the functionality required by several qubit technologies and that can operate close to the physical qubits over a temperature range from 4 K to 300 K. This work focuses on the extensive characterization of the electronic platform over this temperature range. All major FPGA building blocks (such as look-up tables (LUTs), carry chains (CARRY4), mixed-mode clock manager (MMCM), phase-locked loop (PLL), block random access memory, and IDELAY2 (programmable delay element)) operate correctly and the logic speed is very stable. The logic speed of LUTs and CARRY4 changes less then 5%, whereas the jitter of MMCM and PLL clock managers is reduced by 20%. The stability is finally demonstrated by operating an integrated 1.2 GSa/s analog-to-digital converter (ADC) with a relatively stable performance over temperature. The ADCs effective number of bits drops from 6 to 4.5 bits when operating at 15 K.
Collapse
|
50
|
Nikzad S, Hoenk M, Jewell AD, Hennessy JJ, Carver AG, Jones TJ, Goodsall TM, Hamden ET, Suvarna P, Bulmer J, Shahedipour-Sandvik F, Charbon E, Padmanabhan P, Hancock B, Bell LD. Single Photon Counting UV Solar-Blind Detectors Using Silicon and III-Nitride Materials. SENSORS 2016; 16:s16060927. [PMID: 27338399 PMCID: PMC4934352 DOI: 10.3390/s16060927] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/31/2016] [Revised: 06/05/2016] [Accepted: 06/07/2016] [Indexed: 11/16/2022]
Abstract
Ultraviolet (UV) studies in astronomy, cosmology, planetary studies, biological and medical applications often require precision detection of faint objects and in many cases require photon-counting detection. We present an overview of two approaches for achieving photon counting in the UV. The first approach involves UV enhancement of photon-counting silicon detectors, including electron multiplying charge-coupled devices and avalanche photodiodes. The approach used here employs molecular beam epitaxy for delta doping and superlattice doping for surface passivation and high UV quantum efficiency. Additional UV enhancements include antireflection (AR) and solar-blind UV bandpass coatings prepared by atomic layer deposition. Quantum efficiency (QE) measurements show QE > 50% in the 100–300 nm range for detectors with simple AR coatings, and QE ≅ 80% at ~206 nm has been shown when more complex AR coatings are used. The second approach is based on avalanche photodiodes in III-nitride materials with high QE and intrinsic solar blindness.
Collapse
|