1
|
Lee J, Ingle A, Chacko JV, Eliceiri KW, Gupta M. CASPI: collaborative photon processing for active single-photon imaging. Nat Commun 2023; 14:3158. [PMID: 37258509 DOI: 10.1038/s41467-023-38893-9] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Accepted: 05/17/2023] [Indexed: 06/02/2023] Open
Abstract
Image sensors capable of capturing individual photons have made tremendous progress in recent years. However, this technology faces a major limitation. Because they capture scene information at the individual photon level, the raw data is sparse and noisy. Here we propose CASPI: Collaborative Photon Processing for Active Single-Photon Imaging, a technology-agnostic, application-agnostic, and training-free photon processing pipeline for emerging high-resolution single-photon cameras. By collaboratively exploiting both local and non-local correlations in the spatio-temporal photon data cubes, CASPI estimates scene properties reliably even under very challenging lighting conditions. We demonstrate the versatility of CASPI with two applications: LiDAR imaging over a wide range of photon flux levels, from a sub-photon to high ambient regimes, and live-cell autofluorescence FLIM in low photon count regimes. We envision CASPI as a basic building block of general-purpose photon processing units that will be implemented on-chip in future single-photon cameras.
Collapse
Affiliation(s)
- Jongho Lee
- Department of Computer Sciences, University of Wisconsin-Madison, Madison, WI, USA.
| | - Atul Ingle
- Department of Computer Science, Portland State University, Portland, OR, USA
| | - Jenu V Chacko
- Laboratory for Optical and Computational Instrumentation, University of Wisconsin-Madison, Madison, WI, USA
- Center for Quantitative Cell Imaging, University of Wisconsin-Madison, Madison, WI, USA
| | - Kevin W Eliceiri
- Laboratory for Optical and Computational Instrumentation, University of Wisconsin-Madison, Madison, WI, USA
- Center for Quantitative Cell Imaging, University of Wisconsin-Madison, Madison, WI, USA
- Morgridge Institute for Research, Madison, WI, USA
- Department of Biomedical Engineering, University of Wisconsin-Madison, Madison, WI, USA
- Department of Medical Physics, University of Wisconsin-Madison, Madison, WI, USA
- McPherson Eye Research Institute, Madison, WI, USA
| | - Mohit Gupta
- Department of Computer Sciences, University of Wisconsin-Madison, Madison, WI, USA
- McPherson Eye Research Institute, Madison, WI, USA
| |
Collapse
|
2
|
Bowman AJ, Kasevich MA. Resonant Electro-Optic Imaging for Microscopy at Nanosecond Resolution. ACS NANO 2021; 15:16043-16054. [PMID: 34546704 DOI: 10.1021/acsnano.1c04470] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
We demonstrate an electro-optic wide-field method to enable fluorescence lifetime microscopy (FLIM) with high throughput and single-molecule sensitivity. Resonantly driven Pockels cells are used to efficiently gate images at 39 MHz, allowing fluorescence lifetime to be captured on standard camera sensors. Lifetime imaging of single molecules is enabled in wide field with exposure times of less than 100 ms. This capability allows combination of wide-field FLIM with single-molecule super-resolution localization microscopy. Fast single-molecule dynamics such as FRET and molecular binding events are captured from wide-field images without prior spatial knowledge. A lifetime sensitivity of 1.9 times the photon shot-noise limit is achieved, and high throughput is shown by acquiring wide-field FLIM images with millisecond exposure and >108 photons per frame. Resonant electro-optic FLIM allows lifetime contrast in any wide-field microscopy method.
Collapse
Affiliation(s)
- Adam J Bowman
- Physics Department, Stanford University, 382 Via Pueblo Mall, Stanford, California 94305, United States
| | - Mark A Kasevich
- Physics Department, Stanford University, 382 Via Pueblo Mall, Stanford, California 94305, United States
| |
Collapse
|