1
|
Compressed sensing in fluorescence microscopy. PROGRESS IN BIOPHYSICS AND MOLECULAR BIOLOGY 2022; 168:66-80. [PMID: 34153330 DOI: 10.1016/j.pbiomolbio.2021.06.004] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/05/2021] [Revised: 05/29/2021] [Accepted: 06/07/2021] [Indexed: 12/30/2022]
Abstract
Compressed sensing (CS) is a signal processing approach that solves ill-posed inverse problems, from under-sampled data with respect to the Nyquist criterium. CS exploits sparsity constraints based on the knowledge of prior information, relative to the structure of the object in the spatial or other domains. It is commonly used in image and video compression as well as in scientific and medical applications, including computed tomography and magnetic resonance imaging. In the field of fluorescence microscopy, it has been demonstrated to be valuable for fast and high-resolution imaging, from single-molecule localization, super-resolution to light-sheet microscopy. Furthermore, CS has found remarkable applications in the field of mesoscopic imaging, facilitating the study of small animals' organs and entire organisms. This review article illustrates the working principles of CS, its implementations in optical imaging and discusses several relevant uses of CS in the field of fluorescence imaging from super-resolution microscopy to mesoscopy.
Collapse
|
2
|
Durst ME, Yurak S, Moscatelli J, Linhares I, Vargas R. Remote Focusing in a Temporal Focusing Microscope. OSA CONTINUUM 2021; 4:2757-2770. [PMID: 35531308 PMCID: PMC9075704 DOI: 10.1364/osac.443116] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/13/2021] [Accepted: 10/18/2021] [Indexed: 06/14/2023]
Abstract
In a temporal focusing microscope, dispersion can remotely shift the temporal focal plane axially, but only a single depth can be in focus at a time on a fixed camera. In this paper, we demonstrate remote focusing in a temporal focusing microscope. Dispersion tuning with an electrically tunable lens (ETL) in a 4 f pulse shaper scans the excitation plane axially, and another ETL in the detection path keeps the shifted excitation plane in focus on the camera. Image stacks formed using two ETLs versus a traditional stage scan are equivalent.
Collapse
|
3
|
Zheng C, Park JK, Yildirim M, Boivin JR, Xue Y, Sur M, So PTC, Wadduwage DN. De-scattering with Excitation Patterning enables rapid wide-field imaging through scattering media. SCIENCE ADVANCES 2021; 7:eaay5496. [PMID: 34233883 PMCID: PMC8262816 DOI: 10.1126/sciadv.aay5496] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/27/2019] [Accepted: 05/24/2021] [Indexed: 05/04/2023]
Abstract
Nonlinear optical microscopy has enabled in vivo deep tissue imaging on the millimeter scale. A key unmet challenge is its limited throughput especially compared to rapid wide-field modalities that are used ubiquitously in thin specimens. Wide-field imaging methods in tissue specimens have found successes in optically cleared tissues and at shallower depths, but the scattering of emission photons in thick turbid samples severely degrades image quality at the camera. To address this challenge, we introduce a novel technique called De-scattering with Excitation Patterning or "DEEP," which uses patterned nonlinear excitation followed by computational imaging-assisted wide-field detection. Multiphoton temporal focusing allows high-resolution excitation patterns to be projected deep inside specimen at multiple scattering lengths due to the use of long wavelength light. Computational reconstruction allows high-resolution structural features to be reconstructed from tens to hundreds of DEEP images instead of millions of point-scanning measurements.
Collapse
Affiliation(s)
- Cheng Zheng
- Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
- Laser Biomedical Research Center, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
| | - Jong Kang Park
- Laser Biomedical Research Center, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
- Department of Biological Engineering, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
- ASML, Wilton, CT 06897, USA
| | - Murat Yildirim
- Laser Biomedical Research Center, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
| | - Josiah R Boivin
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
| | - Yi Xue
- Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
- Laser Biomedical Research Center, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
- Deptartment of Electrical Engineering and Computer Sciences, University of California, Berkeley, 558 Cory Hall, Berkeley, CA, 94720, USA
| | - Mriganka Sur
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
| | - Peter T C So
- Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
- Laser Biomedical Research Center, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
- Department of Biological Engineering, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
| | - Dushan N Wadduwage
- Laser Biomedical Research Center, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA.
- Department of Biological Engineering, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
- Center for Advanced Imaging, Faculty of Arts and Sciences, Harvard University, Cambridge, MA 02138, USA
| |
Collapse
|
4
|
Single-pixel imaging of dynamic objects using multi-frame motion estimation. Sci Rep 2021; 11:7712. [PMID: 33833258 PMCID: PMC8032706 DOI: 10.1038/s41598-021-83810-z] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2020] [Accepted: 02/02/2021] [Indexed: 01/09/2023] Open
Abstract
Single-pixel imaging (SPI) enables the visualization of objects with a single detector by using a sequence of spatially modulated illumination patterns. For natural images, the number of illumination patterns may be smaller than the number of pixels when compressed-sensing algorithms are used. Nonetheless, the sequential nature of the SPI measurement requires that the object remains static until the signals from all the required patterns have been collected. In this paper, we present a new approach to SPI that enables imaging scenarios in which the imaged object, or parts thereof, moves within the imaging plane during data acquisition. Our algorithms estimate the motion direction from inter-frame cross-correlations and incorporate it in the reconstruction model. Moreover, when the illumination pattern is cyclic, the motion may be estimated directly from the raw data, further increasing the numerical efficiency of the algorithm. A demonstration of our approach is presented for both numerically simulated and measured data.
Collapse
|
5
|
Ghezzi A, Farina A, Bassi A, Valentini G, Labanca I, Acconcia G, Rech I, D'Andrea C. Multispectral compressive fluorescence lifetime imaging microscopy with a SPAD array detector. OPTICS LETTERS 2021; 46:1353-1356. [PMID: 33720185 DOI: 10.1364/ol.419381] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/08/2021] [Accepted: 02/06/2021] [Indexed: 05/22/2023]
Abstract
Multispectral/hyperspectral fluorescence lifetime imaging microscopy (λFLIM) is a promising tool for studying functional and structural biological processes. The rich information content provided by a multidimensional dataset is often in contrast with the acquisition speed. In this work, we develop and experimentally demonstrate a wide-field λFLIM setup, based on a novel time-resolved 18×1 single-photon avalanche diode array detector working in a single-pixel camera scheme, which parallelizes the spectral detection, reducing measurement time. The proposed system, which implements a single-pixel camera with a compressive sensing scheme, represents an optimal microscopy framework towards the design of λFLIM setups.
Collapse
|
6
|
ZHANG KANGNING, HU JUNJIE, YANG WEIJIAN. Deep Compressed Imaging via Optimized-Pattern Scanning. PHOTONICS RESEARCH 2021; 9:B57-B70. [PMID: 34532505 PMCID: PMC8443127 DOI: 10.1364/prj.410556] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/07/2020] [Accepted: 01/13/2021] [Indexed: 05/31/2023]
Abstract
The need for high-speed imaging in applications such as biomedicine, surveillance and consumer electronics has called for new developments of imaging systems. While the industrial effort continuously pushes the advance of silicon focal plane array image sensors, imaging through a single-pixel detector has gained significant interests thanks to the development of computational algorithms. Here, we present a new imaging modality, Deep Compressed Imaging via Optimized-Pattern Scanning (DeCIOPS), which can significantly increase the acquisition speed for a single-detector-based imaging system. We project and scan an illumination pattern across the object and collect the sampling signal with a single-pixel detector. We develop an innovative end-to-end optimized auto-encoder, using a deep neural network and compressed sensing algorithm, to optimize the illumination pattern, which allows us to reconstruct faithfully the image from a small number of samples, and with a high frame rate. Compared with the conventional switching-mask based single-pixel camera and point scanning imaging systems, our method achieves a much higher imaging speed, while retaining a similar imaging quality. We experimentally validated this imaging modality in the settings of both continuous-wave (CW) illumination and pulsed light illumination and showed high-quality image reconstructions with a high compressed sampling rate. This new compressed sensing modality could be widely applied in different imaging systems, enabling new applications which require high imaging speed.
Collapse
Affiliation(s)
- KANGNING ZHANG
- Department of Electrical and Computer Engineering, University of California, Davis, CA 95616, USA
| | - JUNJIE HU
- Department of Electrical and Computer Engineering, University of California, Davis, CA 95616, USA
| | - WEIJIAN YANG
- Department of Electrical and Computer Engineering, University of California, Davis, CA 95616, USA
| |
Collapse
|
7
|
Kabir MM, Rajput HS, Kelkar VA, Salazar Coariti AC, Toussaint KC. Demonstration of flat-top beam illumination in widefield multiphoton microscopy. JOURNAL OF BIOMEDICAL OPTICS 2019; 25:1-8. [PMID: 31729201 PMCID: PMC7008505 DOI: 10.1117/1.jbo.25.1.014503] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/09/2019] [Accepted: 10/24/2019] [Indexed: 06/10/2023]
Abstract
Multiphoton microscopy provides a suitable technique for imaging biological tissues with submicrometer resolution. Usually a Gaussian beam (GB) is used for illumination, leading to a reduced power efficiency in the multiphoton response and vignetting for a square-shaped imaging area. A flat-top beam (FTB) provides a uniform spatial intensity distribution that equalizes the probability of a multiphoton effect across the imaging area. We employ a customized widefield multiphoton microscope to compare the performance of a square-shaped FTB illumination with that based on using a GB, for both two-photon fluorescence (TPF) and second-harmonic generation (SHG) imaging. The variation in signal-to-noise ratio across TPF images of fluorescent dyes spans ∼5.6 dB for the GB and ∼1.2 dB for the FTB illumination, respectively. For the GB modality, TPF images of mouse colon and Convallaria root, and SHG images of chicken tendon and human breast biopsy tissue showcase ∼20 % area that are not imaged due to either insufficient or lack of illumination. For quantitative analysis that depends on the illuminated area, this effect can potentially lead to inaccuracies. This work emphasizes the applicability of FTB illumination to multiphoton applications.
Collapse
Affiliation(s)
- Mohammad M. Kabir
- University of Illinois at Urbana-Champaign, Department of Electrical and Computer Engineering, Urbana, Illinois, United States
- Laboratory for Photonics Research of Bio/Nano Environments (PROBE Lab), Urbana, Illinois and Providence, Rhode Island, United States
| | - Hemangg S. Rajput
- Laboratory for Photonics Research of Bio/Nano Environments (PROBE Lab), Urbana, Illinois and Providence, Rhode Island, United States
- University of Illinois at Urbana-Champaign, Department of Mechanical Science and Engineering, Urbana, Illinois, United States
| | - Varun A. Kelkar
- University of Illinois at Urbana-Champaign, Department of Electrical and Computer Engineering, Urbana, Illinois, United States
- Laboratory for Photonics Research of Bio/Nano Environments (PROBE Lab), Urbana, Illinois and Providence, Rhode Island, United States
| | - Adriana C. Salazar Coariti
- Laboratory for Photonics Research of Bio/Nano Environments (PROBE Lab), Urbana, Illinois and Providence, Rhode Island, United States
| | - Kimani C. Toussaint
- Laboratory for Photonics Research of Bio/Nano Environments (PROBE Lab), Urbana, Illinois and Providence, Rhode Island, United States
- Brown University, School of Engineering, Providence, Rhode Island, United States
| |
Collapse
|
8
|
Wijesinghe P, Escobet-Montalbán A, Chen M, Munro PRT, Dholakia K. Optimal compressive multiphoton imaging at depth using single-pixel detection. OPTICS LETTERS 2019; 44:4981-4984. [PMID: 31613244 DOI: 10.1364/ol.44.004981] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/03/2019] [Accepted: 08/01/2019] [Indexed: 06/10/2023]
Abstract
Compressive sensing can overcome the Nyquist criterion and record images with a fraction of the usual number of measurements required. However, conventional measurement bases are susceptible to diffraction and scattering, prevalent in high-resolution microscopy. In this Letter, we explore the random Morlet basis as an optimal set for compressive multiphoton imaging, based on its ability to minimize the space-frequency uncertainty. We implement this approach for wide-field multiphoton microscopy with single-pixel detection, which allows imaging through turbid media without correction. The Morlet basis promises a route for rapid acquisition with lower photodamage.
Collapse
|