1
|
Taimori A, Mills B, Gaughan E, Ali A, Dhaliwal K, Williams G, Finlayson N, Hopgood JR. A Novel Fit-Flexible Fluorescence Soft Imager: Tri-Sensing of Intensity, Fall-Time, and Life Profile. IEEE Trans Biomed Eng 2024; 71:1864-1878. [PMID: 38300773 DOI: 10.1109/tbme.2024.3354856] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/03/2024]
Abstract
Time-resolved fluorescence imaging techniques, like confocal fluorescence lifetime imaging microscopy, are powerful photonic instrumentation tools of modern science with diverse applications, including: biology, medicine, and chemistry. However, complexities of the systems, both at specimen and device levels, cause difficulties in quantifying soft biomarkers. To address the problems, we first aim to understand and model the underlying photophysics of fluorescence decay curves. For this purpose, we provide a set of mathematical functions, called "life models", fittable with the real temporal recordings of histogram of photon counts. For each model, an equivalent electrical circuit, called a "life circuit", is derived for explaining the whole process. In confocal endomicroscopy, the components of excitation laser, specimen, and fluorescence-emission signal as the histogram of photon counts are modelled by a power source, network of resistor-inductor-capacitor circuitry, and multimetre, respectively. We then design a novel pixel-level temporal classification algorithm, called a "fit-flexible approach", where qualities of "intensity", "fall-time", and "life profile" are identified for each point. A model selection mechanism is used at each pixel to flexibly choose the best representative life model based on a proposed Misfit-percent metric. A two-dimensional arrangement of the quantified information detects some kind of structural information. This approach showed a potential of separating microbeads from lung tissue, distinguishing the tri-sensing from conventional methods. We alleviated by 7% the error of the Misfit-percent for recovering the histograms on real samples than the best state-of-the-art competitor. Codes are available online.
Collapse
|
2
|
Kapitany V, Fatima A, Zickus V, Whitelaw J, McGhee E, Insall R, Machesky L, Faccio D. Single-sample image-fusion upsampling of fluorescence lifetime images. SCIENCE ADVANCES 2024; 10:eadn0139. [PMID: 38781345 PMCID: PMC11114222 DOI: 10.1126/sciadv.adn0139] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/17/2023] [Accepted: 04/17/2024] [Indexed: 05/25/2024]
Abstract
Fluorescence lifetime imaging microscopy (FLIM) provides detailed information about molecular interactions and biological processes. A major bottleneck for FLIM is image resolution at high acquisition speeds due to the engineering and signal-processing limitations of time-resolved imaging technology. Here, we present single-sample image-fusion upsampling, a data-fusion approach to computational FLIM super-resolution that combines measurements from a low-resolution time-resolved detector (that measures photon arrival time) and a high-resolution camera (that measures intensity only). To solve this otherwise ill-posed inverse retrieval problem, we introduce statistically informed priors that encode local and global correlations between the two "single-sample" measurements. This bypasses the risk of out-of-distribution hallucination as in traditional data-driven approaches and delivers enhanced images compared, for example, to standard bilinear interpolation. The general approach laid out by single-sample image-fusion upsampling can be applied to other image super-resolution problems where two different datasets are available.
Collapse
Affiliation(s)
- Valentin Kapitany
- School of Physics & Astronomy, University of Glasgow, Glasgow G12 8QQ, UK
| | - Areeba Fatima
- School of Physics & Astronomy, University of Glasgow, Glasgow G12 8QQ, UK
| | - Vytautas Zickus
- School of Physics & Astronomy, University of Glasgow, Glasgow G12 8QQ, UK
- Department of Laser Technologies, Center for Physical Sciences and Technology, LT-10257 Vilnius, Lithuania
| | | | - Ewan McGhee
- School of Physics & Astronomy, University of Glasgow, Glasgow G12 8QQ, UK
- Cancer Research UK, Beatson Institute, Glasgow, UK
| | | | | | - Daniele Faccio
- School of Physics & Astronomy, University of Glasgow, Glasgow G12 8QQ, UK
| |
Collapse
|
3
|
Feng Y, Yang X, Rao Q, Zhang L, Su Y, Lv Y. Persistent Luminescence Lifetime-Based Near-Infrared Nanoplatform via Deep Learning for High-Fidelity Biosensing of Hypochlorite. Anal Chem 2024; 96:7240-7247. [PMID: 38661330 DOI: 10.1021/acs.analchem.4c00899] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/26/2024]
Abstract
In light of deep tissue penetration and ultralow background, near-infrared (NIR) persistent luminescence (PersL) bioprobes have become powerful tools for bioapplications. However, the inhomogeneous signal attenuation may significantly limit its application for precise biosensing owing to tissue absorption and scattering. In this work, a PersL lifetime-based nanoplatform via deep learning was proposed for high-fidelity bioimaging and biosensing in vivo. The persistent luminescence imaging network (PLI-Net), which consisted of a 3D-deep convolutional neural network (3D-CNN) and the PersL imaging system, was logically constructed to accurately extract the lifetime feature from the profile of PersL intensity-based decay images. Significantly, the NIR PersL nanomaterials represented by Zn1+xGa2-2xSnxO4: 0.4 % Cr (ZGSO) were precisely adjusted over their lifetime, enabling the PersL lifetime-based imaging with high-contrast signals. Inspired by the adjustable and reliable PersL lifetime imaging of ZGSO NPs, a proof-of-concept PersL nanoplatform was further developed and showed exceptional analytical performance for hypochlorite detection via a luminescence resonance energy transfer process. Remarkably, on the merits of the dependable and anti-interference PersL lifetimes, this PersL lifetime-based nanoprobe provided highly sensitive and accurate imaging of both endogenous and exogenous hypochlorite. This breakthrough opened up a new way for the development of high-fidelity biosensing in complex matrix systems.
Collapse
Affiliation(s)
- Yang Feng
- Analytical & Testing Center, Sichuan University, Chengdu 610064, China
| | - Xinyi Yang
- Analytical & Testing Center, Sichuan University, Chengdu 610064, China
| | - Qianli Rao
- Analytical & Testing Center, Sichuan University, Chengdu 610064, China
| | - Lichun Zhang
- Key Laboratory of Green Chemistry & Technology, Ministry of Education, College of Chemistry, Sichuan University, Chengdu 610064, China
| | - Yingying Su
- Analytical & Testing Center, Sichuan University, Chengdu 610064, China
| | - Yi Lv
- Analytical & Testing Center, Sichuan University, Chengdu 610064, China
- Key Laboratory of Green Chemistry & Technology, Ministry of Education, College of Chemistry, Sichuan University, Chengdu 610064, China
| |
Collapse
|
4
|
Todaro B, Pesce L, Cardarelli F, Luin S. Pioglitazone Phases and Metabolic Effects in Nanoparticle-Treated Cells Analyzed via Rapid Visualization of FLIM Images. Molecules 2024; 29:2137. [PMID: 38731628 PMCID: PMC11085555 DOI: 10.3390/molecules29092137] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2024] [Revised: 04/30/2024] [Accepted: 05/01/2024] [Indexed: 05/13/2024] Open
Abstract
Fluorescence lifetime imaging microscopy (FLIM) has proven to be a useful method for analyzing various aspects of material science and biology, like the supramolecular organization of (slightly) fluorescent compounds or the metabolic activity in non-labeled cells; in particular, FLIM phasor analysis (phasor-FLIM) has the potential for an intuitive representation of complex fluorescence decays and therefore of the analyzed properties. Here we present and make available tools to fully exploit this potential, in particular by coding via hue, saturation, and intensity the phasor positions and their weights both in the phasor plot and in the microscope image. We apply these tools to analyze FLIM data acquired via two-photon microscopy to visualize: (i) different phases of the drug pioglitazone (PGZ) in solutions and/or crystals, (ii) the position in the phasor plot of non-labelled poly(lactic-co-glycolic acid) (PLGA) nanoparticles (NPs), and (iii) the effect of PGZ or PGZ-containing NPs on the metabolism of insulinoma (INS-1 E) model cells. PGZ is recognized for its efficacy in addressing insulin resistance and hyperglycemia in type 2 diabetes mellitus, and polymeric nanoparticles offer versatile platforms for drug delivery due to their biocompatibility and controlled release kinetics. This study lays the foundation for a better understanding via phasor-FLIM of the organization and effects of drugs, in particular, PGZ, within NPs, aiming at better control of encapsulation and pharmacokinetics, and potentially at novel anti-diabetics theragnostic nanotools.
Collapse
Affiliation(s)
- Biagio Todaro
- NEST Laboratory, Scuola Normale Superiore, Piazza San Silvestro 12, 56127 Pisa, Italy; (L.P.); (F.C.)
| | - Luca Pesce
- NEST Laboratory, Scuola Normale Superiore, Piazza San Silvestro 12, 56127 Pisa, Italy; (L.P.); (F.C.)
| | - Francesco Cardarelli
- NEST Laboratory, Scuola Normale Superiore, Piazza San Silvestro 12, 56127 Pisa, Italy; (L.P.); (F.C.)
| | - Stefano Luin
- NEST Laboratory, Scuola Normale Superiore, Piazza San Silvestro 12, 56127 Pisa, Italy; (L.P.); (F.C.)
- NEST Laboratory, Istituto Nanoscienze-CNR, Piazza San Silvestro 12, 56127 Pisa, Italy
| |
Collapse
|
5
|
Hu L, De Hoyos D, Lei Y, West AP, Walsh AJ. 3D convolutional neural networks predict cellular metabolic pathway use from fluorescence lifetime decay data. APL Bioeng 2024; 8:016112. [PMID: 38420625 PMCID: PMC10901549 DOI: 10.1063/5.0188476] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2023] [Accepted: 01/30/2024] [Indexed: 03/02/2024] Open
Abstract
Fluorescence lifetime imaging of the co-enzyme reduced nicotinamide adenine dinucleotide (NADH) offers a label-free approach for detecting cellular metabolic perturbations. However, the relationships between variations in NADH lifetime and metabolic pathway changes are complex, preventing robust interpretation of NADH lifetime data relative to metabolic phenotypes. Here, a three-dimensional convolutional neural network (3D CNN) trained at the cell level with 3D NAD(P)H lifetime decay images (two spatial dimensions and one time dimension) was developed to identify metabolic pathway usage by cancer cells. NADH fluorescence lifetime images of MCF7 breast cancer cells with three isolated metabolic pathways, glycolysis, oxidative phosphorylation, and glutaminolysis were obtained by a multiphoton fluorescence lifetime microscope and then segmented into individual cells as the input data for the classification models. The 3D CNN models achieved over 90% accuracy in identifying cancer cells reliant on glycolysis, oxidative phosphorylation, or glutaminolysis. Furthermore, the model trained with human breast cancer cell data successfully predicted the differences in metabolic phenotypes of macrophages from control and POLG-mutated mice. These results suggest that the integration of autofluorescence lifetime imaging with 3D CNNs enables intracellular spatial patterns of NADH intensity and temporal dynamics of the lifetime decay to discriminate multiple metabolic phenotypes. Furthermore, the use of 3D CNNs to identify metabolic phenotypes from NADH fluorescence lifetime decay images eliminates the need for time- and expertise-demanding exponential decay fitting procedures. In summary, metabolic-prediction CNNs will enable live-cell and in vivo metabolic measurements with single-cell resolution, filling a current gap in metabolic measurement technologies.
Collapse
Affiliation(s)
- Linghao Hu
- Department of Biomedical Engineering, Texas A&M University, College Station, Texas 77843, USA
| | - Daniela De Hoyos
- Department of Biomedical Engineering, Texas A&M University, College Station, Texas 77843, USA
| | - Yuanjiu Lei
- Department of Microbial Pathogenesis and Immunology, School of Medicine, Texas A&M University, Bryan, Texas 77807, USA
| | | | - Alex J. Walsh
- Department of Biomedical Engineering, Texas A&M University, College Station, Texas 77843, USA
| |
Collapse
|
6
|
Lin Y, Mos P, Ardelean A, Bruschini C, Charbon E. Coupling a recurrent neural network to SPAD TCSPC systems for real-time fluorescence lifetime imaging. Sci Rep 2024; 14:3286. [PMID: 38331957 PMCID: PMC10853568 DOI: 10.1038/s41598-024-52966-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Accepted: 01/25/2024] [Indexed: 02/10/2024] Open
Abstract
Fluorescence lifetime imaging (FLI) has been receiving increased attention in recent years as a powerful diagnostic technique in biological and medical research. However, existing FLI systems often suffer from a tradeoff between processing speed, accuracy, and robustness. Inspired by the concept of Edge Artificial Intelligence (Edge AI), we propose a robust approach that enables fast FLI with no degradation of accuracy. This approach couples a recurrent neural network (RNN), which is trained to estimate the fluorescence lifetime directly from raw timestamps without building histograms, to SPAD TCSPC systems, thereby drastically reducing transfer data volumes and hardware resource utilization, and enabling real-time FLI acquisition. We train two variants of the RNN on a synthetic dataset and compare the results to those obtained using center-of-mass method (CMM) and least squares fitting (LS fitting). Results demonstrate that two RNN variants, gated recurrent unit (GRU) and long short-term memory (LSTM), are comparable to CMM and LS fitting in terms of accuracy, while outperforming them in the presence of background noise by a large margin. To explore the ultimate limits of the approach, we derive the Cramer-Rao lower bound of the measurement, showing that RNN yields lifetime estimations with near-optimal precision. To demonstrate real-time operation, we build a FLI microscope based on an existing SPAD TCSPC system comprising a 32[Formula: see text]32 SPAD sensor named Piccolo. Four quantized GRU cores, capable of processing up to 4 million photons per second, are deployed on the Xilinx Kintex-7 FPGA that controls the Piccolo. Powered by the GRU, the FLI setup can retrieve real-time fluorescence lifetime images at up to 10 frames per second. The proposed FLI system is promising and ideally suited for biomedical applications, including biological imaging, biomedical diagnostics, and fluorescence-assisted surgery, etc.
Collapse
Affiliation(s)
- Yang Lin
- Advanced Quantum Architecture Laboratory, École polytechnique fédérale de Lausanne, Neuchâtel, 2002, Switzerland
| | - Paul Mos
- Advanced Quantum Architecture Laboratory, École polytechnique fédérale de Lausanne, Neuchâtel, 2002, Switzerland
| | - Andrei Ardelean
- Advanced Quantum Architecture Laboratory, École polytechnique fédérale de Lausanne, Neuchâtel, 2002, Switzerland
| | - Claudio Bruschini
- Advanced Quantum Architecture Laboratory, École polytechnique fédérale de Lausanne, Neuchâtel, 2002, Switzerland
| | - Edoardo Charbon
- Advanced Quantum Architecture Laboratory, École polytechnique fédérale de Lausanne, Neuchâtel, 2002, Switzerland.
| |
Collapse
|
7
|
Gouzou D, Taimori A, Haloubi T, Finlayson N, Wang Q, Hopgood JR, Vallejo M. Applications of machine learning in time-domain fluorescence lifetime imaging: a review. Methods Appl Fluoresc 2024; 12:022001. [PMID: 38055998 PMCID: PMC10851337 DOI: 10.1088/2050-6120/ad12f7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2023] [Revised: 09/25/2023] [Accepted: 12/06/2023] [Indexed: 12/08/2023]
Abstract
Many medical imaging modalities have benefited from recent advances in Machine Learning (ML), specifically in deep learning, such as neural networks. Computers can be trained to investigate and enhance medical imaging methods without using valuable human resources. In recent years, Fluorescence Lifetime Imaging (FLIm) has received increasing attention from the ML community. FLIm goes beyond conventional spectral imaging, providing additional lifetime information, and could lead to optical histopathology supporting real-time diagnostics. However, most current studies do not use the full potential of machine/deep learning models. As a developing image modality, FLIm data are not easily obtainable, which, coupled with an absence of standardisation, is pushing back the research to develop models which could advance automated diagnosis and help promote FLIm. In this paper, we describe recent developments that improve FLIm image quality, specifically time-domain systems, and we summarise sensing, signal-to-noise analysis and the advances in registration and low-level tracking. We review the two main applications of ML for FLIm: lifetime estimation and image analysis through classification and segmentation. We suggest a course of action to improve the quality of ML studies applied to FLIm. Our final goal is to promote FLIm and attract more ML practitioners to explore the potential of lifetime imaging.
Collapse
Affiliation(s)
- Dorian Gouzou
- Dorian Gouzou and Marta Vallejo are with Institute of Signals, Sensors and Systems, School of Engineering and Physical Sciences, Heriot Watt University, Edinburgh, EH14 4AS, United Kingdom
| | - Ali Taimori
- Tarek Haloubi, Ali Taimori, and James R. Hopgood are with Institute for Imaging, Data and Communication, School of Engineering, University of Edinburgh, Edinburgh, EH9 3FG, United Kingdom
| | - Tarek Haloubi
- Tarek Haloubi, Ali Taimori, and James R. Hopgood are with Institute for Imaging, Data and Communication, School of Engineering, University of Edinburgh, Edinburgh, EH9 3FG, United Kingdom
| | - Neil Finlayson
- Neil Finlayson is with Institute for Integrated Micro and Nano Systems, School of Engineering, University ofEdinburgh, Edinburgh EH9 3FF, United Kingdom
| | - Qiang Wang
- Qiang Wang is with Centre for Inflammation Research, University of Edinburgh, Edinburgh, EH16 4TJ, United Kingdom
| | - James R Hopgood
- Tarek Haloubi, Ali Taimori, and James R. Hopgood are with Institute for Imaging, Data and Communication, School of Engineering, University of Edinburgh, Edinburgh, EH9 3FG, United Kingdom
| | - Marta Vallejo
- Dorian Gouzou and Marta Vallejo are with Institute of Signals, Sensors and Systems, School of Engineering and Physical Sciences, Heriot Watt University, Edinburgh, EH14 4AS, United Kingdom
| |
Collapse
|
8
|
Barra J, Crosbourne I, Roberge CL, Bossardi-Ramos R, Warren JSA, Matteson K, Wang L, Jourd'heuil F, Borisov SM, Bresnahan E, Bravo-Cordero JJ, Dmitriev RI, Jourd'heuil D, Adam AP, Lamar JM, Corr DT, Barroso MM. DMT1-dependent endosome-mitochondria interactions regulate mitochondrial iron translocation and metastatic outgrowth. Oncogene 2024; 43:650-667. [PMID: 38184712 PMCID: PMC10890933 DOI: 10.1038/s41388-023-02933-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2023] [Revised: 12/18/2023] [Accepted: 12/21/2023] [Indexed: 01/08/2024]
Abstract
Transient early endosome (EE)-mitochondria interactions can mediate mitochondrial iron translocation, but the associated mechanisms are still elusive. We showed that Divalent Metal Transporter 1 (DMT1) sustains mitochondrial iron translocation via EE-mitochondria interactions in triple-negative MDA-MB-231, but not in luminal A T47D breast cancer cells. DMT1 silencing increases labile iron pool (LIP) levels and activates PINK1/Parkin-dependent mitophagy in MDA-MB-231 cells. Mitochondrial bioenergetics and the iron-associated protein profile were altered by DMT1 silencing and rescued by DMT1 re-expression. Transcriptomic profiles upon DMT1 silencing are strikingly different between 2D and 3D culture conditions, suggesting that the environment context is crucial for the DMT1 knockout phenotype observed in MDA-MB-231 cells. Lastly, in vivo lung metastasis assay revealed that DMT1 silencing promoted the outgrowth of lung metastatic nodules in both human and murine models of triple-negative breast cancer cells. These findings reveal a DMT1-dependent pathway connecting EE-mitochondria interactions to mitochondrial iron translocation and metastatic fitness of breast cancer cells.
Collapse
Affiliation(s)
- Jonathan Barra
- Department of Molecular and Cellular Physiology, Albany Medical College, Albany, NY, 12208, USA
- Department of Medicine, Division of Hematology and Oncology, Tisch Cancer Institute, Icahn School of Medicine at Mount Sinai, New York, NY, 10029, USA
| | - Isaiah Crosbourne
- Department of Molecular and Cellular Physiology, Albany Medical College, Albany, NY, 12208, USA
| | - Cassandra L Roberge
- Department of Molecular and Cellular Physiology, Albany Medical College, Albany, NY, 12208, USA
| | - Ramon Bossardi-Ramos
- Department of Molecular and Cellular Physiology, Albany Medical College, Albany, NY, 12208, USA
| | - Janine S A Warren
- Department of Molecular and Cellular Physiology, Albany Medical College, Albany, NY, 12208, USA
| | - Kailie Matteson
- Department of Molecular and Cellular Physiology, Albany Medical College, Albany, NY, 12208, USA
- Department of Medicine, Division of Hematology and Oncology, Tisch Cancer Institute, Icahn School of Medicine at Mount Sinai, New York, NY, 10029, USA
| | - Ling Wang
- Department of Molecular and Cellular Physiology, Albany Medical College, Albany, NY, 12208, USA
- Department of Biomedical Engineering, Binghamton University, Binghamton, NY, 13902, USA
| | - Frances Jourd'heuil
- Department of Molecular and Cellular Physiology, Albany Medical College, Albany, NY, 12208, USA
| | - Sergey M Borisov
- Institute of Analytical Chemistry and Food Chemistry, Graz University of Technology Stremayrgasse 9, 8010, Graz, Austria
| | - Erin Bresnahan
- Department of Medicine, Division of Hematology and Oncology, Tisch Cancer Institute, Icahn School of Medicine at Mount Sinai, New York, NY, 10029, USA
| | - Jose Javier Bravo-Cordero
- Department of Medicine, Division of Hematology and Oncology, Tisch Cancer Institute, Icahn School of Medicine at Mount Sinai, New York, NY, 10029, USA
| | - Ruslan I Dmitriev
- Tissue Engineering and Biomaterials Group, Department of Human Structure and Repair, Faculty of Medical and Health Sciences, Ghent University, C. Heymanslaan 10, 9000, Ghent, Belgium
| | - David Jourd'heuil
- Department of Molecular and Cellular Physiology, Albany Medical College, Albany, NY, 12208, USA
| | - Alejandro P Adam
- Department of Molecular and Cellular Physiology, Albany Medical College, Albany, NY, 12208, USA
| | - John M Lamar
- Department of Molecular and Cellular Physiology, Albany Medical College, Albany, NY, 12208, USA
| | - David T Corr
- Department of Biomedical Engineering, Rensselaer Polytechnic Institute, Troy, NY, 12180-3590, USA
| | - Margarida M Barroso
- Department of Molecular and Cellular Physiology, Albany Medical College, Albany, NY, 12208, USA.
| |
Collapse
|
9
|
Ghezzi A, Farina A, Vurro V, Bassi A, Valentini G, D'Andrea C. Fast data fitting scheme for compressive multispectral fluorescence lifetime imaging. OPTICS LETTERS 2024; 49:278-281. [PMID: 38194547 DOI: 10.1364/ol.506378] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/22/2023] [Accepted: 11/26/2023] [Indexed: 01/11/2024]
Abstract
A single-pixel camera combined with compressive sensing techniques is a promising fluorescence microscope scheme for acquiring a multidimensional dataset (space, spectrum, and lifetime) and for reducing the measurement time with respect to conventional microscope schemes. However, upon completing the acquisition, a computational step is necessary for image reconstruction and data analysis, which can be time-consuming, potentially canceling out the beneficial effect of compressive sensing. In this work, we propose and experimentally validate a fast-fit workflow based on global analysis and multiple linear fits, which significantly reduces the computation time from tens of minutes to less than 1 s. Moreover, as the method is interlaced with the measurement flow, it can be applied in parallel with the acquisitions.
Collapse
|
10
|
Petusseau AF, Streeter SS, Ulku A, Feng Y, Samkoe KS, Bruschini C, Charbon E, Pogue BW, Bruza P. Subsurface fluorescence time-of-flight imaging using a large-format single-photon avalanche diode sensor for tumor depth assessment. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:016004. [PMID: 38235320 PMCID: PMC10794045 DOI: 10.1117/1.jbo.29.1.016004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Revised: 12/06/2023] [Accepted: 12/20/2023] [Indexed: 01/19/2024]
Abstract
Significance Fluorescence guidance is used clinically by surgeons to visualize anatomical and/or physiological phenomena in the surgical field that are difficult or impossible to detect by the naked eye. Such phenomena include tissue perfusion or molecular phenotypic information about the disease being resected. Conventional fluorescence-guided surgery relies on long, microsecond scale laser pulses to excite fluorescent probes. However, this technique only provides two-dimensional information; crucial depth information, such as the location of malignancy below the tissue surface, is not provided. Aim We developed a depth sensing imaging technique using light detection and ranging (LiDAR) time-of-flight (TOF) technology to sense the depth of target tissue while overcoming the influence of tissue optical properties and fluorescent probe concentration. Approach The technology is based on a large-format (512 × 512 pixel ), binary, gated, single-photon avalanche diode (SPAD) sensor with an 18 ps time-gate step, synchronized with a picosecond pulsed laser. The fast response of the sensor was developed and tested for its ability to quantify fluorescent inclusions at depth and optical properties in tissue-like phantoms through analytical model fitting of the fast temporal remission data. Results After calibration and algorithmic extraction of the data, the SPAD LiDAR technique allowed for sub-mm resolution depth sensing of fluorescent inclusions embedded in tissue-like phantoms, up to a maximum of 5 mm in depth. The approach provides robust depth sensing even in the presence of variable tissue optical properties and separates the effects of fluorescence depth from absorption and scattering variations. Conclusions LiDAR TOF fluorescence imaging using an SPAD camera provides both fluorescence intensity images and the temporal profile of fluorescence, which can be used to determine the depth at which the signal is emitted over a wide field of view. The proposed tool enables fluorescence imaging at a higher depth in tissue and with higher spatial precision than standard, steady-state fluorescence imaging tools, such as intensity-based near-infrared fluorescence imaging, optical coherence tomography, Raman spectroscopy, or confocal microscopy. Integration of this technique into a standard surgical tool could enable rapid, more accurate estimation of resection boundaries, thereby improving the surgeon's efficacy and efficiency, and ultimately improving patient outcomes.
Collapse
Affiliation(s)
- Arthur F Petusseau
- Dartmouth College, Thayer School of Engineering and Dartmouth Cancer Center, Hanover, New Hampshire, United States
| | - Samuel S Streeter
- Geisel School of Medicine at Dartmouth, Department of Orthopaedics, Hanover, New Hampshire, United States
| | - Arin Ulku
- Ecole polytechnique fédérale de Lausanne, Advanced Quantum Architecture Laboratory, Neuchâtel, Switzerland
| | - Yichen Feng
- Geisel School of Medicine at Dartmouth, Department of Surgery, Hanover, New Hampshire, United States
| | - Kimberley S Samkoe
- Geisel School of Medicine at Dartmouth, Department of Surgery, Hanover, New Hampshire, United States
| | - Claudio Bruschini
- Ecole polytechnique fédérale de Lausanne, Advanced Quantum Architecture Laboratory, Neuchâtel, Switzerland
| | - Edoardo Charbon
- Ecole polytechnique fédérale de Lausanne, Advanced Quantum Architecture Laboratory, Neuchâtel, Switzerland
| | - Brian W Pogue
- Dartmouth College, Thayer School of Engineering and Dartmouth Cancer Center, Hanover, New Hampshire, United States
- University of Wisconsin-Madison, Department of Medical Physics, Madison, Wisconsin, United States
| | - Petr Bruza
- Dartmouth College, Thayer School of Engineering and Dartmouth Cancer Center, Hanover, New Hampshire, United States
| |
Collapse
|
11
|
Houston JP, Valentino S, Bitton A. Fluorescence Lifetime Measurements and Analyses: Protocols Using Flow Cytometry and High-Throughput Microscopy. Methods Mol Biol 2024; 2779:323-351. [PMID: 38526793 DOI: 10.1007/978-1-0716-3738-8_15] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/27/2024]
Abstract
This chapter focuses on applications and protocols that involve the measurement of the fluorescence lifetime as an informative cytometric parameter. The timing of fluorescence decay has been well-studied for cell counting, sorting, and imaging. Therefore, provided herein is an overview of the techniques used, how they enhance cytometry protocols, and the modern techniques used for lifetime analysis. The background and theory behind fluorescence decay kinetic measurements in cells is first discussed followed by the history of the development of time-resolved flow cytometry. These sections are followed by a review of applications that benefit from the quantitative nature of fluorescence lifetimes as a photophysical trait. Lastly, perspectives on the modern ways in which the fluorescence lifetime is scanned at high throughputs which include high-speed microscopy and machine learning are provided.
Collapse
Affiliation(s)
- Jessica P Houston
- Department of Chemical & Materials Engineering, New Mexico State University, Las Cruces, NM, USA.
| | - Samantha Valentino
- Department of Chemical & Materials Engineering, New Mexico State University, Las Cruces, NM, USA
| | | |
Collapse
|
12
|
Netaev A, Schierbaum N, Seidl K. Artificial Neural Network (ANN)-Based Determination of Fractional Contributions from Mixed Fluorophores using Fluorescence Lifetime Measurements. J Fluoresc 2024; 34:305-311. [PMID: 37212979 PMCID: PMC10808714 DOI: 10.1007/s10895-023-03261-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Accepted: 05/04/2023] [Indexed: 05/23/2023]
Abstract
Here we present an artificial neural network (ANN)-approach to determine the fractional contributions Pi from fluorophores to a multi-exponential fluorescence decay in time-resolved lifetime measurements. Conventionally, Pi are determined by extracting two parameters (amplitude and lifetime) for each underlying mono-exponential decay using non-linear fitting. However, in this case parameter estimation is highly sensitive to initial guesses and weighting. In contrast, the ANN-based approach robustly gives the Pi without knowledge of amplitudes and lifetimes. By experimental measurements and Monte-Carlo simulations, we comprehensively show that accuracy and precision of Pi determination with ANNs and hence the number of distinguishable fluorophores depend on the fluorescence lifetimes' differences. For mixtures of up to five fluorophores, we determined the minimum uniform spacing Δτmin between lifetimes to obtain fractional contributions with a standard deviation of 5%. In example, five lifetimes can be distinguished with a respective minimum uniform spacing of approx. 10 ns even when the fluorophores' emission spectra are overlapping. This study underlines the enormous potential of ANN-based analysis for multi-fluorophore applications in fluorescence lifetime measurements.
Collapse
Affiliation(s)
- Alexander Netaev
- Fraunhofer Institute for Microelectronic Circuits and Systems, Finkenstr. 61, 47057, Duisburg, Germany.
| | - Nicolas Schierbaum
- Fraunhofer Institute for Microelectronic Circuits and Systems, Finkenstr. 61, 47057, Duisburg, Germany
| | - Karsten Seidl
- Fraunhofer Institute for Microelectronic Circuits and Systems, Finkenstr. 61, 47057, Duisburg, Germany
- Department of Electronic Components and Circuits and Center for Nanointegration Duisburg-Essen (CENIDE), University Duisburg-Essen, 47057, Duisburg, Germany
| |
Collapse
|
13
|
Alexiev U, Rühl E. Visualization of Nanocarriers and Drugs in Cells and Tissue. Handb Exp Pharmacol 2024; 284:153-189. [PMID: 37566121 DOI: 10.1007/164_2023_684] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/12/2023]
Abstract
In this chapter, the visualization of nanocarriers and drugs in cells and tissue is reviewed. This topic is tightly connected to modern drug delivery, which relies on nanoscopic drug formulation approaches and the ability to probe nanoparticulate systems selectively in cells and tissue using advanced spectroscopic and microscopic techniques. We first give an overview of the breadth of this research field. Then, we mainly focus on topical drug delivery to the skin and discuss selected visualization techniques from spectromicroscopy, such as scanning transmission X-ray microscopy and fluorescence lifetime imaging. These techniques rely on the sensitive and quantitative detection of the topically applied drug delivery systems and active substances, either by exploiting their molecular properties or by introducing environmentally sensitive probes that facilitate their detection.
Collapse
Affiliation(s)
- Ulrike Alexiev
- Fachbereich Physik, Freie Universität Berlin, Berlin, Germany.
| | - Eckart Rühl
- Physikalische Chemie, Freie Universität Berlin, Berlin, Germany.
| |
Collapse
|
14
|
Bernardi M, Cardarelli F. Phasor identifier: A cloud-based analysis of phasor-FLIM data on Python notebooks. BIOPHYSICAL REPORTS 2023; 3:100135. [PMID: 38053971 PMCID: PMC10694583 DOI: 10.1016/j.bpr.2023.100135] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Accepted: 11/03/2023] [Indexed: 12/07/2023]
Abstract
This paper introduces an innovative approach utilizing Google Colaboratory for the versatile analysis of phasor fluorescence lifetime imaging microscopy (FLIM) data collected from various samples (e.g., cuvette, cells, tissues) and in various input file formats. In fact, phasor-FLIM widespread adoption has been hampered by complex instrumentation and data analysis requirements. We mean to make advanced FLIM analysis more accessible to researchers through a cloud-based solution that 1) harnesses robust computational resources, 2) eliminates hardware limitations, and 3) supports both CPU and GPU processing. We envision a paradigm shift in FLIM data accessibility and potential, aligning with the evolving field of artificial intelligence-driven FLIM analysis. This approach simplifies FLIM data handling and opens doors for diverse applications, from studying cellular metabolism to investigating drug encapsulation, benefiting researchers across multiple domains. The comparative analysis of freely distributed FLIM tools highlights the unique advantages of this approach in terms of adaptability, scalability, and open-source nature.
Collapse
|
15
|
Pal R, K M, Matsui A, Kang H, Morita S, Taniguchi H, Kobayashi T, Morita A, Choi HS, Duda DG, Kumar ATN. In vivo quantification of programmed death-ligand-1 expression heterogeneity in tumors using fluorescence lifetime imaging. RESEARCH SQUARE 2023:rs.3.rs-3222037. [PMID: 37961361 PMCID: PMC10635296 DOI: 10.21203/rs.3.rs-3222037/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/15/2023]
Abstract
Cancer patient selection for immunotherapy is often based on programmed death-ligand-1 (PD-L1) expression as a biomarker. PD-L1 expression is currently quantified using immunohistochemistry, which can only provide snapshots of PD-L1 expression status in microscopic regions of ex vivo specimens. In vivo imaging using targeted agents can capture dynamic variations of PD-L1 expression in entire tumors within and across multiple subjects. Towards this goal, several PD-L1 targeted molecular imaging probes have been evaluated in murine models and humans. However, clinical translation of these probes has been limited due to a significant non-specific accumulation of the imaging probes and the inability of conventional imaging modalities to provide quantitative readouts that can be compared across multiple subjects. Here we report that in vivo time-domain (TD) fluorescence imaging can provide quantitative estimates of baseline tumor PD-L1 heterogeneity across untreated mice and variations in PD-L1 expression across mice undergoing clinically relevant anti-PD1 treatment. This approach relies on a significantly longer fluorescence lifetime (FLT) of PD-L1 specific anti-PD-L1 antibody tagged to IRDye 800CW (αPDL1-800) compared to nonspecific αPDL1-800. Leveraging this unique FLT contrast, we show that PD-L1 expression can be quantified across mice both in superficial breast tumors using planar FLT imaging, and in deep-seated liver tumors (>5 mm depth) using the asymptotic TD algorithm for fluorescence tomography. Our results suggest that FLT contrast can accelerate the preclinical investigation and clinical translation of novel molecular imaging probes by providing robust quantitative readouts of receptor expression that can be readily compared across subjects.
Collapse
Affiliation(s)
- Rahul Pal
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - Murali K
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - Aya Matsui
- Department of Vascular Physiology, Graduate School of Medical Science, Kanazawa University, Japan
| | - Homan Kang
- Gordon Center for Medical Imaging, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - Satoru Morita
- E. L. Steele Laboratories for Tumor Biology, Department of Radiation Oncology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - Hajime Taniguchi
- E. L. Steele Laboratories for Tumor Biology, Department of Radiation Oncology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
- Department of Surgery, Tohoku Graduate School of Medicine, Sendai, Japan
| | - Tatsuya Kobayashi
- E. L. Steele Laboratories for Tumor Biology, Department of Radiation Oncology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - Atsuyo Morita
- E. L. Steele Laboratories for Tumor Biology, Department of Radiation Oncology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - Hak Soo Choi
- Gordon Center for Medical Imaging, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - Dan G Duda
- E. L. Steele Laboratories for Tumor Biology, Department of Radiation Oncology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| | - Anand T N Kumar
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
| |
Collapse
|
16
|
Fazel M, Jazani S, Scipioni L, Vallmitjana A, Zhu S, Gratton E, Digman MA, Pressé S. Building Fluorescence Lifetime Maps Photon-by-Photon by Leveraging Spatial Correlations. ACS PHOTONICS 2023; 10:3558-3569. [PMID: 38406580 PMCID: PMC10890823 DOI: 10.1021/acsphotonics.3c00595] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/27/2024]
Abstract
Fluorescence lifetime imaging microscopy (FLIM) has become a standard tool in the quantitative characterization of subcellular environments. However, quantitative FLIM analyses face several challenges. First, spatial correlations between pixels are often ignored as signal from individual pixels is analyzed independently thereby limiting spatial resolution. Second, existing methods deduce photon ratios instead of absolute lifetime maps. Next, the number of fluorophore species contributing to the signal is unknown, while excited state lifetimes with <1 ns difference are difficult to discriminate. Finally, existing analyses require high photon budgets and often cannot rigorously propagate experimental uncertainty into values over lifetime maps and number of species involved. To overcome all of these challenges simultaneously and self-consistently at once, we propose the first doubly nonparametric framework. That is, we learn the number of species (using Beta-Bernoulli process priors) and absolute maps of these fluorophore species (using Gaussian process priors) by leveraging information from pulses not leading to observed photon. We benchmark our framework using a broad range of synthetic and experimental data and demonstrate its robustness across a number of scenarios including cases where we recover lifetime differences between species as small as 0.3 ns with merely 1000 photons.
Collapse
Affiliation(s)
- Mohamadreza Fazel
- Center for Biological Physics and Department of Physics, Arizona State University, Tempe, Arizona 85287, United States
| | - Sina Jazani
- Center for Biological Physics and Department of Physics, Arizona State University, Tempe, Arizona 85287, United States
| | - Lorenzo Scipioni
- Department of Biomedical Engineering, University of California Irvine, Irvine, California 92697, United States; Laboratory of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California, Irvine, California 92697, United States
| | - Alexander Vallmitjana
- Department of Biomedical Engineering, University of California Irvine, Irvine, California 92697, United States; Laboratory of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California, Irvine, California 92697, United States
| | - Songning Zhu
- Department of Biomedical Engineering, University of California Irvine, Irvine, California 92697, United States; Laboratory of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California, Irvine, California 92697, United States
| | - Enrico Gratton
- Department of Biomedical Engineering, University of California Irvine, Irvine, California 92697, United States; Laboratory of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California, Irvine, California 92697, United States
| | - Michelle A Digman
- Department of Biomedical Engineering, University of California Irvine, Irvine, California 92697, United States; Laboratory of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California, Irvine, California 92697, United States
| | - Steve Pressé
- Center for Biological Physics and Department of Physics, Arizona State University, Tempe, Arizona 85287, United States; School of Molecular Science, Arizona State University, Tempe, Arizona 85287, United States
| |
Collapse
|
17
|
Barroso M, Monaghan MG, Niesner R, Dmitriev RI. Probing organoid metabolism using fluorescence lifetime imaging microscopy (FLIM): The next frontier of drug discovery and disease understanding. Adv Drug Deliv Rev 2023; 201:115081. [PMID: 37647987 PMCID: PMC10543546 DOI: 10.1016/j.addr.2023.115081] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Revised: 04/20/2023] [Accepted: 08/24/2023] [Indexed: 09/01/2023]
Abstract
Organoid models have been used to address important questions in developmental and cancer biology, tissue repair, advanced modelling of disease and therapies, among other bioengineering applications. Such 3D microenvironmental models can investigate the regulation of cell metabolism, and provide key insights into the mechanisms at the basis of cell growth, differentiation, communication, interactions with the environment and cell death. Their accessibility and complexity, based on 3D spatial and temporal heterogeneity, make organoids suitable for the application of novel, dynamic imaging microscopy methods, such as fluorescence lifetime imaging microscopy (FLIM) and related decay time-assessing readouts. Several biomarkers and assays have been proposed to study cell metabolism by FLIM in various organoid models. Herein, we present an expert-opinion discussion on the principles of FLIM and PLIM, instrumentation and data collection and analysis protocols, and general and emerging biosensor-based approaches, to highlight the pioneering work being performed in this field.
Collapse
Affiliation(s)
- Margarida Barroso
- Department of Molecular and Cellular Physiology, Albany Medical College, Albany, NY 12208, USA
| | - Michael G Monaghan
- Department of Mechanical, Manufacturing and Biomedical Engineering, Trinity College Dublin, Dublin 02, Ireland
| | - Raluca Niesner
- Dynamic and Functional In Vivo Imaging, Freie Universität Berlin and Biophysical Analytics, German Rheumatism Research Center, Berlin, Germany
| | - Ruslan I Dmitriev
- Tissue Engineering and Biomaterials Group, Department of Human Structure and Repair, Faculty of Medicine and Health Sciences, Ghent University, C. Heymanslaan 10, 9000 Ghent, Belgium; Ghent Light Microscopy Core, Ghent University, 9000 Ghent, Belgium.
| |
Collapse
|
18
|
Deng B, Gu H, Zhu H, Chang K, Hoebel KV, Patel JB, Kalpathy-Cramer J, Carp SA. FDU-Net: Deep Learning-Based Three-Dimensional Diffuse Optical Image Reconstruction. IEEE TRANSACTIONS ON MEDICAL IMAGING 2023; 42:2439-2450. [PMID: 37028063 PMCID: PMC10446911 DOI: 10.1109/tmi.2023.3252576] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Near-infrared diffuse optical tomography (DOT) is a promising functional modality for breast cancer imaging; however, the clinical translation of DOT is hampered by technical limitations. Specifically, conventional finite element method (FEM)-based optical image reconstruction approaches are time-consuming and ineffective in recovering full lesion contrast. To address this, we developed a deep learning-based reconstruction model (FDU-Net) comprised of a Fully connected subnet, followed by a convolutional encoder-Decoder subnet, and a U-Net for fast, end-to-end 3D DOT image reconstruction. The FDU-Net was trained on digital phantoms that include randomly located singular spherical inclusions of various sizes and contrasts. Reconstruction performance was evaluated in 400 simulated cases with realistic noise profiles for the FDU-Net and conventional FEM approaches. Our results show that the overall quality of images reconstructed by FDU-Net is significantly improved compared to FEM-based methods and a previously proposed deep-learning network. Importantly, once trained, FDU-Net demonstrates substantially better capability to recover true inclusion contrast and location without using any inclusion information during reconstruction. The model was also generalizable to multi-focal and irregularly shaped inclusions unseen during training. Finally, FDU-Net, trained on simulated data, could successfully reconstruct a breast tumor from a real patient measurement. Overall, our deep learning-based approach demonstrates marked superiority over the conventional DOT image reconstruction methods while also offering over four orders of magnitude acceleration in computational time. Once adapted to the clinical breast imaging workflow, FDU-Net has the potential to provide real-time accurate lesion characterization by DOT to assist the clinical diagnosis and management of breast cancer.
Collapse
|
19
|
Smith JT, Sinsuebphon N, Rudkouskaya A, Michalet X, Intes X, Barroso M. In vivo quantitative FRET small animal imaging: Intensity versus lifetime-based FRET. BIOPHYSICAL REPORTS 2023; 3:100110. [PMID: 37251213 PMCID: PMC10209493 DOI: 10.1016/j.bpr.2023.100110] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/26/2023] [Accepted: 04/27/2023] [Indexed: 05/31/2023]
Abstract
Förster resonance energy transfer (FRET) microscopy is used in numerous biophysical and biomedical applications to monitor inter- and intramolecular interactions and conformational changes in the 2-10 nm range. FRET is currently being extended to in vivo optical imaging, its main application being in quantifying drug-target engagement or drug release in animal models of cancer using organic dye or nanoparticle-labeled probes. Herein, we compared FRET quantification using intensity-based FRET (sensitized emission FRET analysis with the three-cube approach using an IVIS imager) and macroscopic fluorescence lifetime (MFLI) FRET using a custom system using a time-gated-intensified charge-coupled device, for small animal optical in vivo imaging. The analytical expressions and experimental protocols required to quantify the product f D E of the FRET efficiency E and the fraction of donor molecules involved in FRET, f D , are described in detail for both methodologies. Dynamic in vivo FRET quantification of transferrin receptor-transferrin binding was acquired in live intact nude mice upon intravenous injection of a near-infrared-labeled transferrin FRET pair and benchmarked against in vitro FRET using hybridized oligonucleotides. Even though both in vivo imaging techniques provided similar dynamic trends for receptor-ligand engagement, we demonstrate that MFLI-FRET has significant advantages. Whereas the sensitized emission FRET approach using the IVIS imager required nine measurements (six of which are used for calibration) acquired from three mice, MFLI-FRET needed only one measurement collected from a single mouse, although a control mouse might be needed in a more general situation. Based on our study, MFLI therefore represents the method of choice for longitudinal preclinical FRET studies such as that of targeted drug delivery in intact, live mice.
Collapse
Affiliation(s)
- Jason T. Smith
- Center for Modeling, Simulation and Imaging in Medicine (CeMSIM), Rensselaer Polytechnic Institute, Troy, New York
| | - Nattawut Sinsuebphon
- Center for Modeling, Simulation and Imaging in Medicine (CeMSIM), Rensselaer Polytechnic Institute, Troy, New York
| | - Alena Rudkouskaya
- Department of Molecular and Cellular Physiology, Albany Medical College, Albany, New York
| | - Xavier Michalet
- Department of Chemistry & Biochemistry, University of California at Los Angeles, Los Angeles, California
| | - Xavier Intes
- Center for Modeling, Simulation and Imaging in Medicine (CeMSIM), Rensselaer Polytechnic Institute, Troy, New York
| | - Margarida Barroso
- Department of Molecular and Cellular Physiology, Albany Medical College, Albany, New York
| |
Collapse
|
20
|
Abstract
Over the last half century, the autofluorescence of the metabolic cofactors NADH (reduced nicotinamide adenine dinucleotide) and FAD (flavin adenine dinucleotide) has been quantified in a variety of cell types and disease states. With the spread of nonlinear optical microscopy techniques in biomedical research, NADH and FAD imaging has offered an attractive solution to noninvasively monitor cell and tissue status and elucidate dynamic changes in cell or tissue metabolism. Various tools and methods to measure the temporal, spectral, and spatial properties of NADH and FAD autofluorescence have been developed. Specifically, an optical redox ratio of cofactor fluorescence intensities and NADH fluorescence lifetime parameters have been used in numerous applications, but significant work remains to mature this technology for understanding dynamic changes in metabolism. This article describes the current understanding of our optical sensitivity to different metabolic pathways and highlights current challenges in the field. Recent progress in addressing these challenges and acquiring more quantitative information in faster and more metabolically relevant formats is also discussed.
Collapse
Affiliation(s)
- Irene Georgakoudi
- Department of Biomedical Engineering, Tufts University, Medford, Massachusetts, USA;
- Genetics, Molecular and Cellular Biology Program, Graduate School of Biomedical Sciences, Tufts University, Boston, Massachusetts, USA
| | - Kyle P Quinn
- Department of Biomedical Engineering and the Arkansas Integrative Metabolic Research Center, University of Arkansas, Fayetteville, Arkansas, USA
| |
Collapse
|
21
|
Smith JT, Sinsuebphon N, Rudkouskaya A, Michalet X, Intes X, Barroso M. in vivo quantitative FRET small animal imaging: intensity versus lifetime-based FRET. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.01.24.525411. [PMID: 36747671 PMCID: PMC9900789 DOI: 10.1101/2023.01.24.525411] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 01/26/2023]
Abstract
Förster Resonance Energy Transfer (FRET) microscopy is used in numerous biophysical and biomedical applications to monitor inter- and intramolecular interactions and conformational changes in the 2-10 nm range. FRET is currently being extended to in vivo optical imaging, its main application being in quantifying drug-target engagement or drug release in animal models of cancer using organic dye or nanoparticle-labeled probes. Herein, we compared FRET quantification using intensity-based FRET (sensitized emission FRET analysis with the 3-cube approach using an IVIS imager) and macroscopic fluorescence lifetime (MFLI) FRET using a custom system using a time-gated ICCD, for small animal optical in vivo imaging. The analytical expressions and experimental protocols required to quantify the product f D E of the FRET efficiency E and the fraction of donor molecules involved in FRET, f D , are described in detail for both methodologies. Dynamic in vivo FRET quantification of transferrin receptor-transferrin binding was acquired in live intact nude mice upon intravenous injection of near infrared-labeled transferrin FRET pair and benchmarked against in vitro FRET using hybridized oligonucleotides. Even though both in vivo imaging techniques provided similar dynamic trends for receptor-ligand engagement, we demonstrate that MFLI FRET has significant advantages. Whereas the sensitized emission FRET approach using the IVIS imager required 9 measurements (6 of which are used for calibration) acquired from three mice, MFLI FRET needed only one measurement collected from a single mouse, although a control mouse might be needed in a more general situation. Based on our study, MFLI therefore represents the method of choice for longitudinal preclinical FRET studies such as that of targeted drug delivery in intact, live mice.
Collapse
Affiliation(s)
- Jason T. Smith
- Center for Modeling, Simulation and Imaging in Medicine (CeMSIM), Rensselaer Polytechnic Institute, Troy, NY 12180, USA
- Present address: Elephas, 1 Erdman Pl., Madison, WI 53705, USA
| | - Nattawut Sinsuebphon
- Center for Modeling, Simulation and Imaging in Medicine (CeMSIM), Rensselaer Polytechnic Institute, Troy, NY 12180, USA
- Present address: Assistive Technology and Medical Devices Research Center, National Science and Technology Development Agency, 12120 Pathum Thani, Thailand
| | - Alena Rudkouskaya
- Department of Molecular and Cellular Physiology, Albany Medical College, Albany, NY 12208, USA
| | - Xavier Michalet
- Department of Chemistry & Biochemistry, University of California at Los Angeles, Los Angeles, California, CA 90095, USA
| | - Xavier Intes
- Center for Modeling, Simulation and Imaging in Medicine (CeMSIM), Rensselaer Polytechnic Institute, Troy, NY 12180, USA
| | - Margarida Barroso
- Department of Molecular and Cellular Physiology, Albany Medical College, Albany, NY 12208, USA
| |
Collapse
|
22
|
Zang Z, Xiao D, Wang Q, Jiao Z, Chen Y, Li DDU. Compact and robust deep learning architecture for fluorescence lifetime imaging and FPGA implementation. Methods Appl Fluoresc 2023; 11. [PMID: 36863024 DOI: 10.1088/2050-6120/acc0d9] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Accepted: 03/01/2023] [Indexed: 03/04/2023]
Abstract
This paper reports a bespoke adder-based deep learning network for time-domain fluorescence lifetime imaging (FLIM). By leveraging thel1-norm extraction method, we propose a 1D Fluorescence Lifetime AdderNet (FLAN) without multiplication-based convolutions to reduce the computational complexity. Further, we compressed fluorescence decays in temporal dimension using a log-scale merging technique to discard redundant temporal information derived as log-scaling FLAN (FLAN+LS). FLAN+LS achieves 0.11 and 0.23 compression ratios compared with FLAN and a conventional 1D convolutional neural network (1D CNN) while maintaining high accuracy in retrieving lifetimes. We extensively evaluated FLAN and FLAN+LS using synthetic and real data. A traditional fitting method and other non-fitting, high-accuracy algorithms were compared with our networks for synthetic data. Our networks attained a minor reconstruction error in different photon-count scenarios. For real data, we used fluorescent beads' data acquired by a confocal microscope to validate the effectiveness of real fluorophores, and our networks can differentiate beads with different lifetimes. Additionally, we implemented the network architecture on a field-programmable gate array (FPGA) with a post-quantization technique to shorten the bit-width, thereby improving computing efficiency. FLAN+LS on hardware achieves the highest computing efficiency compared to 1D CNN and FLAN. We also discussed the applicability of our network and hardware architecture for other time-resolved biomedical applications using photon-efficient, time-resolved sensors.
Collapse
Affiliation(s)
- Zhenya Zang
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G1 1XQ, United Kingdom
| | - Dong Xiao
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G1 1XQ, United Kingdom
| | - Quan Wang
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G1 1XQ, United Kingdom
| | - Ziao Jiao
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G1 1XQ, United Kingdom
| | - Yu Chen
- Department of Physics, University of Strathclyde, Glasgow G4 0NG, United Kingdom
| | - David Day Uei Li
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G1 1XQ, United Kingdom
| |
Collapse
|
23
|
Bene L, Damjanovich L. Deep-learning FRET visualization in flow cytometry: At the cross road of the signaling and FRET pathways. Cytometry A 2023; 103:177-183. [PMID: 36571425 DOI: 10.1002/cyto.a.24711] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2022] [Accepted: 12/10/2022] [Indexed: 12/27/2022]
Affiliation(s)
- László Bene
- Department of Surgery, Faculty of Medicine, University of Debrecen, Debrecen, Hungary
- Department of Biophysics and Cell Biology, Faculty of Medicine, University of Debrecen, Debrecen, Hungary
| | - László Damjanovich
- Department of Surgery, Faculty of Medicine, University of Debrecen, Debrecen, Hungary
| |
Collapse
|
24
|
Nizam NI, Ochoa M, Smith JT, Intes X. Deep learning-based fusion of widefield diffuse optical tomography and micro-CT structural priors for accurate 3D reconstructions. BIOMEDICAL OPTICS EXPRESS 2023; 14:1041-1053. [PMID: 36950248 PMCID: PMC10026582 DOI: 10.1364/boe.480091] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Revised: 01/10/2023] [Accepted: 01/24/2023] [Indexed: 06/17/2023]
Abstract
Widefield illumination and detection strategies leveraging structured light have enabled fast and robust probing of tissue properties over large surface areas and volumes. However, when applied to diffuse optical tomography (DOT) applications, they still require a time-consuming and expert-centric solving of an ill-posed inverse problem. Deep learning (DL) models have been recently proposed to facilitate this challenging step. Herein, we expand on a previously reported deep neural network (DNN) -based architecture (modified AUTOMAP - ModAM) for accurate and fast reconstructions of the absorption coefficient in 3D DOT based on a structured light illumination and detection scheme. Furthermore, we evaluate the improved performances when incorporating a micro-CT structural prior in the DNN-based workflow, named Z-AUTOMAP. This Z-AUTOMAP significantly improves the widefield imaging process's spatial resolution, especially in the transverse direction. The reported DL-based strategies are validated both in silico and in experimental phantom studies using spectral micro-CT priors. Overall, this is the first successful demonstration of micro-CT and DOT fusion using deep learning, greatly enhancing the prospect of rapid data-integration strategies, often demanded in challenging pre-clinical scenarios.
Collapse
|
25
|
Zhao Y, Raghuram A, Wang F, Kim SH, Hielscher A, Robinson JT, Veeraraghavan A. Unrolled-DOT: an interpretable deep network for diffuse optical tomography. JOURNAL OF BIOMEDICAL OPTICS 2023; 28:036002. [PMID: 36908760 PMCID: PMC9995139 DOI: 10.1117/1.jbo.28.3.036002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/19/2022] [Accepted: 02/09/2023] [Indexed: 06/18/2023]
Abstract
SIGNIFICANCE Imaging through scattering media is critical in many biomedical imaging applications, such as breast tumor detection and functional neuroimaging. Time-of-flight diffuse optical tomography (ToF-DOT) is one of the most promising methods for high-resolution imaging through scattering media. ToF-DOT and many traditional DOT methods require an image reconstruction algorithm. Unfortunately, this algorithm often requires long computational runtimes and may produce lower quality reconstructions in the presence of model mismatch or improper hyperparameter tuning. AIM We used a data-driven unrolled network as our ToF-DOT inverse solver. The unrolled network is faster than traditional inverse solvers and achieves higher reconstruction quality by accounting for model mismatch. APPROACH Our model "Unrolled-DOT" uses the learned iterative shrinkage thresholding algorithm. In addition, we incorporate a refinement U-Net and Visual Geometry Group (VGG) perceptual loss to further increase the reconstruction quality. We trained and tested our model on simulated and real-world data and benchmarked against physics-based and learning-based inverse solvers. RESULTS In experiments on real-world data, Unrolled-DOT outperformed learning-based algorithms and achieved over 10× reduction in runtime and mean-squared error, compared to traditional physics-based solvers. CONCLUSION We demonstrated a learning-based ToF-DOT inverse solver that achieves state-of-the-art performance in speed and reconstruction quality, which can aid in future applications for noninvasive biomedical imaging.
Collapse
Affiliation(s)
- Yongyi Zhao
- Rice University, Department of Electrical and Computer Engineering, Houston, Texas, United States
| | - Ankit Raghuram
- Rice University, Department of Electrical and Computer Engineering, Houston, Texas, United States
| | - Fay Wang
- Columbia University, Department of Biomedical Engineering, New York, New York, United States
| | - Stephen Hyunkeol Kim
- Columbia University Irvine Medical Center, Department of Radiology, New York, New York, United States
- New York University - Tandon School of Engineering, Department of Biomedical Engineering, New York, New York, United States
| | - Andreas Hielscher
- New York University - Tandon School of Engineering, Department of Biomedical Engineering, New York, New York, United States
| | - Jacob T. Robinson
- Rice University, Department of Electrical and Computer Engineering, Houston, Texas, United States
| | - Ashok Veeraraghavan
- Rice University, Department of Electrical and Computer Engineering, Houston, Texas, United States
| |
Collapse
|
26
|
Fazel M, Vallmitjana A, Scipioni L, Gratton E, Digman MA, Pressé S. Fluorescence lifetime: Beating the IRF and interpulse window. Biophys J 2023; 122:672-683. [PMID: 36659850 PMCID: PMC9989884 DOI: 10.1016/j.bpj.2023.01.014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Revised: 11/29/2022] [Accepted: 01/11/2023] [Indexed: 01/20/2023] Open
Abstract
Fluorescence lifetime imaging captures the spatial distribution of chemical species across cellular environments employing pulsed illumination confocal setups. However, quantitative interpretation of lifetime data continues to face critical challenges. For instance, fluorescent species with known in vitro excited-state lifetimes may split into multiple species with unique lifetimes when introduced into complex living environments. What is more, mixtures of species, which may be both endogenous and introduced into the sample, may exhibit 1) very similar lifetimes as well as 2) wide ranges of lifetimes including lifetimes shorter than the instrumental response function or whose duration may be long enough to be comparable to the interpulse window. By contrast, existing methods of analysis are optimized for well-separated and intermediate lifetimes. Here, we broaden the applicability of fluorescence lifetime analysis by simultaneously treating unknown mixtures of arbitrary lifetimes-outside the intermediate, Goldilocks, zone-for data drawn from a single confocal spot leveraging the tools of Bayesian nonparametrics (BNP). We benchmark our algorithm, termed BNP lifetime analysis, using a range of synthetic and experimental data. Moreover, we show that the BNP lifetime analysis method can distinguish and deduce lifetimes using photon counts as small as 500.
Collapse
Affiliation(s)
- Mohamadreza Fazel
- Center for Biological Physics, Arizona State University, Tempe, Arizona; Department of Physics, Arizona State University, Tempe, Arizona
| | - Alexander Vallmitjana
- Department of Biomedical Engineering, University of California Irvine, Irvine, California; Laboratory of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California Irvine, Irvine, California
| | - Lorenzo Scipioni
- Department of Biomedical Engineering, University of California Irvine, Irvine, California; Laboratory of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California Irvine, Irvine, California
| | - Enrico Gratton
- Department of Biomedical Engineering, University of California Irvine, Irvine, California; Laboratory of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California Irvine, Irvine, California
| | - Michelle A Digman
- Department of Biomedical Engineering, University of California Irvine, Irvine, California; Laboratory of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California Irvine, Irvine, California
| | - Steve Pressé
- Center for Biological Physics, Arizona State University, Tempe, Arizona; Department of Physics, Arizona State University, Tempe, Arizona; School of Molecular Science, Arizona State University, Tempe, Arizona.
| |
Collapse
|
27
|
Batista A, Guimarães P, Domingues JP, Quadrado MJ, Morgado AM. Two-Photon Imaging for Non-Invasive Corneal Examination. SENSORS (BASEL, SWITZERLAND) 2022; 22:9699. [PMID: 36560071 PMCID: PMC9783858 DOI: 10.3390/s22249699] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Revised: 12/02/2022] [Accepted: 12/04/2022] [Indexed: 06/17/2023]
Abstract
Two-photon imaging (TPI) microscopy, namely, two-photon excited fluorescence (TPEF), fluorescence lifetime imaging (FLIM), and second-harmonic generation (SHG) modalities, has emerged in the past years as a powerful tool for the examination of biological tissues. These modalities rely on different contrast mechanisms and are often used simultaneously to provide complementary information on morphology, metabolism, and structural properties of the imaged tissue. The cornea, being a transparent tissue, rich in collagen and with several cellular layers, is well-suited to be imaged by TPI microscopy. In this review, we discuss the physical principles behind TPI as well as its instrumentation. We also provide an overview of the current advances in TPI instrumentation and image analysis. We describe how TPI can be leveraged to retrieve unique information on the cornea and to complement the information provided by current clinical devices. The present state of corneal TPI is outlined. Finally, we discuss the obstacles that must be overcome and offer perspectives and outlooks to make clinical TPI of the human cornea a reality.
Collapse
Affiliation(s)
- Ana Batista
- Coimbra Institute for Biomedical Imaging and Translational Research (CIBIT), University of Coimbra, 3000-548 Coimbra, Portugal
- Institute for Nuclear Sciences Applied to Health (ICNAS), University of Coimbra, 3000-548 Coimbra, Portugal
- Department of Physics, Faculty of Science and Technology, University of Coimbra, 3004-516 Coimbra, Portugal
| | - Pedro Guimarães
- Coimbra Institute for Biomedical Imaging and Translational Research (CIBIT), University of Coimbra, 3000-548 Coimbra, Portugal
- Institute for Nuclear Sciences Applied to Health (ICNAS), University of Coimbra, 3000-548 Coimbra, Portugal
| | - José Paulo Domingues
- Coimbra Institute for Biomedical Imaging and Translational Research (CIBIT), University of Coimbra, 3000-548 Coimbra, Portugal
- Institute for Nuclear Sciences Applied to Health (ICNAS), University of Coimbra, 3000-548 Coimbra, Portugal
- Department of Physics, Faculty of Science and Technology, University of Coimbra, 3004-516 Coimbra, Portugal
| | - Maria João Quadrado
- Department of Ophthalmology, Centro Hospitalar e Universitário de Coimbra, 3004-561 Coimbra, Portugal
- Coimbra Institute for Clinical and Biomedical Research (iCBR), Faculty of Medicine, University of Coimbra, 3000-548 Coimbra, Portugal
| | - António Miguel Morgado
- Coimbra Institute for Biomedical Imaging and Translational Research (CIBIT), University of Coimbra, 3000-548 Coimbra, Portugal
- Institute for Nuclear Sciences Applied to Health (ICNAS), University of Coimbra, 3000-548 Coimbra, Portugal
- Department of Physics, Faculty of Science and Technology, University of Coimbra, 3004-516 Coimbra, Portugal
| |
Collapse
|
28
|
Ochoa M, Smith JT, Gao S, Intes X. Computational macroscopic lifetime imaging and concentration unmixing of autofluorescence. JOURNAL OF BIOPHOTONICS 2022; 15:e202200133. [PMID: 36546622 PMCID: PMC10026351 DOI: 10.1002/jbio.202200133] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/02/2022] [Revised: 07/06/2022] [Accepted: 07/12/2022] [Indexed: 06/17/2023]
Abstract
Single-pixel computational imaging can leverage highly sensitive detectors that concurrently acquire data across spectral and temporal domains. For molecular imaging, such methodology enables to collect rich intensity and lifetime multiplexed fluorescence datasets. Herein we report on the application of a single-pixel structured light-based platform for macroscopic imaging of tissue autofluorescence. The super-continuum visible excitation and hyperspectral single-pixel detection allow for parallel characterization of autofluorescence intensity and lifetime. Furthermore, we exploit a deep learning based data processing pipeline, to perform autofluorescence unmixing while yielding the autofluorophores' concentrations. The full scheme (setup and processing) is validated in silico and in vitro with clinically relevant autofluorophores flavin adenine dinucleotide, riboflavin, and protoporphyrin. The presented results demonstrate the potential of the methodology for macroscopically quantifying the intensity and lifetime of autofluorophores, with higher specificity for cases of mixed emissions, which are ubiquitous in autofluorescence and multiplexed in vivo imaging.
Collapse
Affiliation(s)
- Marien Ochoa
- Center for Modeling, Simulation and Imaging in Medicine (CeMSIM), Rensselaer Polytechnic Institute, Troy, New York, USA
| | - Jason T Smith
- Center for Modeling, Simulation and Imaging in Medicine (CeMSIM), Rensselaer Polytechnic Institute, Troy, New York, USA
| | - Shan Gao
- Center for Modeling, Simulation and Imaging in Medicine (CeMSIM), Rensselaer Polytechnic Institute, Troy, New York, USA
| | - Xavier Intes
- Center for Modeling, Simulation and Imaging in Medicine (CeMSIM), Rensselaer Polytechnic Institute, Troy, New York, USA
| |
Collapse
|
29
|
Rozowski M, Palumbo J, Bisen J, Bi C, Bouhrara M, Czaja W, Spencer RG. Input layer regularization for magnetic resonance relaxometry biexponential parameter estimation. MAGNETIC RESONANCE IN CHEMISTRY : MRC 2022; 60:1076-1086. [PMID: 35593385 PMCID: PMC10185331 DOI: 10.1002/mrc.5289] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/22/2021] [Revised: 04/22/2022] [Accepted: 05/17/2022] [Indexed: 05/17/2023]
Abstract
Many methods have been developed for estimating the parameters of biexponential decay signals, which arise throughout magnetic resonance relaxometry (MRR) and the physical sciences. This is an intrinsically ill-posed problem so that estimates can depend strongly on noise and underlying parameter values. Regularization has proven to be a remarkably efficient procedure for providing more reliable solutions to ill-posed problems, while, more recently, neural networks have been used for parameter estimation. We re-address the problem of parameter estimation in biexponential models by introducing a novel form of neural network regularization which we call input layer regularization (ILR). Here, inputs to the neural network are composed of a biexponential decay signal augmented by signals constructed from parameters obtained from a regularized nonlinear least-squares estimate of the two decay time constants. We find that ILR results in a reduction in the error of time constant estimates on the order of 15%-50% or more, depending on the metric used and signal-to-noise level, with greater improvement seen for the time constant of the more rapidly decaying component. ILR is compatible with existing regularization techniques and should be applicable to a wide range of parameter estimation problems.
Collapse
Affiliation(s)
- Michael Rozowski
- Applied Mathematics and Statistics, and Scientific Computation, University of Maryland, College Park, Maryland, USA
- Department of Mathematics, University of Maryland, College Park, Maryland, USA
| | - Jonathan Palumbo
- National Institute on Aging, National Institutes of Health, Baltimore, Maryland, USA
| | - Jay Bisen
- National Institute on Aging, National Institutes of Health, Baltimore, Maryland, USA
| | - Chuan Bi
- National Institute on Aging, National Institutes of Health, Baltimore, Maryland, USA
| | - Mustapha Bouhrara
- National Institute on Aging, National Institutes of Health, Baltimore, Maryland, USA
| | - Wojciech Czaja
- Department of Mathematics, University of Maryland, College Park, Maryland, USA
| | - Richard G. Spencer
- National Institute on Aging, National Institutes of Health, Baltimore, Maryland, USA
| |
Collapse
|
30
|
Ji M, Zhong J, Xue R, Su W, Kong Y, Fei Y, Ma J, Wang Y, Mi L. Early Detection of Cervical Cancer by Fluorescence Lifetime Imaging Microscopy Combined with Unsupervised Machine Learning. Int J Mol Sci 2022; 23:ijms231911476. [PMID: 36232778 PMCID: PMC9570424 DOI: 10.3390/ijms231911476] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2022] [Revised: 09/19/2022] [Accepted: 09/23/2022] [Indexed: 12/24/2022] Open
Abstract
Cervical cancer has high morbidity and mortality rates, affecting hundreds of thousands of women worldwide and requiring more accurate screening for early intervention and follow-up treatment. Cytology is the current dominant clinical screening approach, and though it has been used for decades, it has unsatisfactory sensitivity and specificity. In this work, fluorescence lifetime imaging microscopy (FLIM) was used for the imaging of exfoliated cervical cells in which an endogenous coenzyme involved in metabolism, namely, reduced nicotinamide adenine dinucleotide (phosphate) [NAD(P)H], was detected to evaluate the metabolic status of cells. FLIM images from 71 participants were analyzed by the unsupervised machine learning method to build a prediction model for cervical cancer risk. The FLIM method combined with unsupervised machine learning (FLIM-ML) had a sensitivity and specificity of 90.9% and 100%, respectively, significantly higher than those of the cytology approach. One cancer recurrence case was predicted as high-risk several months earlier using this method as compared to using current clinical methods, implying that FLIM-ML may be very helpful for follow-up cancer care. This study illustrates the clinical applicability of FLIM-ML as a detection method for cervical cancer screening and a convenient tool for follow-up cancer care.
Collapse
Affiliation(s)
- Mingmei Ji
- Department of Optical Science and Engineering, Shanghai Engineering Research Center of Ultra-Precision Optical Manufacturing, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), School of Information Science and Technology, Fudan University, 220 Handan Road, Shanghai 200433, China
| | - Jiahui Zhong
- Institute of Biomedical Engineering and Technology, Academy for Engineering and Technology, Fudan University, Shanghai 200433, China
| | - Runzhe Xue
- Department of Optical Science and Engineering, Shanghai Engineering Research Center of Ultra-Precision Optical Manufacturing, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), School of Information Science and Technology, Fudan University, 220 Handan Road, Shanghai 200433, China
| | - Wenhua Su
- Department of Optical Science and Engineering, Shanghai Engineering Research Center of Ultra-Precision Optical Manufacturing, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), School of Information Science and Technology, Fudan University, 220 Handan Road, Shanghai 200433, China
| | - Yawei Kong
- Department of Optical Science and Engineering, Shanghai Engineering Research Center of Ultra-Precision Optical Manufacturing, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), School of Information Science and Technology, Fudan University, 220 Handan Road, Shanghai 200433, China
| | - Yiyan Fei
- Department of Optical Science and Engineering, Shanghai Engineering Research Center of Ultra-Precision Optical Manufacturing, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), School of Information Science and Technology, Fudan University, 220 Handan Road, Shanghai 200433, China
| | - Jiong Ma
- Department of Optical Science and Engineering, Shanghai Engineering Research Center of Ultra-Precision Optical Manufacturing, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), School of Information Science and Technology, Fudan University, 220 Handan Road, Shanghai 200433, China
- Institute of Biomedical Engineering and Technology, Academy for Engineering and Technology, Fudan University, Shanghai 200433, China
- Shanghai Engineering Research Center of Industrial Microorganisms, The Multiscale Research Institute of Complex Systems (MRICS), School of Life Sciences, Fudan University, 220 Handan Road, Shanghai 200433, China
| | - Yulan Wang
- Department of Gynecology and Obstetrics, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, 26 Shengli Street, Wuhan 430014, China
- Correspondence: (Y.W.); (L.M.)
| | - Lan Mi
- Department of Optical Science and Engineering, Shanghai Engineering Research Center of Ultra-Precision Optical Manufacturing, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), School of Information Science and Technology, Fudan University, 220 Handan Road, Shanghai 200433, China
- Institute of Biomedical Engineering and Technology, Academy for Engineering and Technology, Fudan University, Shanghai 200433, China
- Correspondence: (Y.W.); (L.M.)
| |
Collapse
|
31
|
Wang Q, Li Y, Xiao D, Zang Z, Jiao Z, Chen Y, Li DDU. Simple and Robust Deep Learning Approach for Fast Fluorescence Lifetime Imaging. SENSORS (BASEL, SWITZERLAND) 2022; 22:7293. [PMID: 36236390 PMCID: PMC9572653 DOI: 10.3390/s22197293] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Revised: 09/17/2022] [Accepted: 09/22/2022] [Indexed: 06/16/2023]
Abstract
Fluorescence lifetime imaging (FLIM) is a powerful tool that provides unique quantitative information for biomedical research. In this study, we propose a multi-layer-perceptron-based mixer (MLP-Mixer) deep learning (DL) algorithm named FLIM-MLP-Mixer for fast and robust FLIM analysis. The FLIM-MLP-Mixer has a simple network architecture yet a powerful learning ability from data. Compared with the traditional fitting and previously reported DL methods, the FLIM-MLP-Mixer shows superior performance in terms of accuracy and calculation speed, which has been validated using both synthetic and experimental data. All results indicate that our proposed method is well suited for accurately estimating lifetime parameters from measured fluorescence histograms, and it has great potential in various real-time FLIM applications.
Collapse
Affiliation(s)
- Quan Wang
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G4 0RU, UK
| | - Yahui Li
- Key Laboratory of Ultra-Fast Photoelectric Diagnostics Technology, Xi’an Institute of Optics and Precision Mechanics, Xi’an 710049, China
| | - Dong Xiao
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G4 0RU, UK
| | - Zhenya Zang
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G4 0RU, UK
| | - Zi’ao Jiao
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G4 0RU, UK
| | - Yu Chen
- Department of Physics, University of Strathclyde, Glasgow G4 0NG, UK
| | - David Day Uei Li
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G4 0RU, UK
| |
Collapse
|
32
|
Masia F, Dewitte W, Borri P, Langbein W. uFLIM - Unsupervised analysis of FLIM-FRET microscopy data. Med Image Anal 2022; 82:102579. [PMID: 36049452 DOI: 10.1016/j.media.2022.102579] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2021] [Revised: 05/24/2022] [Accepted: 08/11/2022] [Indexed: 11/16/2022]
Abstract
Despite their widespread use in cell biology, fluorescence lifetime imaging microscopy (FLIM) data-sets are challenging to analyse, because each spatial position can contain a superposition of multiple fluorescent components. Here, we present a data analysis method employing all information in the available photon budget, as well as being fast. The method, called uFLIM, determines spatial distributions and temporal dynamics of multiple fluorescent components with no prior knowledge. It goes significantly beyond current approaches which either assume the functional dependence of the dynamics, e.g. an exponential decay, or require dynamics to be known, or calibrated. Its efficient non-negative matrix factorization algorithm allows for real-time data processing. We validate in silico that uFLIM is capable to disentangle the spatial distribution and spectral properties of five fluorescing probes, from only two excitation and detection channels and a photon budget of 100 detected photons per pixel. By adapting the method to data exhibiting Förster resonant energy transfer (FRET), we retrieve the spatial and transfer rate distribution of the bound species, without constrains on donor and acceptor dynamics.
Collapse
Affiliation(s)
- Francesco Masia
- School of Physics and Astronomy, Cardiff University, The Parade, Cardiff CF24 3AA, UK; School of Biosciences, Cardiff University, Museum Avenue, Cardiff CF10 3AX, UK.
| | - Walter Dewitte
- School of Biosciences, Cardiff University, Museum Avenue, Cardiff CF10 3AX, UK.
| | - Paola Borri
- School of Biosciences, Cardiff University, Museum Avenue, Cardiff CF10 3AX, UK.
| | - Wolfgang Langbein
- School of Physics and Astronomy, Cardiff University, The Parade, Cardiff CF24 3AA, UK.
| |
Collapse
|
33
|
Wu S, Yan Y, Hou H, Huang Z, Li D, Zhang X, Xiao Y. Polarity-Sensitive and Membrane-Specific Probe Quantitatively Monitoring Ferroptosis through Fluorescence Lifetime Imaging. Anal Chem 2022; 94:11238-11247. [PMID: 35926123 DOI: 10.1021/acs.analchem.2c01737] [Citation(s) in RCA: 23] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
As a new form of regulated cell death, ferroptosis is closely related to various diseases. To interpret this biological behavior and monitor related pathological processes, it is necessary to develop appropriate detection strategies and tools. Considering that ferroptosis is featured with remarkable lipid peroxidation of various cell membranes, it is logical to detect membranes' structural and environmental changes for the direct assessment of ferroptosis. For this sake, we designed novel polarity-sensitive fluorescent probes Mem-C1C18 and Mem-C18C18, which have superior plasma membrane anchorage, high brightness, and sensitive responses to environmental polarity by changing their fluorescence lifetimes. Mem-C1C18 with much less tendency to aggregate than Mem-C18C18 outperformed the latter in high resolution fluorescence labeling of artificial vesicle membranes and plasma membranes of live cells. Thus, Mem-C1C18 was selected to monitor plasma membranes damaged along ferroptosis process for the first time, in combination with the technique of fluorescence lifetime imaging (FLIM). After treating HeLa cells with Erastin, a typical ferroptosis inducer, the mean fluorescence lifetime of Mem-C1C18 displayed a considerable increase from 3.00 to 4.93 ns, with a 64% increase (corresponding to the polarity parameter Δf increased from 0.213 to 0.232). Therefore, our idea to utilize a probe to quantitate the changes in polarity of plasma membranes proves to be an effective method in the evaluation of the ferroptosis process.
Collapse
Affiliation(s)
- Shuyao Wu
- State Key Laboratory of Fine Chemicals, Dalian University of Technology, Dalian 116024, China
| | - Yu Yan
- State Key Laboratory of Fine Chemicals, Dalian University of Technology, Dalian 116024, China
| | - Haoran Hou
- State Key Laboratory of Fine Chemicals, Dalian University of Technology, Dalian 116024, China
| | - Zhenlong Huang
- State Key Laboratory of Fine Chemicals, Dalian University of Technology, Dalian 116024, China
| | - Dingxuan Li
- State Key Laboratory of Fine Chemicals, Dalian University of Technology, Dalian 116024, China
| | - Xinfu Zhang
- State Key Laboratory of Fine Chemicals, Dalian University of Technology, Dalian 116024, China
| | - Yi Xiao
- State Key Laboratory of Fine Chemicals, Dalian University of Technology, Dalian 116024, China
| |
Collapse
|
34
|
Young K, Ma E, Kejriwal S, Nielsen T, Aulakh SS, Birkeland AC. Intraoperative In Vivo Imaging Modalities in Head and Neck Cancer Surgical Margin Delineation: A Systematic Review. Cancers (Basel) 2022; 14:cancers14143416. [PMID: 35884477 PMCID: PMC9323577 DOI: 10.3390/cancers14143416] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2022] [Revised: 07/06/2022] [Accepted: 07/12/2022] [Indexed: 11/16/2022] Open
Abstract
Surgical margin status is one of the strongest prognosticators in predicting patient outcomes in head and neck cancer, yet head and neck surgeons continue to face challenges in the accurate detection of these margins with the current standard of care. Novel intraoperative imaging modalities have demonstrated great promise for potentially increasing the accuracy and efficiency in surgical margin delineation. In this current study, we collated and analyzed various intraoperative imaging modalities utilized in head and neck cancer to evaluate their use in discriminating malignant from healthy tissues. The authors conducted a systematic database search through PubMed/Medline, Web of Science, and EBSCOhost (CINAHL). Study screening and data extraction were performed and verified by the authors, and more studies were added through handsearching. Here, intraoperative imaging modalities are described, including optical coherence tomography, narrow band imaging, autofluorescence, and fluorescent-tagged probe techniques. Available sensitivities and specificities in delineating cancerous from healthy tissues ranged from 83.0% to 100.0% and 79.2% to 100.0%, respectively, across the different imaging modalities. Many of these initial studies are in small sample sizes, with methodological differences that preclude more extensive quantitative comparison. Thus, there is impetus for future larger studies examining and comparing the efficacy of these intraoperative imaging technologies.
Collapse
Affiliation(s)
- Kurtis Young
- John A. Burns School of Medicine, Honolulu, HI 96813, USA; (K.Y.); (E.M.); (S.K.); (T.N.)
| | - Enze Ma
- John A. Burns School of Medicine, Honolulu, HI 96813, USA; (K.Y.); (E.M.); (S.K.); (T.N.)
| | - Sameer Kejriwal
- John A. Burns School of Medicine, Honolulu, HI 96813, USA; (K.Y.); (E.M.); (S.K.); (T.N.)
| | - Torbjoern Nielsen
- John A. Burns School of Medicine, Honolulu, HI 96813, USA; (K.Y.); (E.M.); (S.K.); (T.N.)
| | | | - Andrew C. Birkeland
- Department of Otolaryngology—Head and Neck Surgery, University of California, Davis, CA 95817, USA
- Correspondence:
| |
Collapse
|
35
|
Xiao D, Zang Z, Wang Q, Jiao Z, Rocca FMD, Chen Y, Li DDU. Smart Wide-field Fluorescence Lifetime Imaging System with CMOS Single-photon Avalanche Diode Arrays. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:1887-1890. [PMID: 36086288 DOI: 10.1109/embc48229.2022.9870996] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Wide-field fluorescence lifetime imaging (FLIM) is a promising technique for biomedical and clinic applications. Integrating with CMOS single-photon avalanche diode (SPAD) sensor arrays can lead to cheaper and portable real-time FLIM systems. However, the FLIM data obtained by such sensor systems often have sophisticated noise features. There is still a lack of fast tools to recover lifetime parameters from highly noise-corrupted fluorescence signals efficiently. This paper proposes a smart wide-field FLIM system containing a 192×128 COMS SPAD sensor and a field-programmable gate array (FPGA) embedded deep learning (DL) FLIM processor. The processor adopts a hardware-friendly and light-weighted neural network for fluorescence lifetime analysis, showing the advantages of high accuracy against noise, fast speed, and low power consumption. Experimental results demonstrate the proposed system's superior and robust performances, promising for many FLIM applications such as FLIM-guided clinical surgeries, cancer diagnosis, and biomedical imaging.
Collapse
|
36
|
Zang Z, Xiao D, Wang Q, Jiao Z, Li Z, Chen Y, Li DDU. Hardware Inspired Neural Network for Efficient Time-Resolved Biomedical Imaging. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:1883-1886. [PMID: 36085638 DOI: 10.1109/embc48229.2022.9871214] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Convolutional neural networks (CNN) have revealed exceptional performance for fluorescence lifetime imaging (FLIM). However, redundant parameters and complicated topologies make it challenging to implement such networks on embedded hardware to achieve real-time processing. We report a lightweight, quantized neural architecture that can offer fast FLIM imaging. The forward-propagation is significantly simplified by replacing matrix multiplications in each convolution layer with additions and data quantization using a low bit-width. We first used synthetic 3-D lifetime data with given lifetime ranges and photon counts to assure correct average lifetimes can be obtained. Afterwards, human prostatic cancer cells incubated with gold nanoprobes were utilized to validate the feasibility of the network for real-world data. The quantized network yielded a 37.8% compression ratio without performance degradation. Clinical relevance - This neural network can be applied to diagnose cancer early based on fluorescence lifetime in a non-invasive way. This approach brings high accuracy and accelerates diagnostic processes for clinicians who are not experts in biomedical signal processing.
Collapse
|
37
|
Hu J, Sun Y, Geng X, Wang J, Guo Y, Qu L, Zhang K, Li Z. High-fidelity carbon dots polarity probes: revealing the heterogeneity of lipids in oncology. LIGHT, SCIENCE & APPLICATIONS 2022; 11:185. [PMID: 35718791 PMCID: PMC9207028 DOI: 10.1038/s41377-022-00873-x] [Citation(s) in RCA: 34] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/23/2022] [Revised: 05/10/2022] [Accepted: 06/03/2022] [Indexed: 05/07/2023]
Abstract
Polarity is an integral microenvironment parameter in biological systems closely associated with a multitude of cellular processes. Abnormal polarity variations accompany the initiation and development of pathophysiological processes. Thus, monitoring the abnormal polarity is of scientific and practical importance. Current state-of-the-art monitoring techniques are primarily based on fluorescence imaging which relies on a single emission intensity and may cause inaccurate detection due to heterogeneous accumulation of the probes. Herein, we report carbon dots (CDs) with ultra-sensitive responses to polarity. The CDs exhibit two linear relationships: one between fluorescence intensity and polarity and the other between polarity and the maximum emission wavelength. The emission spectrum is an intrinsic property of the probes, independent of the excitation intensity or probe concentration. These features enable two-color imaging/quantitation of polarity changes in lipid droplets (LDs) and in the cytoplasm via in situ emission spectroscopy. The probes reveal the polarity heterogeneity in LDs which can be applied to make a distinction between cancer and normal cells, and reveal the polarity homogeneity in cytoplasm.
Collapse
Affiliation(s)
- Jingyu Hu
- College of Chemistry, Institute of Analytical Chemistry for Life Science, Zhengzhou Key Laboratory of Functional Nanomaterial and Medical Theranostic, Zhengzhou University, 450001, Zhengzhou, China
| | - Yuanqiang Sun
- College of Chemistry, Institute of Analytical Chemistry for Life Science, Zhengzhou Key Laboratory of Functional Nanomaterial and Medical Theranostic, Zhengzhou University, 450001, Zhengzhou, China
| | - Xin Geng
- College of Chemistry, Institute of Analytical Chemistry for Life Science, Zhengzhou Key Laboratory of Functional Nanomaterial and Medical Theranostic, Zhengzhou University, 450001, Zhengzhou, China
| | - Junli Wang
- College of Chemistry, Institute of Analytical Chemistry for Life Science, Zhengzhou Key Laboratory of Functional Nanomaterial and Medical Theranostic, Zhengzhou University, 450001, Zhengzhou, China
| | - Yifei Guo
- College of Chemistry, Institute of Analytical Chemistry for Life Science, Zhengzhou Key Laboratory of Functional Nanomaterial and Medical Theranostic, Zhengzhou University, 450001, Zhengzhou, China
| | - Lingbo Qu
- College of Chemistry, Institute of Analytical Chemistry for Life Science, Zhengzhou Key Laboratory of Functional Nanomaterial and Medical Theranostic, Zhengzhou University, 450001, Zhengzhou, China
- Institute of Chemical Biology and Clinical Application at the First Affiliate Hospital, Zhengzhou University, 450001, Zhengzhou, China
| | - Ke Zhang
- Department of Chemistry and Chemical Biology, Northeastern University, Boston, MA, 02115, USA
| | - Zhaohui Li
- College of Chemistry, Institute of Analytical Chemistry for Life Science, Zhengzhou Key Laboratory of Functional Nanomaterial and Medical Theranostic, Zhengzhou University, 450001, Zhengzhou, China.
- Institute of Chemical Biology and Clinical Application at the First Affiliate Hospital, Zhengzhou University, 450001, Zhengzhou, China.
| |
Collapse
|
38
|
Zang Z, Xiao D, Wang Q, Li Z, Xie W, Chen Y, Li DDU. Fast Analysis of Time-Domain Fluorescence Lifetime Imaging via Extreme Learning Machine. SENSORS 2022; 22:s22103758. [PMID: 35632167 PMCID: PMC9146214 DOI: 10.3390/s22103758] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/22/2022] [Revised: 05/11/2022] [Accepted: 05/13/2022] [Indexed: 01/25/2023]
Abstract
We present a fast and accurate analytical method for fluorescence lifetime imaging microscopy (FLIM), using the extreme learning machine (ELM). We used extensive metrics to evaluate ELM and existing algorithms. First, we compared these algorithms using synthetic datasets. The results indicate that ELM can obtain higher fidelity, even in low-photon conditions. Afterwards, we used ELM to retrieve lifetime components from human prostate cancer cells loaded with gold nanosensors, showing that ELM also outperforms the iterative fitting and non-fitting algorithms. By comparing ELM with a computational efficient neural network, ELM achieves comparable accuracy with less training and inference time. As there is no back-propagation process for ELM during the training phase, the training speed is much higher than existing neural network approaches. The proposed strategy is promising for edge computing with online training.
Collapse
Affiliation(s)
- Zhenya Zang
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G4 0RE, UK; (Z.Z.); (D.X.); (Q.W.); (W.X.)
| | - Dong Xiao
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G4 0RE, UK; (Z.Z.); (D.X.); (Q.W.); (W.X.)
| | - Quan Wang
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G4 0RE, UK; (Z.Z.); (D.X.); (Q.W.); (W.X.)
| | - Zinuo Li
- Department of Physics, University of Strathclyde, Glasgow G4 0NG, UK; (Z.L.); (Y.C.)
| | - Wujun Xie
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G4 0RE, UK; (Z.Z.); (D.X.); (Q.W.); (W.X.)
| | - Yu Chen
- Department of Physics, University of Strathclyde, Glasgow G4 0NG, UK; (Z.L.); (Y.C.)
| | - David Day Uei Li
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G4 0RE, UK; (Z.Z.); (D.X.); (Q.W.); (W.X.)
- Correspondence:
| |
Collapse
|
39
|
Xiao D, Zang Z, Xie W, Sapermsap N, Chen Y, Uei Li DD. Spatial resolution improved fluorescence lifetime imaging via deep learning. OPTICS EXPRESS 2022; 30:11479-11494. [PMID: 35473091 DOI: 10.1364/oe.451215] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/07/2022] [Accepted: 03/12/2022] [Indexed: 06/14/2023]
Abstract
We present a deep learning approach to obtain high-resolution (HR) fluorescence lifetime images from low-resolution (LR) images acquired from fluorescence lifetime imaging (FLIM) systems. We first proposed a theoretical method for training neural networks to generate massive semi-synthetic FLIM data with various cellular morphologies, a sizeable dynamic lifetime range, and complex decay components. We then developed a degrading model to obtain LR-HR pairs and created a hybrid neural network, the spatial resolution improved FLIM net (SRI-FLIMnet) to simultaneously estimate fluorescence lifetimes and realize the nonlinear transformation from LR to HR images. The evaluative results demonstrate SRI-FLIMnet's superior performance in reconstructing spatial information from limited pixel resolution. We also verified SRI-FLIMnet using experimental images of bacterial infected mouse raw macrophage cells. Results show that the proposed data generation method and SRI-FLIMnet efficiently achieve superior spatial resolution for FLIM applications. Our study provides a solution for fast obtaining HR FLIM images.
Collapse
|
40
|
Fazel M, Jazani S, Scipioni L, Vallmitjana A, Gratton E, Digman MA, Pressé S. High Resolution Fluorescence Lifetime Maps from Minimal Photon Counts. ACS PHOTONICS 2022; 9:1015-1025. [PMID: 35847830 PMCID: PMC9278809 DOI: 10.1021/acsphotonics.1c01936] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/02/2023]
Abstract
Fluorescence lifetime imaging microscopy (FLIM) may reveal subcellular spatial lifetime maps of key molecular species. Yet, such a quantitative picture of life necessarily demands high photon budgets at every pixel under the current analysis paradigm, thereby increasing acquisition time and photodamage to the sample. Motivated by recent developments in computational statistics, we provide a direct means to update our knowledge of the lifetime maps of species of different lifetimes from direct photon arrivals, while accounting for experimental features such as arbitrary forms of the instrument response function (IRF) and exploiting information from empty laser pulses not resulting in photon detection. Our ability to construct lifetime maps holds for arbitrary lifetimes, from short lifetimes (comparable to the IRF) to lifetimes exceeding interpulse times. As our method is highly data efficient, for the same amount of data normally used to determine lifetimes and photon ratios, working within the Bayesian paradigm, we report direct blind unmixing of lifetimes with subnanosecond resolution and subpixel spatial resolution using standard raster scan FLIM images. We demonstrate our method using a wide range of simulated and experimental data.
Collapse
Affiliation(s)
- Mohamadreza Fazel
- Center
for Biological Physics, Department of Physics, Arizona State University, Tempe, Arizona 85287, United States
| | - Sina Jazani
- Center
for Biological Physics, Department of Physics, Arizona State University, Tempe, Arizona 85287, United States
| | - Lorenzo Scipioni
- Department
of Biomedical Engineering, University of
California Irvine, Irvine, California 92697, United States
- Laboratory
of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California, Irvine, California 92697, United States
| | - Alexander Vallmitjana
- Department
of Biomedical Engineering, University of
California Irvine, Irvine, California 92697, United States
- Laboratory
of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California, Irvine, California 92697, United States
| | - Enrico Gratton
- Department
of Biomedical Engineering, University of
California Irvine, Irvine, California 92697, United States
- Laboratory
of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California, Irvine, California 92697, United States
| | - Michelle A. Digman
- Department
of Biomedical Engineering, University of
California Irvine, Irvine, California 92697, United States
- Laboratory
of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California, Irvine, California 92697, United States
| | - Steve Pressé
- Center
for Biological Physics, Department of Physics, Arizona State University, Tempe, Arizona 85287, United States
- School
of Molecular Science, Arizona State University, Tempe, Arizona 85287, United States
- E-mail:
| |
Collapse
|
41
|
Label-free sensing of cells with fluorescence lifetime imaging: The quest for metabolic heterogeneity. Proc Natl Acad Sci U S A 2022; 119:2118241119. [PMID: 35217616 PMCID: PMC8892511 DOI: 10.1073/pnas.2118241119] [Citation(s) in RCA: 32] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/14/2022] [Indexed: 12/22/2022] Open
Abstract
Molecular, morphological, and physiological heterogeneity is the inherent property of cells which governs differences in their response to external influence. Tumor cell metabolic heterogeneity is of a special interest due to its clinical relevance to tumor progression and therapeutic outcomes. Rapid, sensitive, and noninvasive assessment of metabolic heterogeneity of cells is a great demand for biomedical sciences. Fluorescence lifetime imaging (FLIM), which is an all-optical technique, is an emerging tool for sensing and quantifying cellular metabolism by measuring fluorescence decay parameters of endogenous fluorophores, such as NAD(P)H. To achieve accurate discrimination between metabolically diverse cellular subpopulations, appropriate approaches to FLIM data collection and analysis are needed. In this paper, the unique capability of FLIM to attain the overarching goal of discriminating metabolic heterogeneity is demonstrated. This has been achieved using an approach to data analysis based on the nonparametric analysis, which revealed a much better sensitivity to the presence of metabolically distinct subpopulations compared to more traditional approaches of FLIM measurements and analysis. The approach was further validated for imaging cultured cancer cells treated with chemotherapy. These results pave the way for accurate detection and quantification of cellular metabolic heterogeneity using FLIM, which will be valuable for assessing therapeutic vulnerabilities and predicting clinical outcomes.
Collapse
|
42
|
Smith JT, Ochoa M, Faulkner D, Haskins G, Intes X. Deep learning in macroscopic diffuse optical imaging. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:JBO-210288VRR. [PMID: 35218169 PMCID: PMC8881080 DOI: 10.1117/1.jbo.27.2.020901] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/14/2021] [Accepted: 02/09/2022] [Indexed: 05/02/2023]
Abstract
SIGNIFICANCE Biomedical optics system design, image formation, and image analysis have primarily been guided by classical physical modeling and signal processing methodologies. Recently, however, deep learning (DL) has become a major paradigm in computational modeling and has demonstrated utility in numerous scientific domains and various forms of data analysis. AIM We aim to comprehensively review the use of DL applied to macroscopic diffuse optical imaging (DOI). APPROACH First, we provide a layman introduction to DL. Then, the review summarizes current DL work in some of the most active areas of this field, including optical properties retrieval, fluorescence lifetime imaging, and diffuse optical tomography. RESULTS The advantages of using DL for DOI versus conventional inverse solvers cited in the literature reviewed herein are numerous. These include, among others, a decrease in analysis time (often by many orders of magnitude), increased quantitative reconstruction quality, robustness to noise, and the unique capability to learn complex end-to-end relationships. CONCLUSIONS The heavily validated capability of DL's use across a wide range of complex inverse solving methodologies has enormous potential to bring novel DOI modalities, otherwise deemed impractical for clinical translation, to the patient's bedside.
Collapse
Affiliation(s)
- Jason T Smith
- Rensselaer Polytechnic Institute, Department of Biomedical Engineering, Troy, New York, United States
| | - Marien Ochoa
- Rensselaer Polytechnic Institute, Department of Biomedical Engineering, Troy, New York, United States
| | - Denzel Faulkner
- Rensselaer Polytechnic Institute, Department of Biomedical Engineering, Troy, New York, United States
| | - Grant Haskins
- Rensselaer Polytechnic Institute, Department of Biomedical Engineering, Troy, New York, United States
| | - Xavier Intes
- Rensselaer Polytechnic Institute, Center for Modeling, Simulation and Imaging for Medicine, Troy, Ne, United States
| |
Collapse
|
43
|
Chen YI, Chang YJ, Liao SC, Nguyen TD, Yang J, Kuo YA, Hong S, Liu YL, Rylander HG, Santacruz SR, Yankeelov TE, Yeh HC. Generative adversarial network enables rapid and robust fluorescence lifetime image analysis in live cells. Commun Biol 2022; 5:18. [PMID: 35017629 PMCID: PMC8752789 DOI: 10.1038/s42003-021-02938-w] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2021] [Accepted: 11/24/2021] [Indexed: 11/09/2022] Open
Abstract
Fluorescence lifetime imaging microscopy (FLIM) is a powerful tool to quantify molecular compositions and study molecular states in complex cellular environment as the lifetime readings are not biased by fluorophore concentration or excitation power. However, the current methods to generate FLIM images are either computationally intensive or unreliable when the number of photons acquired at each pixel is low. Here we introduce a new deep learning-based method termed flimGANE (fluorescence lifetime imaging based on Generative Adversarial Network Estimation) that can rapidly generate accurate and high-quality FLIM images even in the photon-starved conditions. We demonstrated our model is up to 2,800 times faster than the gold standard time-domain maximum likelihood estimation (TD_MLE) and that flimGANE provides a more accurate analysis of low-photon-count histograms in barcode identification, cellular structure visualization, Förster resonance energy transfer characterization, and metabolic state analysis in live cells. With its advantages in speed and reliability, flimGANE is particularly useful in fundamental biological research and clinical applications, where high-speed analysis is critical.
Collapse
Affiliation(s)
- Yuan-I Chen
- Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX, 78712, USA
| | - Yin-Jui Chang
- Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX, 78712, USA
| | - Shih-Chu Liao
- ISS, Inc., 1602 Newton Drive, Champaign, IL, 61822, USA
| | - Trung Duc Nguyen
- Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX, 78712, USA
| | - Jianchen Yang
- Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX, 78712, USA
| | - Yu-An Kuo
- Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX, 78712, USA
| | - Soonwoo Hong
- Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX, 78712, USA
| | - Yen-Liang Liu
- Master Program for Biomedical Engineering, China Medical University, Taichung, 406040, Taiwan
- Research Center for Cancer Biology, China Medical University, Taichung, 406040, Taiwan
| | - H Grady Rylander
- Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX, 78712, USA
| | - Samantha R Santacruz
- Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX, 78712, USA
- Institute for Neuroscience, The University of Texas at Austin, Austin, TX, 78712, USA
- Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, 78712, USA
| | - Thomas E Yankeelov
- Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX, 78712, USA
- Oden Institute for Computational Engineering and Sciences, The University of Texas at Austin, Austin, TX, 78712, USA
- Department of Diagnostic Medicine, The University of Texas at Austin, Austin, TX, 78712, USA
- Department of Oncology, The University of Texas at Austin, Austin, TX, 78712, USA
- Livestrong Cancer Institutes, The University of Texas at Austin, Austin, TX, 78712, USA
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, 77030, USA
| | - Hsin-Chih Yeh
- Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX, 78712, USA.
- Texas Materials Institute, The University of Texas at Austin, Austin, TX, 78712, USA.
| |
Collapse
|
44
|
Ochoa M, Rudkouskaya A, Smith JT, Intes X, Barroso M. Macroscopic Fluorescence Lifetime Imaging for Monitoring of Drug-Target Engagement. Methods Mol Biol 2022; 2394:837-856. [PMID: 35094361 PMCID: PMC8941982 DOI: 10.1007/978-1-0716-1811-0_44] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/01/2023]
Abstract
Precision medicine promises to improve therapeutic efficacy while reducing adverse effects, especially in oncology. However, despite great progresses in recent years, precision medicine for cancer treatment is not always part of routine care. Indeed, the ability to specifically tailor therapies to distinct patient profiles requires still significant improvements in targeted therapy development as well as decreases in drug treatment failures. In this regard, preclinical animal research is fundamental to advance our understanding of tumor biology, and diagnostic and therapeutic response. Most importantly, the ability to measure drug-target engagement accurately in live and intact animals is critical in guiding the development and optimization of targeted therapy. However, a major limitation of preclinical molecular imaging modalities is their lack of capability to directly and quantitatively discriminate between drug accumulation and drug-target engagement at the pathological site. Recently, we have developed Macroscopic Fluorescence Lifetime Imaging (MFLI) as a unique feature of optical imaging to quantitate in vivo drug-target engagement. MFLI quantitatively reports on nanoscale interactions via lifetime-sensing of Förster Resonance Energy Transfer (FRET) in live, intact animals. Hence, MFLI FRET acts as a direct reporter of receptor dimerization and target engagement via the measurement of the fraction of labeled-donor entity undergoing binding to its respective receptor. MFLI is expected to greatly impact preclinical imaging and also adjacent fields such as image-guided surgery and drug development.
Collapse
Affiliation(s)
- Marien Ochoa
- Department of Biomedical Engineering, Rensselaer Polytechnic Institute, Troy, NY, USA
| | - Alena Rudkouskaya
- Department of Cellular and Molecular Physiology, Albany Medical College, Albany, NY, USA
| | - Jason T Smith
- Department of Biomedical Engineering, Rensselaer Polytechnic Institute, Troy, NY, USA
| | - Xavier Intes
- Department of Biomedical Engineering, Rensselaer Polytechnic Institute, Troy, NY, USA
| | - Margarida Barroso
- Department of Cellular and Molecular Physiology, Albany Medical College, Albany, NY, USA.
| |
Collapse
|
45
|
Héliot L, Leray A. Simple phasor-based deep neural network for fluorescence lifetime imaging microscopy. Sci Rep 2021; 11:23858. [PMID: 34903737 PMCID: PMC8668934 DOI: 10.1038/s41598-021-03060-x] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2021] [Accepted: 11/23/2021] [Indexed: 12/29/2022] Open
Abstract
Fluorescence lifetime imaging microscopy (FLIM) is a powerful technique to probe the molecular environment of fluorophores. The analysis of FLIM images is usually performed with time consuming fitting methods. For accelerating this analysis, sophisticated deep learning architectures based on convolutional neural networks have been developed for restrained lifetime ranges but they require long training time. In this work, we present a simple neural network formed only with fully connected layers able to analyze fluorescence lifetime images. It is based on the reduction of high dimensional fluorescence intensity temporal decays into four parameters which are the phasor coordinates, the mean and amplitude-weighted lifetimes. This network called Phasor-Net has been applied for a time domain FLIM system excited with an 80 MHz laser repetition frequency, with negligible jitter and afterpulsing. Due to the restricted time interval of 12.5 ns, the training range of the lifetimes was limited between 0.2 and 3.0 ns; and the total photon number was lower than 106, as encountered in live cell imaging. From simulated biexponential decays, we demonstrate that Phasor-Net is more precise and less biased than standard fitting methods. We demonstrate also that this simple architecture gives almost comparable performance than those obtained from more sophisticated networks but with a faster training process (15 min instead of 30 min). We finally apply successfully our method to determine biexponential decays parameters for FLIM experiments in living cells expressing EGFP linked to mCherry and fused to a plasma membrane protein.
Collapse
Affiliation(s)
- Laurent Héliot
- PhLAM Laboratoire de Physique Des Lasers, Atomes Et Molécules, UMR 8523, CNRS, University of Lille, Lille, France.
| | - Aymeric Leray
- Laboratoire Interdisciplinaire Carnot de Bourgogne, UMR 6303, CNRS, Université de Bourgogne Franche-Comté, Dijon, France.
| |
Collapse
|
46
|
Zhao R, Wu D, Wen J, Zhang Q, Zhang G, Li J. Robustness and accuracy improvement of data processing with 2D neural networks for transient absorption dynamics. Phys Chem Chem Phys 2021; 23:16998-17008. [PMID: 34338705 DOI: 10.1039/d1cp02521j] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
Abstract
To achieve the goal of efficiently analyzing transient absorption spectra without arbitrary assumption and to overcome the limitations of conventional methods in fitting ability and highly noised backgrounds, it is essential to develop new tools to achieve more accurate and robust prediction based on the intrinsic properties of a spectrum even under strong noise. In this work, Lasso regression and neural network were combined to achieve an effective fitting. Compared to the conventional global fitting method, our network could automatically determine the exponential form on each wave unit, in which the accuracy was as high as 97%. Thereafter, the lifetime with the corresponding amplitude ratio could be easily predicted by the neural network on each wave unit. This kind of prediction is difficult to achieve by global fitting due to the limitation of computational resources. Furthermore, more accurate fitting even under weak signals could be achieved for the mean square error (MSE) decreasing by more than 100 times on average compared to conventional global fitting methods. Attributed to its improved accuracy and robustness, our developed algorithm could be readily applied to analyze time-resolved transient spectra.
Collapse
Affiliation(s)
- Ruixuan Zhao
- Institute of Medical Photonics, Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing 100191, P. R. China.
| | | | | | | | | | | |
Collapse
|
47
|
Tian L, Hunt B, Bell MAL, Yi J, Smith JT, Ochoa M, Intes X, Durr NJ. Deep Learning in Biomedical Optics. Lasers Surg Med 2021; 53:748-775. [PMID: 34015146 PMCID: PMC8273152 DOI: 10.1002/lsm.23414] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2021] [Revised: 04/02/2021] [Accepted: 04/15/2021] [Indexed: 01/02/2023]
Abstract
This article reviews deep learning applications in biomedical optics with a particular emphasis on image formation. The review is organized by imaging domains within biomedical optics and includes microscopy, fluorescence lifetime imaging, in vivo microscopy, widefield endoscopy, optical coherence tomography, photoacoustic imaging, diffuse tomography, and functional optical brain imaging. For each of these domains, we summarize how deep learning has been applied and highlight methods by which deep learning can enable new capabilities for optics in medicine. Challenges and opportunities to improve translation and adoption of deep learning in biomedical optics are also summarized. Lasers Surg. Med. © 2021 Wiley Periodicals LLC.
Collapse
Affiliation(s)
- L. Tian
- Department of Electrical and Computer Engineering, Boston University, Boston, MA, USA
| | - B. Hunt
- Thayer School of Engineering, Dartmouth College, Hanover, NH, USA
| | - M. A. L. Bell
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
- Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA
| | - J. Yi
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
- Department of Ophthalmology, Johns Hopkins University, Baltimore, MD, USA
| | - J. T. Smith
- Center for Modeling, Simulation, and Imaging in Medicine, Rensselaer Polytechnic Institute, Troy, New York NY 12180
| | - M. Ochoa
- Center for Modeling, Simulation, and Imaging in Medicine, Rensselaer Polytechnic Institute, Troy, New York NY 12180
| | - X. Intes
- Center for Modeling, Simulation, and Imaging in Medicine, Rensselaer Polytechnic Institute, Troy, New York NY 12180
| | - N. J. Durr
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
| |
Collapse
|
48
|
Li Y, Sapermsap N, Yu J, Tian J, Chen Y, Day-Uei Li D. Histogram clustering for rapid time-domain fluorescence lifetime image analysis. BIOMEDICAL OPTICS EXPRESS 2021; 12:4293-4307. [PMID: 34457415 PMCID: PMC8367240 DOI: 10.1364/boe.427532] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/13/2021] [Revised: 05/30/2021] [Accepted: 06/08/2021] [Indexed: 05/03/2023]
Abstract
We propose a histogram clustering (HC) method to accelerate fluorescence lifetime imaging (FLIM) analysis in pixel-wise and global fitting modes. The proposed method's principle was demonstrated, and the combinations of HC with traditional FLIM analysis were explained. We assessed HC methods with both simulated and experimental datasets. The results reveal that HC not only increases analysis speed (up to 106 times) but also enhances lifetime estimation accuracy. Fast lifetime analysis strategies were suggested with execution times around or below 30 μs per histograms on MATLAB R2016a, 64-bit with the Intel Celeron CPU (2950M @ 2GHz).
Collapse
Affiliation(s)
- Yahui Li
- Key Laboratory of Ultra-fast Photoelectric Diagnostics Technology, Xi'an Institute of Optics and Precision Mechanics, Xi'an Shaanxi 710049, China
- Collaborative Innovation Center of Extreme Optics, Shanxi University, Taiyuan Shanxi 030006, China
| | - Natakorn Sapermsap
- Department of Physics, Scottish Universities Physics Alliance, University of Strathclyde, Glasgow, G4 0NG, United Kingdom
| | - Jun Yu
- Strathclyde Institute of Pharmacy and Biomedical Sciences, University of Strathclyde, Glasgow G4 0RE, United Kingdom
| | - Jinshou Tian
- Key Laboratory of Ultra-fast Photoelectric Diagnostics Technology, Xi'an Institute of Optics and Precision Mechanics, Xi'an Shaanxi 710049, China
- Collaborative Innovation Center of Extreme Optics, Shanxi University, Taiyuan Shanxi 030006, China
| | - Yu Chen
- Department of Physics, Scottish Universities Physics Alliance, University of Strathclyde, Glasgow, G4 0NG, United Kingdom
| | - David Day-Uei Li
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G1 0NW, United Kingdom
| |
Collapse
|
49
|
Xiao D, Zang Z, Sapermsap N, Wang Q, Xie W, Chen Y, Uei Li DD. Dynamic fluorescence lifetime sensing with CMOS single-photon avalanche diode arrays and deep learning processors. BIOMEDICAL OPTICS EXPRESS 2021; 12:3450-3462. [PMID: 34221671 PMCID: PMC8221960 DOI: 10.1364/boe.425663] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/19/2021] [Revised: 05/07/2021] [Accepted: 05/10/2021] [Indexed: 06/13/2023]
Abstract
Measuring fluorescence lifetimes of fast-moving cells or particles have broad applications in biomedical sciences. This paper presents a dynamic fluorescence lifetime sensing (DFLS) system based on the time-correlated single-photon counting (TCSPC) principle. It integrates a CMOS 192 × 128 single-photon avalanche diode (SPAD) array, offering an enormous photon-counting throughput without pile-up effects. We also proposed a quantized convolutional neural network (QCNN) algorithm and designed a field-programmable gate array embedded processor for fluorescence lifetime determinations. The processor uses a simple architecture, showing unparallel advantages in accuracy, analysis speed, and power consumption. It can resolve fluorescence lifetimes against disturbing noise. We evaluated the DFLS system using fluorescence dyes and fluorophore-tagged microspheres. The system can effectively measure fluorescence lifetimes within a single exposure period of the SPAD sensor, paving the way for portable time-resolved devices and shows potential in various applications.
Collapse
Affiliation(s)
- Dong Xiao
- Strathclyde Institute of Pharmacy and Biomedical Sciences, University of Strathclyde, Glasgow G4 0RE, Scotland, UK
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G1 1XQ, Scotland, UK
| | - Zhenya Zang
- Strathclyde Institute of Pharmacy and Biomedical Sciences, University of Strathclyde, Glasgow G4 0RE, Scotland, UK
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G1 1XQ, Scotland, UK
| | - Natakorn Sapermsap
- Department of Physics, University of Strathclyde, Glasgow, G4 0RE, Scotland, UK
| | - Quan Wang
- Strathclyde Institute of Pharmacy and Biomedical Sciences, University of Strathclyde, Glasgow G4 0RE, Scotland, UK
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G1 1XQ, Scotland, UK
| | - Wujun Xie
- Strathclyde Institute of Pharmacy and Biomedical Sciences, University of Strathclyde, Glasgow G4 0RE, Scotland, UK
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G1 1XQ, Scotland, UK
| | - Yu Chen
- Department of Physics, University of Strathclyde, Glasgow, G4 0RE, Scotland, UK
| | - David Day Uei Li
- Strathclyde Institute of Pharmacy and Biomedical Sciences, University of Strathclyde, Glasgow G4 0RE, Scotland, UK
- Department of Biomedical Engineering, University of Strathclyde, Glasgow G1 1XQ, Scotland, UK
| |
Collapse
|
50
|
Alfonso-Garcia A, Bec J, Weyers B, Marsden M, Zhou X, Li C, Marcu L. Mesoscopic fluorescence lifetime imaging: Fundamental principles, clinical applications and future directions. JOURNAL OF BIOPHOTONICS 2021; 14:e202000472. [PMID: 33710785 PMCID: PMC8579869 DOI: 10.1002/jbio.202000472] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/27/2020] [Revised: 03/03/2021] [Accepted: 03/05/2021] [Indexed: 05/16/2023]
Abstract
Fluorescence lifetime imaging (FLIm) is an optical spectroscopic imaging technique capable of real-time assessments of tissue properties in clinical settings. Label-free FLIm is sensitive to changes in tissue structure and biochemistry resulting from pathological conditions, thus providing optical contrast to identify and monitor the progression of disease. Technical and methodological advances over the last two decades have enabled the development of FLIm instrumentation for real-time, in situ, mesoscopic imaging compatible with standard clinical workflows. Herein, we review the fundamental working principles of mesoscopic FLIm, discuss the technical characteristics of current clinical FLIm instrumentation, highlight the most commonly used analytical methods to interpret fluorescence lifetime data and discuss the recent applications of FLIm in surgical oncology and cardiovascular diagnostics. Finally, we conclude with an outlook on the future directions of clinical FLIm.
Collapse
Affiliation(s)
- Alba Alfonso-Garcia
- Department of Biomedical Engineering, University of California, Davis, Davis, California
| | - Julien Bec
- Department of Biomedical Engineering, University of California, Davis, Davis, California
| | - Brent Weyers
- Department of Biomedical Engineering, University of California, Davis, Davis, California
| | - Mark Marsden
- Department of Biomedical Engineering, University of California, Davis, Davis, California
| | - Xiangnan Zhou
- Department of Biomedical Engineering, University of California, Davis, Davis, California
| | - Cai Li
- Department of Biomedical Engineering, University of California, Davis, Davis, California
| | - Laura Marcu
- Department of Biomedical Engineering, University of California, Davis, Davis, California
- Department Neurological Surgery, University of California, Davis, California
| |
Collapse
|