1
|
Ming L, Zabala-Gutierrez I, Rodríguez-Sevilla P, Retama JR, Jaque D, Marin R, Ximendes E. Neural Networks Push the Limits of Luminescence Lifetime Nanosensing. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2023; 35:e2306606. [PMID: 37787978 DOI: 10.1002/adma.202306606] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/06/2023] [Revised: 09/18/2023] [Indexed: 10/04/2023]
Abstract
Luminescence lifetime-based sensing is ideally suited to monitor biological systems due to its minimal invasiveness and remote working principle. Yet, its applicability is limited in conditions of low signal-to-noise ratio (SNR) induced by, e.g., short exposure times and presence of opaque tissues. Herein this limitation is overcome by applying a U-shaped convolutional neural network (U-NET) to improve luminescence lifetime estimation under conditions of extremely low SNR. Specifically, the prowess of the U-NET is showcased in the context of luminescence lifetime thermometry, achieving more precise thermal readouts using Ag2 S nanothermometers. Compared to traditional analysis methods of decay curve fitting and integration, the U-NET can extract average lifetimes more precisely and consistently regardless of the SNR value. The improvement achieved in the sensing performance using the U-NET is demonstrated with two experiments characterized by extreme measurement conditions: thermal monitoring of free-falling droplets, and monitoring of thermal transients in suspended droplets through an opaque medium. These results broaden the applicability of luminescence lifetime-based sensing in fields including in vivo experimentation and microfluidics, while, hopefully, spurring further research on the implementation of machine learning (ML) in luminescence sensing.
Collapse
Affiliation(s)
- Liyan Ming
- Nanomaterials for Bioimaging Group (nanoBIG), Departamento de Física de Materiales, Facultad de Ciencias, Autonomous University of Madrid, Madrid, 28049, Spain
- Departamento de Química en Ciencias Farmacéuticas, Complutense University of Madrid, Madrid, 28040, Spain
| | - Irene Zabala-Gutierrez
- Nanomaterials for Bioimaging Group (nanoBIG), Instituto Ramón y Cajal de Investigación Sanitaria (IRYCIS), Hospital Ramón y Cajal, Madrid, 28034, Spain
| | - Paloma Rodríguez-Sevilla
- Nanomaterials for Bioimaging Group (nanoBIG), Departamento de Física de Materiales, Facultad de Ciencias, Autonomous University of Madrid, Madrid, 28049, Spain
| | - Jorge Rubio Retama
- Nanomaterials for Bioimaging Group (nanoBIG), Instituto Ramón y Cajal de Investigación Sanitaria (IRYCIS), Hospital Ramón y Cajal, Madrid, 28034, Spain
| | - Daniel Jaque
- Nanomaterials for Bioimaging Group (nanoBIG), Departamento de Física de Materiales, Facultad de Ciencias, Autonomous University of Madrid, Madrid, 28049, Spain
- Departamento de Química en Ciencias Farmacéuticas, Complutense University of Madrid, Madrid, 28040, Spain
- Institute for Advanced Research in Chemical Sciences (IAdChem), Autonomous University of Madrid, Madrid, 28049, Spain
| | - Riccardo Marin
- Nanomaterials for Bioimaging Group (nanoBIG), Departamento de Física de Materiales, Facultad de Ciencias, Autonomous University of Madrid, Madrid, 28049, Spain
- Institute for Advanced Research in Chemical Sciences (IAdChem), Autonomous University of Madrid, Madrid, 28049, Spain
| | - Erving Ximendes
- Nanomaterials for Bioimaging Group (nanoBIG), Departamento de Física de Materiales, Facultad de Ciencias, Autonomous University of Madrid, Madrid, 28049, Spain
- Departamento de Química en Ciencias Farmacéuticas, Complutense University of Madrid, Madrid, 28040, Spain
| |
Collapse
|
2
|
Fazel M, Jazani S, Scipioni L, Vallmitjana A, Zhu S, Gratton E, Digman MA, Pressé S. Building Fluorescence Lifetime Maps Photon-by-Photon by Leveraging Spatial Correlations. ACS PHOTONICS 2023; 10:3558-3569. [PMID: 38406580 PMCID: PMC10890823 DOI: 10.1021/acsphotonics.3c00595] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/27/2024]
Abstract
Fluorescence lifetime imaging microscopy (FLIM) has become a standard tool in the quantitative characterization of subcellular environments. However, quantitative FLIM analyses face several challenges. First, spatial correlations between pixels are often ignored as signal from individual pixels is analyzed independently thereby limiting spatial resolution. Second, existing methods deduce photon ratios instead of absolute lifetime maps. Next, the number of fluorophore species contributing to the signal is unknown, while excited state lifetimes with <1 ns difference are difficult to discriminate. Finally, existing analyses require high photon budgets and often cannot rigorously propagate experimental uncertainty into values over lifetime maps and number of species involved. To overcome all of these challenges simultaneously and self-consistently at once, we propose the first doubly nonparametric framework. That is, we learn the number of species (using Beta-Bernoulli process priors) and absolute maps of these fluorophore species (using Gaussian process priors) by leveraging information from pulses not leading to observed photon. We benchmark our framework using a broad range of synthetic and experimental data and demonstrate its robustness across a number of scenarios including cases where we recover lifetime differences between species as small as 0.3 ns with merely 1000 photons.
Collapse
Affiliation(s)
- Mohamadreza Fazel
- Center for Biological Physics and Department of Physics, Arizona State University, Tempe, Arizona 85287, United States
| | - Sina Jazani
- Center for Biological Physics and Department of Physics, Arizona State University, Tempe, Arizona 85287, United States
| | - Lorenzo Scipioni
- Department of Biomedical Engineering, University of California Irvine, Irvine, California 92697, United States; Laboratory of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California, Irvine, California 92697, United States
| | - Alexander Vallmitjana
- Department of Biomedical Engineering, University of California Irvine, Irvine, California 92697, United States; Laboratory of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California, Irvine, California 92697, United States
| | - Songning Zhu
- Department of Biomedical Engineering, University of California Irvine, Irvine, California 92697, United States; Laboratory of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California, Irvine, California 92697, United States
| | - Enrico Gratton
- Department of Biomedical Engineering, University of California Irvine, Irvine, California 92697, United States; Laboratory of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California, Irvine, California 92697, United States
| | - Michelle A Digman
- Department of Biomedical Engineering, University of California Irvine, Irvine, California 92697, United States; Laboratory of Fluorescence Dynamics, The Henry Samueli School of Engineering, University of California, Irvine, California 92697, United States
| | - Steve Pressé
- Center for Biological Physics and Department of Physics, Arizona State University, Tempe, Arizona 85287, United States; School of Molecular Science, Arizona State University, Tempe, Arizona 85287, United States
| |
Collapse
|
3
|
Thiele JC, Jungblut M, Helmerich DA, Tsukanov R, Chizhik A, Chizhik AI, Schnermann MJ, Sauer M, Nevskyi O, Enderlein J. Isotropic three-dimensional dual-color super-resolution microscopy with metal-induced energy transfer. SCIENCE ADVANCES 2022; 8:eabo2506. [PMID: 35675401 PMCID: PMC9176750 DOI: 10.1126/sciadv.abo2506] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/22/2022] [Accepted: 04/25/2022] [Indexed: 05/25/2023]
Abstract
Over the past two decades, super-resolution microscopy has seen a tremendous development in speed and resolution, but for most of its methods, there exists a remarkable gap between lateral and axial resolution, which is by a factor of 2 to 3 worse. One recently developed method to close this gap is metal-induced energy transfer (MIET) imaging, which achieves an axial resolution down to nanometers. It exploits the distance-dependent quenching of fluorescence when a fluorescent molecule is brought close to a metal surface. In the present manuscript, we combine the extreme axial resolution of MIET imaging with the extraordinary lateral resolution of single-molecule localization microscopy, in particular with direct stochastic optical reconstruction microscopy (dSTORM). This combination allows us to achieve isotropic three-dimensional super-resolution imaging of subcellular structures. Moreover, we used spectral demixing for implementing dual-color MIET-dSTORM that allows us to image and colocalize, in three dimensions, two different cellular structures simultaneously.
Collapse
Affiliation(s)
- Jan Christoph Thiele
- Third Institute of Physics–Biophysics, Georg August University, 37077 Göttingen, Germany
| | - Marvin Jungblut
- Department of Biotechnology and Biophysics, Biocenter, University of Würzburg, Am Hubland, 97074 Würzburg, Germany
| | - Dominic A. Helmerich
- Department of Biotechnology and Biophysics, Biocenter, University of Würzburg, Am Hubland, 97074 Würzburg, Germany
| | - Roman Tsukanov
- Third Institute of Physics–Biophysics, Georg August University, 37077 Göttingen, Germany
| | - Anna Chizhik
- Third Institute of Physics–Biophysics, Georg August University, 37077 Göttingen, Germany
| | - Alexey I. Chizhik
- Third Institute of Physics–Biophysics, Georg August University, 37077 Göttingen, Germany
| | - Martin J. Schnermann
- Chemical Biology Laboratory, Center for Cancer Research, National Cancer Institute, Frederick, MD 21702, USA
| | - Markus Sauer
- Department of Biotechnology and Biophysics, Biocenter, University of Würzburg, Am Hubland, 97074 Würzburg, Germany
| | - Oleksii Nevskyi
- Third Institute of Physics–Biophysics, Georg August University, 37077 Göttingen, Germany
| | - Jörg Enderlein
- Third Institute of Physics–Biophysics, Georg August University, 37077 Göttingen, Germany
- Cluster of Excellence “Multiscale Bioimaging: From Molecular Machines to Networks of Excitable Cells” (MBExC), Georg August University, Göttingen, Germany
| |
Collapse
|