1
|
Liu S, Chen J, Hellgoth J, Müller LR, Ferdman B, Karras C, Xiao D, Lidke KA, Heintzmann R, Shechtman Y, Li Y, Ries J. Universal inverse modeling of point spread functions for SMLM localization and microscope characterization. Nat Methods 2024; 21:1082-1093. [PMID: 38831208 DOI: 10.1038/s41592-024-02282-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2023] [Accepted: 04/16/2024] [Indexed: 06/05/2024]
Abstract
The point spread function (PSF) of a microscope describes the image of a point emitter. Knowing the accurate PSF model is essential for various imaging tasks, including single-molecule localization, aberration correction and deconvolution. Here we present universal inverse modeling of point spread functions (uiPSF), a toolbox to infer accurate PSF models from microscopy data, using either image stacks of fluorescent beads or directly images of blinking fluorophores, the raw data in single-molecule localization microscopy (SMLM). Our modular framework is applicable to a variety of microscope modalities and the PSF model incorporates system- or sample-specific characteristics, for example, the bead size, field- and depth- dependent aberrations, and transformations among channels. We demonstrate its application in single or multiple channels or large field-of-view SMLM systems, 4Pi-SMLM, and lattice light-sheet microscopes using either bead data or single-molecule blinking data.
Collapse
Affiliation(s)
- Sheng Liu
- Department of Physics and Astronomy, University of New Mexico, Albuquerque, NM, USA
- European Molecular Biology Laboratory, Cell Biology and Biophysics, Heidelberg, Germany
| | - Jianwei Chen
- Department of Biomedical Engineering, Guangdong Provincial Key Laboratory of Advanced Biomaterials, Southern University of Science and Technology, Shenzhen, China
- Collaboration for joint PhD degree between Southern University of Science and Technology and Harbin Institute of Technology, Harbin, China
| | - Jonas Hellgoth
- European Molecular Biology Laboratory, Cell Biology and Biophysics, Heidelberg, Germany
- Faculty of Biosciences, Collaboration for joint PhD degree from EMBL and Heidelberg University, Heidelberg, Germany
| | - Lucas-Raphael Müller
- European Molecular Biology Laboratory, Cell Biology and Biophysics, Heidelberg, Germany
- Machine Learning in Science, Excellence Cluster Machine Learning, University of Tübingen, Tübingen, Germany
| | - Boris Ferdman
- Department of Biomedical Engineering, Technion-Israel Institute of Technology, Haifa, Israel
| | - Christian Karras
- Leibniz Institute of Photonic Technology, Jena, Germany
- JENOPTIK Optical Systems, Jena, Germany
| | - Dafei Xiao
- Russell Berrie Nanotechnology Institute, Technion-Israel Institute of Technology, Haifa, Israel
| | - Keith A Lidke
- Department of Physics and Astronomy, University of New Mexico, Albuquerque, NM, USA
| | - Rainer Heintzmann
- Leibniz Institute of Photonic Technology, Jena, Germany
- Institute of Physical Chemistry and Abbe Center of Photonics, Friedrich-Schiller-University Jena, Jena, Germany
| | - Yoav Shechtman
- Department of Biomedical Engineering, Technion-Israel Institute of Technology, Haifa, Israel
- Department of Mechanical Engineering, The University of Texas at Austin, Austin, TX, USA
| | - Yiming Li
- Department of Biomedical Engineering, Guangdong Provincial Key Laboratory of Advanced Biomaterials, Southern University of Science and Technology, Shenzhen, China.
| | - Jonas Ries
- European Molecular Biology Laboratory, Cell Biology and Biophysics, Heidelberg, Germany.
- Max Perutz Labs, Vienna Biocenter Campus, Vienna, Austria.
- Department of Structural and Computational Biology, Center for Molecular Biology, University of Vienna, Vienna, Austria.
- Faculty of Physics, University of Vienna, Vienna, Austria.
| |
Collapse
|
2
|
Xiao D, Kedem Orange R, Opatovski N, Parizat A, Nehme E, Alalouf O, Shechtman Y. Large-FOV 3D localization microscopy by spatially variant point spread function generation. SCIENCE ADVANCES 2024; 10:eadj3656. [PMID: 38457497 PMCID: PMC10923516 DOI: 10.1126/sciadv.adj3656] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Accepted: 02/05/2024] [Indexed: 03/10/2024]
Abstract
Accurate characterization of the microscopic point spread function (PSF) is crucial for achieving high-performance localization microscopy (LM). Traditionally, LM assumes a spatially invariant PSF to simplify the modeling of the imaging system. However, for large fields of view (FOV) imaging, it becomes important to account for the spatially variant nature of the PSF. Here, we propose an accurate and fast principal components analysis-based field-dependent 3D PSF generator (PPG3D) and localizer for LM. Through simulations and experimental three-dimensional (3D) single-molecule localization microscopy (SMLM), we demonstrate the effectiveness of PPG3D, enabling super-resolution imaging of mitochondria and microtubules with high fidelity over a large FOV. A comparison of PPG3D with a shift-variant PSF generator for 3D LM reveals a threefold improvement in accuracy. Moreover, PPG3D is approximately 100 times faster than existing PSF generators, when used in image plane-based interpolation mode. Given its user-friendliness, we believe that PPG3D holds great potential for widespread application in SMLM and other imaging modalities.
Collapse
Affiliation(s)
- Dafei Xiao
- Russell Berrie Nanotechnology Institute, Technion—Israel Institute of Technology, Haifa, Israel
| | - Reut Kedem Orange
- Russell Berrie Nanotechnology Institute, Technion—Israel Institute of Technology, Haifa, Israel
| | - Nadav Opatovski
- Russell Berrie Nanotechnology Institute, Technion—Israel Institute of Technology, Haifa, Israel
| | - Amit Parizat
- Department of Biomedical Engineering, Technion—Israel Institute of Technology, Haifa, Israel
| | - Elias Nehme
- Department of Biomedical Engineering, Technion—Israel Institute of Technology, Haifa, Israel
- Department of Electrical and Computer Engineering, Technion—Israel Institute of Technology, Haifa, Israel
| | - Onit Alalouf
- Department of Biomedical Engineering, Technion—Israel Institute of Technology, Haifa, Israel
| | - Yoav Shechtman
- Russell Berrie Nanotechnology Institute, Technion—Israel Institute of Technology, Haifa, Israel
- Department of Biomedical Engineering, Technion—Israel Institute of Technology, Haifa, Israel
- Walker Department of Mechanical Engineering, The University of Texas at Austin, Austin, TX, USA
| |
Collapse
|
3
|
Liu S, Chen J, Hellgoth J, Müller LR, Ferdman B, Karras C, Xiao D, Lidke KA, Heintzmann R, Shechtman Y, Li Y, Ries J. Universal inverse modelling of point spread functions for SMLM localization and microscope characterization. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.10.26.564064. [PMID: 37961269 PMCID: PMC10634843 DOI: 10.1101/2023.10.26.564064] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/15/2023]
Abstract
The point spread function (PSF) of a microscope describes the image of a point emitter. Knowing the accurate PSF model is essential for various imaging tasks, including single molecule localization, aberration correction and deconvolution. Here we present uiPSF (universal inverse modelling of Point Spread Functions), a toolbox to infer accurate PSF models from microscopy data, using either image stacks of fluorescent beads or directly images of blinking fluorophores, the raw data in single molecule localization microscopy (SMLM). The resulting PSF model enables accurate 3D super-resolution imaging using SMLM. Additionally, uiPSF can be used to characterize and optimize a microscope system by quantifying the aberrations, including field-dependent aberrations, and resolutions. Our modular framework is applicable to a variety of microscope modalities and the PSF model incorporates system or sample specific characteristics, e.g., the bead size, depth dependent aberrations and transformations among channels. We demonstrate its application in single or multiple channels or large field-of-view SMLM systems, 4Pi-SMLM, and lattice light-sheet microscopes using either bead data or single molecule blinking data.
Collapse
Affiliation(s)
- Sheng Liu
- Department of Physics and Astronomy, University of New Mexico, Albuquerque, NM, USA
| | - Jianwei Chen
- Department of Biomedical Engineering, Guangdong Provincial Key Laboratory of Advanced Biomaterials, Southern University of Science and Technology, Shenzhen, China
- Collaboration for joint PhD degree between Southern University of Science and Technology and Harbin Institute of Technology, Harbin, 150001, China
| | - Jonas Hellgoth
- European Molecular Biology Laboratory, Cell Biology and Biophysics, Heidelberg, Germany
| | - Lucas-Raphael Müller
- European Molecular Biology Laboratory, Cell Biology and Biophysics, Heidelberg, Germany
| | - Boris Ferdman
- Department of Biomedical Engineering, Technion–Israel Institute of Technology, Haifa, Israel
| | - Christian Karras
- Leibniz Institute of Photonic Technology, Albert-Einstein-Straße 9, 07745, Jena, Germany
- Currently at JENOPTIK Optical Systems GmbH, Jena, Germany
| | - Dafei Xiao
- Department of Biomedical Engineering, Technion–Israel Institute of Technology, Haifa, Israel
| | - Keith A. Lidke
- Department of Physics and Astronomy, University of New Mexico, Albuquerque, NM, USA
| | - Rainer Heintzmann
- Institute of Physical Chemistry and Abbe Center of Photonics, Friedrich-Schiller-University Jena, Jena, Germany
- Leibniz Institute of Photonic Technology, Albert-Einstein-Straße 9, 07745, Jena, Germany
| | - Yoav Shechtman
- Department of Biomedical Engineering, Technion–Israel Institute of Technology, Haifa, Israel
- Department of Mechanical Engineering, The University of Texas at Austin, Austin, TX, USA
| | - Yiming Li
- Department of Biomedical Engineering, Guangdong Provincial Key Laboratory of Advanced Biomaterials, Southern University of Science and Technology, Shenzhen, China
| | - Jonas Ries
- European Molecular Biology Laboratory, Cell Biology and Biophysics, Heidelberg, Germany
- Max Perutz Labs, Vienna Biocenter Campus (VBC), Dr.-Bohr-Gasse 9, 1030, Vienna, Austria
- University of Vienna, Center for Molecular Biology, Department of Structural and Computational Biology, Dr.-Bohr-Gasse 9, 1030, Vienna, Austria
- University of Vienna, Faculty of Physics, Boltzmanngasse 5, 1090 Vienna, Austria
| |
Collapse
|
4
|
Fazel M, Grussmayer KS, Ferdman B, Radenovic A, Shechtman Y, Enderlein J, Pressé S. Fluorescence Microscopy: a statistics-optics perspective. ARXIV 2023:arXiv:2304.01456v3. [PMID: 37064525 PMCID: PMC10104198] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 04/18/2023]
Abstract
Fundamental properties of light unavoidably impose features on images collected using fluorescence microscopes. Modeling these features is ever more important in quantitatively interpreting microscopy images collected at scales on par or smaller than light's wavelength. Here we review the optics responsible for generating fluorescent images, fluorophore properties, microscopy modalities leveraging properties of both light and fluorophores, in addition to the necessarily probabilistic modeling tools imposed by the stochastic nature of light and measurement.
Collapse
Affiliation(s)
- Mohamadreza Fazel
- Department of Physics, Arizona State University, Tempe, Arizona, USA
- Center for Biological Physics, Arizona State University, Tempe, Arizona, USA
| | - Kristin S Grussmayer
- Department of Bionanoscience, Faculty of Applied Science and Kavli Institute for Nanoscience, Delft University of Technology, Delft, Netherlands
| | - Boris Ferdman
- Russel Berrie Nanotechnology Institute and Department of Biomedical Engineering, Technion - Israel Institute of Technology, Haifa, Israel
| | - Aleksandra Radenovic
- Laboratory of Nanoscale Biology, Institute of Bioengineering, Ecole Polytechnique Federale de Lausanne (EPFL), Lausanne, Switzerland
| | - Yoav Shechtman
- Russel Berrie Nanotechnology Institute and Department of Biomedical Engineering, Technion - Israel Institute of Technology, Haifa, Israel
| | - Jörg Enderlein
- III. Institute of Physics - Biophysics, Georg August University, Göttingen, Germany
| | - Steve Pressé
- Department of Physics, Arizona State University, Tempe, Arizona, USA
- Center for Biological Physics, Arizona State University, Tempe, Arizona, USA
| |
Collapse
|
5
|
Zhou Z, Wu J, Wang Z, Huang ZL. Deep learning using a residual deconvolutional network enables real-time high-density single-molecule localization microscopy. BIOMEDICAL OPTICS EXPRESS 2023; 14:1833-1847. [PMID: 37078057 PMCID: PMC10110325 DOI: 10.1364/boe.484540] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Revised: 03/03/2023] [Accepted: 03/17/2023] [Indexed: 05/03/2023]
Abstract
High-density localization based on deep learning is a very effective method to accelerate single molecule localization microscopy (SMLM). Compared with traditional high-density localization methods, deep learning-based methods enable a faster data processing speed and a higher localization accuracy. However, the reported high-density localization methods based on deep learning are still not fast enough to enable real time data processing for large batches of raw images, which is probably due to the heavy computational burden and computation complexity in the U-shape architecture used in these models. Here we propose a high-density localization method called FID-STORM, which is based on an improved residual deconvolutional network for the real-time processing of raw images. In FID-STORM, we use a residual network to extract the features directly from low-resolution raw images rather than the U-shape network from interpolated images. We also use a model fusion from TensorRT to further accelerate the inference of the model. In addition, we process the sum of the localization images directly on GPU to obtain an additional speed gain. Using simulated and experimental data, we verified that the FID-STORM method achieves a processing speed of 7.31 ms/frame at 256 × 256 pixels @ Nvidia RTX 2080 Ti graphic card, which is shorter than the typical exposure time of 10∼30 ms, thus enabling real-time data processing in high-density SMLM. Moreover, compared with a popular interpolated image-based method called Deep-STORM, FID-STORM enables a speed gain of ∼26 times, without loss of reconstruction accuracy. We also provided an ImageJ plugin for our new method.
Collapse
Affiliation(s)
- Zhiwei Zhou
- Britton Chance Center and MoE Key Laboratory for Biomedical Photonics, School of Engineering Sciences, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan 430074, China
| | - Junnan Wu
- Key laboratory of Biomedical Engineering of Hainan Province, School of Biomedical Engineering, Hainan University, Haikou 570228, China
| | - Zhengxia Wang
- School of Computer Science and Technology, Hainan University, Haikou 570228, China
| | - Zhen-Li Huang
- Key laboratory of Biomedical Engineering of Hainan Province, School of Biomedical Engineering, Hainan University, Haikou 570228, China
| |
Collapse
|
6
|
Alemán-Castañeda LA, Feng SYT, Gutiérrez-Cuevas R, Herrera I, Brown TG, Brasselet S, Alonso MA. Using fluorescent beads to emulate single fluorophores. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2022; 39:C167-C178. [PMID: 36520768 DOI: 10.1364/josaa.474837] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/02/2022] [Accepted: 11/12/2022] [Indexed: 06/17/2023]
Abstract
We study the conditions under which fluorescent beads can be used to emulate single fluorescent molecules in the calibration of optical microscopes. Although beads are widely used due to their brightness and easy manipulation, there can be notable differences between the point spread functions (PSFs) they produce and those for single-molecule fluorophores, caused by their different emission patterns and sizes. We study theoretically these differences for various scenarios, e.g., with or without polarization channel splitting, to determine the conditions under which the use of beads as a model for single molecules is valid. We also propose methods to model the blurring due to the size difference and compensate for it to produce PSFs that are more similar to those for single molecules.
Collapse
|
7
|
Jusuf JM, Lew MD. Towards optimal point spread function design for resolving closely spaced emitters in three dimensions. OPTICS EXPRESS 2022; 30:37154-37174. [PMID: 36258632 DOI: 10.1364/oe.472067] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/03/2022] [Accepted: 09/12/2022] [Indexed: 06/16/2023]
Abstract
The past decade has brought many innovations in optical design for 3D super-resolution imaging of point-like emitters, but these methods often focus on single-emitter localization precision as a performance metric. Here, we propose a simple heuristic for designing a point spread function (PSF) that allows for precise measurement of the distance between two emitters. We discover that there are two types of PSFs that achieve high performance for resolving emitters in 3D, as quantified by the Cramér-Rao bounds for estimating the separation between two closely spaced emitters. One PSF is very similar to the existing Tetrapod PSFs; the other is a rotating single-spot PSF, which we call the crescent PSF. The latter exhibits excellent performance for localizing single emitters throughout a 1-µm focal volume (localization precisions of 7.3 nm in x, 7.7 nm in y, and 18.3 nm in z using 1000 detected photons), and it distinguishes between one and two closely spaced emitters with superior accuracy (25-53% lower error rates than the best-performing Tetrapod PSF, averaged throughout a 1-µm focal volume). Our study provides additional insights into optimal strategies for encoding 3D spatial information into optical PSFs.
Collapse
|
8
|
Toader B, Boulanger J, Korolev Y, Lenz MO, Manton J, Schönlieb CB, Mureşan L. Image Reconstruction in Light-Sheet Microscopy: Spatially Varying Deconvolution and Mixed Noise. JOURNAL OF MATHEMATICAL IMAGING AND VISION 2022; 64:968-992. [PMID: 36329880 PMCID: PMC7613773 DOI: 10.1007/s10851-022-01100-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/08/2021] [Accepted: 04/23/2022] [Indexed: 06/16/2023]
Abstract
We study the problem of deconvolution for light-sheet microscopy, where the data is corrupted by spatially varying blur and a combination of Poisson and Gaussian noise. The spatial variation of the point spread function of a light-sheet microscope is determined by the interaction between the excitation sheet and the detection objective PSF. We introduce a model of the image formation process that incorporates this interaction and we formulate a variational model that accounts for the combination of Poisson and Gaussian noise through a data fidelity term consisting of the infimal convolution of the single noise fidelities, first introduced in L. Calatroni et al. (SIAM J Imaging Sci 10(3):1196-1233, 2017). We establish convergence rates and a discrepancy principle for the infimal convolution fidelity and the inverse problem is solved by applying the primal-dual hybrid gradient (PDHG) algorithm in a novel way. Numerical experiments performed on simulated and real data show superior reconstruction results in comparison with other methods.
Collapse
Affiliation(s)
- Bogdan Toader
- Cambridge Advanced Imaging Centre, University of Cambridge, Anatomy School, Downing Street, Cambridge, CB2 3DY UK
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Wilberforce Road, Cambridge, CB3 0WA UK
- Department of Physiology, Development and Neuroscience, University of Cambridge, Downing Street, Cambridge, CB2 3DY UK
| | - Jérôme Boulanger
- MRC Laboratory of Molecular Biology, Francis Crick Avenue, Cambridge, CB2 0QH UK
| | - Yury Korolev
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Wilberforce Road, Cambridge, CB3 0WA UK
| | - Martin O. Lenz
- Cambridge Advanced Imaging Centre, University of Cambridge, Anatomy School, Downing Street, Cambridge, CB2 3DY UK
- Sainsbury Laboratory, University of Cambridge, 47 Bateman Street, Cambridge, CB2 1LR UK
| | - James Manton
- MRC Laboratory of Molecular Biology, Francis Crick Avenue, Cambridge, CB2 0QH UK
| | - Carola-Bibiane Schönlieb
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Wilberforce Road, Cambridge, CB3 0WA UK
| | - Leila Mureşan
- Cambridge Advanced Imaging Centre, University of Cambridge, Anatomy School, Downing Street, Cambridge, CB2 3DY UK
- Department of Physiology, Development and Neuroscience, University of Cambridge, Downing Street, Cambridge, CB2 3DY UK
- Sainsbury Laboratory, University of Cambridge, 47 Bateman Street, Cambridge, CB2 1LR UK
| |
Collapse
|
9
|
Wang F, Li H, Xiao Y, Zhao M, Zhang Y. Phase optimization algorithm for 3D particle localization with large axial depth. OPTICS LETTERS 2022; 47:182-185. [PMID: 34951918 DOI: 10.1364/ol.446947] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/28/2021] [Accepted: 11/29/2021] [Indexed: 06/14/2023]
Abstract
We propose an optimization algorithm based on Fresnel approximation (FA) imaging to optimize an extended-axial-depth point spread function (PSF) for 3D particle localization. The transfer function efficiency of the PSF is improved by repeatedly imposing constraints in the object plane, the spatial domain, and the Fourier domain. During the iterative calculation, the effective photon number or Cramer-Rao lower bound is used as the termination condition of the iteration. The algorithm allows flexible adjustment of the peak intensity ratio of the two main lobes. Moreover, the transfer function efficiency can be balanced by increasing the weight of the modulation function of the expected PSF at each axial position. The twin-Airy (TA) PSF optimized by the FA optimization algorithm does not require complex post-processing, whereas post-processing is an essential step for the unoptimized TA-PSF. The optimization algorithm is significant for extended-axial-depth PSFs used for 3D particle localization, as it improves localization precision and temporal resolution.
Collapse
|
10
|
Copeland CR, McGray CD, Ilic BR, Geist J, Stavis SM. Accurate localization microscopy by intrinsic aberration calibration. Nat Commun 2021; 12:3925. [PMID: 34168121 PMCID: PMC8225824 DOI: 10.1038/s41467-021-23419-y] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2020] [Accepted: 04/28/2021] [Indexed: 02/02/2023] Open
Abstract
A standard paradigm of localization microscopy involves extension from two to three dimensions by engineering information into emitter images, and approximation of errors resulting from the field dependence of optical aberrations. We invert this standard paradigm, introducing the concept of fully exploiting the latent information of intrinsic aberrations by comprehensive calibration of an ordinary microscope, enabling accurate localization of single emitters in three dimensions throughout an ultrawide and deep field. To complete the extraction of spatial information from microscale bodies ranging from imaging substrates to microsystem technologies, we introduce a synergistic concept of the rigid transformation of the positions of multiple emitters in three dimensions, improving precision, testing accuracy, and yielding measurements in six degrees of freedom. Our study illuminates the challenge of aberration effects in localization microscopy, redefines the challenge as an opportunity for accurate, precise, and complete localization, and elucidates the performance and reliability of a complex microelectromechanical system.
Collapse
Affiliation(s)
- Craig R Copeland
- Microsystems and Nanotechnology Division, National Institute of Standards and Technology, Gaithersburg, MD, USA
| | - Craig D McGray
- Quantum Measurement Division, National Institute of Standards and Technology, Gaithersburg, MD, USA
| | - B Robert Ilic
- Microsystems and Nanotechnology Division, National Institute of Standards and Technology, Gaithersburg, MD, USA
- CNST NanoFab, National Institute of Standards and Technology, Gaithersburg, MD, USA
| | - Jon Geist
- Quantum Measurement Division, National Institute of Standards and Technology, Gaithersburg, MD, USA
| | - Samuel M Stavis
- Microsystems and Nanotechnology Division, National Institute of Standards and Technology, Gaithersburg, MD, USA.
| |
Collapse
|
11
|
Mazidi H, Ding T, Nehorai A, Lew MD. Quantifying accuracy and heterogeneity in single-molecule super-resolution microscopy. Nat Commun 2020; 11:6353. [PMID: 33311471 PMCID: PMC7732856 DOI: 10.1038/s41467-020-20056-9] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2020] [Accepted: 11/10/2020] [Indexed: 12/03/2022] Open
Abstract
The resolution and accuracy of single-molecule localization microscopes (SMLMs) are routinely benchmarked using simulated data, calibration rulers, or comparisons to secondary imaging modalities. However, these methods cannot quantify the nanoscale accuracy of an arbitrary SMLM dataset. Here, we show that by computing localization stability under a well-chosen perturbation with accurate knowledge of the imaging system, we can robustly measure the confidence of individual localizations without ground-truth knowledge of the sample. We demonstrate that our method, termed Wasserstein-induced flux (WIF), measures the accuracy of various reconstruction algorithms directly on experimental 2D and 3D data of microtubules and amyloid fibrils. We further show that WIF confidences can be used to evaluate the mismatch between computational models and imaging data, enhance the accuracy and resolution of reconstructed structures, and discover hidden molecular heterogeneities. As a computational methodology, WIF is broadly applicable to any SMLM dataset, imaging system, and localization algorithm. Standard benchmarking of single-molecule localization microscopy cannot quantify nanoscale accuracy of arbitrary datasets. Here, the authors present Wasserstein-induced flux, a method using a chosen perturbation and knowledge of the imaging system to measure confidence of individual localizations.
Collapse
Affiliation(s)
- Hesam Mazidi
- Department of Electrical and Systems Engineering, Washington University in St. Louis, St. Louis, MO, 63130, USA
| | - Tianben Ding
- Department of Electrical and Systems Engineering, Washington University in St. Louis, St. Louis, MO, 63130, USA
| | - Arye Nehorai
- Department of Electrical and Systems Engineering, Washington University in St. Louis, St. Louis, MO, 63130, USA
| | - Matthew D Lew
- Department of Electrical and Systems Engineering, Washington University in St. Louis, St. Louis, MO, 63130, USA.
| |
Collapse
|
12
|
Vahid MR, Hanzon B, Ober RJ. Effect of Pixelation on the Parameter Estimation of Single Molecule Trajectories. IEEE TRANSACTIONS ON COMPUTATIONAL IMAGING 2020; 7:98-113. [PMID: 33604418 PMCID: PMC7879562 DOI: 10.1109/tci.2020.3039951] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/16/2019] [Revised: 08/13/2020] [Accepted: 11/15/2020] [Indexed: 06/12/2023]
Abstract
The advent of single molecule microscopy has revolutionized biological investigations by providing a powerful tool for the study of intercellular and intracellular trafficking processes of protein molecules which was not available before through conventional microscopy. In practice, pixelated detectors are used to acquire the images of fluorescently labeled objects moving in cellular environments. Then, the acquired fluorescence microscopy images contain the numbers of the photons detected in each pixel, during an exposure time interval. Moreover, instead of having the exact locations of detection of the photons, we only know the pixel areas in which the photons impact the detector. These challenges make the analysis of single molecule trajectories, from pixelated images, a complex problem. Here, we investigate the effect of pixelation on the parameter estimation of single molecule trajectories. In particular, we develop a stochastic framework to calculate the maximum likelihood estimates of the parameters of a stochastic differential equation that describes the motion of the molecule in living cells. We also calculate the Fisher information matrix for this parameter estimation problem. The analytical results are complicated through the fact that the observation process in a microscope prohibits the use of standard Kalman filter type approaches. The analytical framework presented here is illustrated with examples of low photon count scenarios for which we rely on Monte Carlo methods to compute the associated probability distributions.
Collapse
Affiliation(s)
- Milad R. Vahid
- Department of Biomedical EngineeringTexas A&M UniversityCollege StationTX77843USA
- Department of Biomedical Data ScienceStanford UniversityStanfordCA94305USA
| | - Bernard Hanzon
- Department of MathematicsUniversity College CorkT12YX86CorkIreland
| | - Raimund J. Ober
- Centre for Cancer ImmunologyFaculty of Medicine, University of SouthamptonSouthamptonSO16 6YDU.K.
| |
Collapse
|
13
|
Shechtman Y. Recent advances in point spread function engineering and related computational microscopy approaches: from one viewpoint. Biophys Rev 2020; 12:10.1007/s12551-020-00773-7. [PMID: 33210213 PMCID: PMC7755951 DOI: 10.1007/s12551-020-00773-7] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/05/2020] [Indexed: 01/13/2023] Open
Abstract
This personal hybrid review piece, written in light of my recipience of the UIPAB 2020 young investigator award, contains a mixture of my scientific biography and work so far. This paper is not intended to be a comprehensive review, but only to highlight my contributions to computation-related aspects of super-resolution microscopy, as well as their origins and future directions.
Collapse
Affiliation(s)
- Yoav Shechtman
- Department of Biomedical Engineering and Lorry Lokey Interdisciplinary Center for Life Sciences and Engineering, Technion-Israel Institute of Technology, 3200003, Haifa, Israel.
| |
Collapse
|
14
|
Gordon-Soffer R, Weiss LE, Eshel R, Ferdman B, Nehme E, Bercovici M, Shechtman Y. Microscopic scan-free surface profiling over extended axial ranges by point-spread-function engineering. SCIENCE ADVANCES 2020; 6:6/44/eabc0332. [PMID: 33115742 PMCID: PMC7608779 DOI: 10.1126/sciadv.abc0332] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/02/2020] [Accepted: 09/14/2020] [Indexed: 06/11/2023]
Abstract
The shape of a surface, i.e., its topography, influences many functional properties of a material; hence, characterization is critical in a wide variety of applications. Two notable challenges are profiling temporally changing structures, which requires high-speed acquisition, and capturing geometries with large axial steps. Here, we leverage point-spread-function engineering for scan-free, dynamic, microsurface profiling. The presented method is robust to axial steps and acquires full fields of view at camera-limited framerates. We present two approaches for implementation: fluorescence-based and label-free surface profiling, demonstrating the applicability to a variety of sample geometries and surface types.
Collapse
Affiliation(s)
- Racheli Gordon-Soffer
- Department of Biomedical Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel
- Lorry Lokey Interdisciplinary Center for Life Sciences and Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel
| | - Lucien E Weiss
- Department of Biomedical Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel
- Lorry Lokey Interdisciplinary Center for Life Sciences and Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel
| | - Ran Eshel
- Faculty of Mechanical Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel
| | - Boris Ferdman
- Lorry Lokey Interdisciplinary Center for Life Sciences and Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel
- Russell Berrie Nanotechnology Institute, Technion-Israel Institute of Technology, Haifa 3200003, Israel
| | - Elias Nehme
- Department of Biomedical Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel
- Lorry Lokey Interdisciplinary Center for Life Sciences and Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel
- Department of Electrical Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel
| | - Moran Bercovici
- Department of Biomedical Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel
- Faculty of Mechanical Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel
| | - Yoav Shechtman
- Department of Biomedical Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel.
- Lorry Lokey Interdisciplinary Center for Life Sciences and Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel
- Russell Berrie Nanotechnology Institute, Technion-Israel Institute of Technology, Haifa 3200003, Israel
| |
Collapse
|
15
|
Wang W, Wu B, Zhang B, Li X, Tan J. Correction of refractive index mismatch-induced aberrations under radially polarized illumination by deep learning. OPTICS EXPRESS 2020; 28:26028-26040. [PMID: 32906880 DOI: 10.1364/oe.402109] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/06/2020] [Accepted: 08/12/2020] [Indexed: 06/11/2023]
Abstract
Radially polarized field under strong focusing has emerged as a powerful manner for fluorescence microscopy. However, the refractive index (RI) mismatch-induced aberrations seriously degrade imaging performance, especially under high numerical aperture (NA). Traditional adaptive optics (AO) method is limited by its tedious procedure. Here, we present a computational strategy that uses artificial neural networks to correct the aberrations induced by RI mismatch. There are no requirements for expensive hardware and complicated wavefront sensing in our framework when the deep network training is completed. The structural similarity index (SSIM) criteria and spatial frequency spectrum analysis demonstrate that our deep-learning-based method has a better performance compared to the widely used Richardson-Lucy (RL) deconvolution method at different imaging depth on simulation data. Additionally, the generalization of our trained network model is tested on new types of samples that are not present in the training procedure to further evaluate the utility of the network, and the performance is also superior to RL deconvolution.
Collapse
|
16
|
Martens KJA, Jabermoradi A, Yang S, Hohlbein J. Integrating engineered point spread functions into the phasor-based single-molecule localization microscopy framework. Methods 2020; 193:107-115. [PMID: 32745620 DOI: 10.1016/j.ymeth.2020.07.010] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2020] [Revised: 07/17/2020] [Accepted: 07/27/2020] [Indexed: 10/23/2022] Open
Abstract
In single-molecule localization microscopy (SMLM), the use of engineered point spread functions (PSFs) provides access to three-dimensional localization information. The conventional approach of fitting PSFs with a single 2-dimensional Gaussian profile, however, often falls short in analyzing complex PSFs created by placing phase masks, deformable mirrors or spatial light modulators in the optical detection pathway. Here, we describe the integration of PSF modalities known as double-helix, saddle-point or tetra-pod into the phasor-based SMLM (pSMLM) framework enabling fast CPU based localization of single-molecule emitters with sub-pixel accuracy in three dimensions. For the double-helix PSF, pSMLM identifies the two individual lobes and uses their relative rotation for obtaining z-resolved localizations. For the analysis of saddle-point or tetra-pod PSFs, we present a novel phasor-based deconvolution approach entitled circular-tangent pSMLM. Saddle-point PSFs were experimentally realized by placing a deformable mirror in the Fourier plane and modulating the incoming wavefront with specific Zernike modes. Our pSMLM software package delivers similar precision and recall rates to the best-in-class software package (SMAP) at signal-to-noise ratios typical for organic fluorophores and achieves localization rates of up to 15 kHz (double-helix) and 250 kHz (saddle-point/tetra-pod) on a standard CPU. We further integrated pSMLM into an existing software package (SMALL-LABS) suitable for single-particle imaging and tracking in environments with obscuring backgrounds. Taken together, we provide a powerful hardware and software environment for advanced single-molecule studies.
Collapse
Affiliation(s)
- Koen J A Martens
- Laboratory of Biophysics, Wageningen University and Research, Stippeneng 4, 6708 WE Wageningen, The Netherlands; Laboratory of Bionanotechnology, Wageningen University and Research, Bornse Weilanden 9, 6708 WG Wageningen, The Netherlands
| | - Abbas Jabermoradi
- Laboratory of Biophysics, Wageningen University and Research, Stippeneng 4, 6708 WE Wageningen, The Netherlands
| | - Suyeon Yang
- Laboratory of Biophysics, Wageningen University and Research, Stippeneng 4, 6708 WE Wageningen, The Netherlands
| | - Johannes Hohlbein
- Laboratory of Biophysics, Wageningen University and Research, Stippeneng 4, 6708 WE Wageningen, The Netherlands; Microspectroscopy Research Facility, Wageningen University and Research, Stippeneng 4, 6708 WE Wageningen, The Netherlands.
| |
Collapse
|
17
|
Petrov PN, Moerner WE. Addressing systematic errors in axial distance measurements in single-emitter localization microscopy. OPTICS EXPRESS 2020; 28:18616-18632. [PMID: 32672159 PMCID: PMC7340385 DOI: 10.1364/oe.391496] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/25/2020] [Revised: 05/31/2020] [Accepted: 06/02/2020] [Indexed: 05/05/2023]
Abstract
Nanoscale localization of point emitters is critical to several methods in optical fluorescence microscopy, including single-molecule super-resolution imaging and tracking. While the precision of the localization procedure has been the topic of extensive study, localization accuracy has been less emphasized, in part due to the challenge of producing an experimental sample containing unperturbed point emitters at known three-dimensional positions in a relevant geometry. We report a new experimental system which reproduces a widely-adopted geometry in high-numerical aperture localization microscopy, in which molecules are situated in an aqueous medium above a glass coverslip imaged with an oil-immersion objective. We demonstrate a calibration procedure that enables measurement of the depth-dependent point spread function (PSF) for open aperture imaging as well as imaging with engineered PSFs with index mismatch. We reveal the complicated, depth-varying behavior of the focal plane position in this system and discuss the axial localization biases incurred by common approximations of this behavior. We compare our results to theoretical calculations.
Collapse
Affiliation(s)
- Petar N. Petrov
- Department of Chemistry, Stanford University, 333 Campus Drive, Stanford, CA 94305, USA
| | - W. E. Moerner
- Department of Chemistry, Stanford University, 333 Campus Drive, Stanford, CA 94305, USA
| |
Collapse
|
18
|
Liu S, Huh H, Lee SH, Huang F. Three-Dimensional Single-Molecule Localization Microscopy in Whole-Cell and Tissue Specimens. Annu Rev Biomed Eng 2020; 22:155-184. [PMID: 32243765 PMCID: PMC7430714 DOI: 10.1146/annurev-bioeng-060418-052203] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
Super-resolution microscopy techniques are versatile and powerful tools for visualizing organelle structures, interactions, and protein functions in biomedical research. However, whole-cell and tissue specimens challenge the achievable resolution and depth of nanoscopy methods. We focus on three-dimensional single-molecule localization microscopy and review some of the major roadblocks and developing solutions to resolving thick volumes of cells and tissues at the nanoscale in three dimensions. These challenges include background fluorescence, system- and sample-induced aberrations, and information carried by photons, as well as drift correction, volume reconstruction, and photobleaching mitigation. We also highlight examples of innovations that have demonstrated significant breakthroughs in addressing the abovementioned challenges together with their core concepts as well as their trade-offs.
Collapse
Affiliation(s)
- Sheng Liu
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, Indiana 47907, USA;
| | - Hyun Huh
- Institute for Quantitative Biomedicine, Rutgers University, Piscataway, New Jersey 08854, USA
| | - Sang-Hyuk Lee
- Institute for Quantitative Biomedicine, Rutgers University, Piscataway, New Jersey 08854, USA
- Department of Physics and Astronomy, Rutgers University, Piscataway, New Jersey 08854, USA;
| | - Fang Huang
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, Indiana 47907, USA;
- Purdue Institute for Integrative Neuroscience, Purdue University, West Lafayette, Indiana 47907, USA
- Purdue Institute of Inflammation, Immunology, and Infectious Disease, Purdue University, West Lafayette, Indiana 47907, USA
| |
Collapse
|
19
|
Xu F, Ma D, MacPherson KP, Liu S, Bu Y, Wang Y, Tang Y, Bi C, Kwok T, Chubykin AA, Yin P, Calve S, Landreth GE, Huang F. Three-dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval. Nat Methods 2020; 17:531-540. [PMID: 32371980 PMCID: PMC7289454 DOI: 10.1038/s41592-020-0816-x] [Citation(s) in RCA: 40] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2019] [Accepted: 03/19/2020] [Indexed: 02/07/2023]
Abstract
Single-molecule localization microscopy is a powerful tool for visualizing subcellular structures, interactions and protein functions in biological research. However, inhomogeneous refractive indices inside cells and tissues distort the fluorescent signal emitted from single-molecule probes, which rapidly degrades resolution with increasing depth. We propose a method that enables the construction of an in situ 3D response of single emitters directly from single-molecule blinking datasets, and therefore allows their locations to be pinpointed with precision that achieves the Cramér-Rao lower bound and uncompromised fidelity. We demonstrate this method, named in situ PSF retrieval (INSPR), across a range of cellular and tissue architectures, from mitochondrial networks and nuclear pores in mammalian cells to amyloid-β plaques and dendrites in brain tissues and elastic fibers in developing cartilage of mice. This advancement expands the routine applicability of super-resolution microscopy from selected cellular targets near coverslips to intra- and extracellular targets deep inside tissues.
Collapse
Affiliation(s)
- Fan Xu
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN, USA
| | - Donghan Ma
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN, USA
| | - Kathryn P MacPherson
- Department of Anatomy and Cell Biology, Indiana University School of Medicine, Indianapolis, IN, USA
| | - Sheng Liu
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN, USA
| | - Ye Bu
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN, USA
| | - Yu Wang
- Wyss Institute for Biologically Inspired Engineering, Harvard University, Boston, MA, USA.,Department of Systems Biology, Harvard Medical School, Boston, MA, USA
| | - Yu Tang
- Department of Biological Sciences, Purdue University, West Lafayette, IN, USA.,Purdue Institute for Integrative Neuroscience, Purdue University, West Lafayette, IN, USA
| | - Cheng Bi
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN, USA
| | - Tim Kwok
- Birck Nanotechnology Center, Purdue University, West Lafayette, IN, USA
| | - Alexander A Chubykin
- Department of Biological Sciences, Purdue University, West Lafayette, IN, USA.,Purdue Institute for Integrative Neuroscience, Purdue University, West Lafayette, IN, USA
| | - Peng Yin
- Wyss Institute for Biologically Inspired Engineering, Harvard University, Boston, MA, USA.,Department of Systems Biology, Harvard Medical School, Boston, MA, USA
| | - Sarah Calve
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN, USA.
| | - Gary E Landreth
- Department of Anatomy and Cell Biology, Indiana University School of Medicine, Indianapolis, IN, USA. .,Stark Neurosciences Research Institute, Indiana University School of Medicine, Indianapolis, IN, USA.
| | - Fang Huang
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN, USA. .,Purdue Institute for Integrative Neuroscience, Purdue University, West Lafayette, IN, USA. .,Purdue Institute of Inflammation, Immunology and Infectious Disease, Purdue University, West Lafayette, IN, USA.
| |
Collapse
|
20
|
Ferdman B, Nehme E, Weiss LE, Orange R, Alalouf O, Shechtman Y. VIPR: vectorial implementation of phase retrieval for fast and accurate microscopic pixel-wise pupil estimation. OPTICS EXPRESS 2020; 28:10179-10198. [PMID: 32225609 DOI: 10.1364/oe.388248] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/16/2023]
Abstract
In microscopy, proper modeling of the image formation has a substantial effect on the precision and accuracy in localization experiments and facilitates the correction of aberrations in adaptive optics experiments. The observed images are subject to polarization effects, refractive index variations, and system specific constraints. Previously reported techniques have addressed these challenges by using complicated calibration samples, computationally heavy numerical algorithms, and various mathematical simplifications. In this work, we present a phase retrieval approach based on an analytical derivation of the vectorial diffraction model. Our method produces an accurate estimate of the system's phase information, without any prior knowledge about the aberrations, in under a minute.
Collapse
|
21
|
Möckl L, Roy AR, Moerner WE. Deep learning in single-molecule microscopy: fundamentals, caveats, and recent developments [Invited]. BIOMEDICAL OPTICS EXPRESS 2020; 11:1633-1661. [PMID: 32206433 PMCID: PMC7075610 DOI: 10.1364/boe.386361] [Citation(s) in RCA: 24] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/17/2019] [Revised: 02/10/2020] [Accepted: 02/13/2020] [Indexed: 05/08/2023]
Abstract
Deep learning-based data analysis methods have gained considerable attention in all fields of science over the last decade. In recent years, this trend has reached the single-molecule community. In this review, we will survey significant contributions of the application of deep learning in single-molecule imaging experiments. Additionally, we will describe the historical events that led to the development of modern deep learning methods, summarize the fundamental concepts of deep learning, and highlight the importance of proper data composition for accurate, unbiased results.
Collapse
|
22
|
Möckl L, Roy AR, Petrov PN, Moerner WE. Accurate and rapid background estimation in single-molecule localization microscopy using the deep neural network BGnet. Proc Natl Acad Sci U S A 2020; 117:60-67. [PMID: 31871202 PMCID: PMC6955367 DOI: 10.1073/pnas.1916219117] [Citation(s) in RCA: 31] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022] Open
Abstract
Background fluorescence, especially when it exhibits undesired spatial features, is a primary factor for reduced image quality in optical microscopy. Structured background is particularly detrimental when analyzing single-molecule images for 3-dimensional localization microscopy or single-molecule tracking. Here, we introduce BGnet, a deep neural network with a U-net-type architecture, as a general method to rapidly estimate the background underlying the image of a point source with excellent accuracy, even when point-spread function (PSF) engineering is in use to create complex PSF shapes. We trained BGnet to extract the background from images of various PSFs and show that the identification is accurate for a wide range of different interfering background structures constructed from many spatial frequencies. Furthermore, we demonstrate that the obtained background-corrected PSF images, for both simulated and experimental data, lead to a substantial improvement in localization precision. Finally, we verify that structured background estimation with BGnet results in higher quality of superresolution reconstructions of biological structures.
Collapse
Affiliation(s)
- Leonhard Möckl
- Department of Chemistry, Stanford University, Stanford, CA 94305
| | - Anish R Roy
- Department of Chemistry, Stanford University, Stanford, CA 94305
| | - Petar N Petrov
- Department of Chemistry, Stanford University, Stanford, CA 94305
| | - W E Moerner
- Department of Chemistry, Stanford University, Stanford, CA 94305
| |
Collapse
|
23
|
Möckl L, Petrov PN, Moerner WE. Accurate phase retrieval of complex 3D point spread functions with deep residual neural networks. APPLIED PHYSICS LETTERS 2019; 115:251106. [PMID: 32127719 PMCID: PMC7043838 DOI: 10.1063/1.5125252] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/21/2019] [Accepted: 12/09/2019] [Indexed: 05/03/2023]
Abstract
Phase retrieval, i.e., the reconstruction of phase information from intensity information, is a central problem in many optical systems. Imaging the emission from a point source such as a single molecule is one example. Here, we demonstrate that a deep residual neural net is able to quickly and accurately extract the hidden phase for general point spread functions (PSFs) formed by Zernike-type phase modulations. Five slices of the 3D PSF at different focal positions within a two micrometer range around the focus are sufficient to retrieve the first six orders of Zernike coefficients.
Collapse
|
24
|
CHANDLER TALON, SHROFF HARI, OLDENBOURG RUDOLF, LA RIVIÈRE PATRICK. Spatio-angular fluorescence microscopy II. Paraxial 4f imaging. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2019; 36:1346-1360. [PMID: 31503560 PMCID: PMC7045803 DOI: 10.1364/josaa.36.001346] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/06/2023]
Abstract
We investigate the properties of a single-view fluorescence microscope in a 4f geometry when imaging fluorescent dipoles without using the monopole or scalar approximations. We show that this imaging system has a spatio-angular band limit, and we exploit the band limit to perform efficient simulations. Notably, we show that information about the out-of-plane orientation of ensembles of in-focus fluorophores is recorded by paraxial fluorescence microscopes. Additionally, we show that the monopole approximation may cause biased estimates of fluorophore concentrations, but these biases are small when the sample contains either many randomly oriented fluorophores in each resolvable volume or unconstrained rotating fluorophores.
Collapse
Affiliation(s)
- TALON CHANDLER
- University of Chicago, Department of Radiology, Chicago, Illinois 60637, USA
| | - HARI SHROFF
- Section on High Resolution Optical Imaging, National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health, Bethesda, Maryland 20892, USA
- Marine Biological Laboratory, Bell Center, Woods Hole, Massachusetts 02543, USA
| | - RUDOLF OLDENBOURG
- Marine Biological Laboratory, Bell Center, Woods Hole, Massachusetts 02543, USA
| | - PATRICK LA RIVIÈRE
- University of Chicago, Department of Radiology, Chicago, Illinois 60637, USA
- Marine Biological Laboratory, Bell Center, Woods Hole, Massachusetts 02543, USA
| |
Collapse
|
25
|
Yan T, Richardson CJ, Zhang M, Gahlmann A. Computational correction of spatially variant optical aberrations in 3D single-molecule localization microscopy. OPTICS EXPRESS 2019; 27:12582-12599. [PMID: 31052798 DOI: 10.1364/oe.27.012582] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/07/2019] [Accepted: 04/03/2019] [Indexed: 05/20/2023]
Abstract
3D single-molecule localization microscopy relies on fitting the shape of point-spread-functions (PSFs) recorded on a wide-field detector. However, optical aberrations distort those shapes, which compromises the accuracy and precision of single-molecule localization microscopy. Here, we employ a computational phase retrieval based on a vectorial PSF model to quantify the spatial variance of optical aberrations in a two-channel ultrawide-field single-molecule localization microscope. The use of a spatially variant PSF model enables accurate and precise emitter localization in x-, y- and z-directions throughout the entire field of view.
Collapse
|
26
|
Žurauskas M, Dobbie IM, Parton RM, Phillips MA, Göhler A, Davis I, Booth MJ. IsoSense: frequency enhanced sensorless adaptive optics through structured illumination. OPTICA 2019; 6:370-379. [PMID: 31417942 PMCID: PMC6683765 DOI: 10.1364/optica.6.000370] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/23/2018] [Revised: 02/07/2019] [Accepted: 02/07/2019] [Indexed: 05/03/2023]
Abstract
We present IsoSense, a wavefront sensing method that mitigates sample dependency in image-based sensorless adaptive optics applications in microscopy. Our method employs structured illumination to create additional high spatial frequencies in the image through custom illumination patterns. This improves the reliability of image quality metric calculations and enables sensorless wavefront measurement even in samples with sparse spatial frequency content. We demonstrate the feasibility of IsoSense for aberration correction in a deformable-mirror-based structured illumination super-resolution fluorescence microscope.
Collapse
Affiliation(s)
- Mantas Žurauskas
- Centre for Neural Circuits and Behaviour, University of Oxford, Mansfield Road, Oxford OX1 3SR, UK
- Micron Advanced Bioimaging Unit, Department of Biochemistry, University of Oxford, South Parks Road, Oxford OX1 3QU, UK
| | - Ian M. Dobbie
- Micron Advanced Bioimaging Unit, Department of Biochemistry, University of Oxford, South Parks Road, Oxford OX1 3QU, UK
| | - Richard M. Parton
- Micron Advanced Bioimaging Unit, Department of Biochemistry, University of Oxford, South Parks Road, Oxford OX1 3QU, UK
| | - Mick A. Phillips
- Micron Advanced Bioimaging Unit, Department of Biochemistry, University of Oxford, South Parks Road, Oxford OX1 3QU, UK
| | - Antonia Göhler
- Micron Advanced Bioimaging Unit, Department of Biochemistry, University of Oxford, South Parks Road, Oxford OX1 3QU, UK
- Currently at SOMNOmedics GmbH, 97236 Randersacker, Germany
| | - Ilan Davis
- Micron Advanced Bioimaging Unit, Department of Biochemistry, University of Oxford, South Parks Road, Oxford OX1 3QU, UK
| | - Martin J. Booth
- Centre for Neural Circuits and Behaviour, University of Oxford, Mansfield Road, Oxford OX1 3SR, UK
- Department of Engineering Science, University of Oxford, Parks Road, Oxford OX1 3PJ, UK
| |
Collapse
|
27
|
Hershko E, Weiss LE, Michaeli T, Shechtman Y. Multicolor localization microscopy and point-spread-function engineering by deep learning. OPTICS EXPRESS 2019; 27:6158-6183. [PMID: 30876208 DOI: 10.1364/oe.27.006158] [Citation(s) in RCA: 46] [Impact Index Per Article: 9.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/01/2018] [Accepted: 01/26/2019] [Indexed: 05/21/2023]
Abstract
Deep learning has become an extremely effective tool for image classification and image restoration problems. Here, we apply deep learning to microscopy and demonstrate how neural networks can exploit the chromatic dependence of the point-spread function to classify the colors of single emitters imaged on a grayscale camera. While existing localization microscopy methods for spectral classification require additional optical elements in the emission path, e.g., spectral filters, prisms, or phase masks, our neural net correctly identifies static and mobile emitters with high efficiency using a standard, unmodified single-channel configuration. Furthermore, we show how deep learning can be used to design new phase-modulating elements that, when implemented into the imaging path, result in further improved color differentiation between species, including simultaneously differentiating four species in a single image.
Collapse
|
28
|
Park AHK, Shoman H, Ma M, Shekhar S, Chrostowski L. Ring resonator based polarization diversity WDM receiver. OPTICS EXPRESS 2019; 27:6147-6157. [PMID: 30876207 DOI: 10.1364/oe.27.006147] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/01/2018] [Accepted: 01/26/2019] [Indexed: 06/09/2023]
Abstract
A ring resonator based 4 channel wavelength division multiplexing (WDM) receiver with polarization diversity is demonstrated at 10 Gb/s per channel. By forming a waveguide loop between the two output ports of a polarization splitter-rotator (PSR), the input signals in the quasi-transverse-electric (quasi-TE) and the quasi-transverse-magnetic (quasi-TM) polarizations can be demultiplexed by the same set of ring resonator filters, thus reducing the number of required channel control circuits by half compared to methods which process the two polarizations individually. Large signal measurement results indicate that the design can tolerate a signal delay of up to 30% of the unit interval (UI) between the two polarizations, which implies that compensating for manufacturing variability with optical delay lines on chip is not necessary for a robust operation. The inter-channel crosstalk is found negligible down to 0.4nm (50 GHz) spacing, at which point the adjacent channel isolation is 17 dB, proving the design's compatibility for dense WDM application.
Collapse
|
29
|
Wang W, Ye F, Shen H, Moringo NA, Dutta C, Robinson JT, Landes CF. Generalized method to design phase masks for 3D super-resolution microscopy. OPTICS EXPRESS 2019; 27:3799-3816. [PMID: 30732394 DOI: 10.1364/oe.27.003799] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/17/2018] [Accepted: 01/20/2019] [Indexed: 05/20/2023]
Abstract
Point spread function (PSF) engineering by phase modulation is a novel approach to three-dimensional (3D) super-resolution microscopy, with different point spread functions being proposed for specific applications. It is often not easy to achieve the desired shape of engineered point spread functions because it is challenging to determine the correct phase mask. Additionally, a phase mask can either encode 3D space information or additional time information, but not both simultaneously. A robust algorithm for recovering a phase mask to generate arbitrary point spread functions is needed. In this work, a generalized phase mask design method is introduced by performing an optimization. A stochastic gradient descent algorithm and a Gauss-Newton algorithm are developed and compared for their ability to recover the phase masks for previously reported point spread functions. The new Gauss-Newton algorithm converges to a minimum at much higher speeds. This algorithm is used to design a novel stretching-lobe phase mask to encode temporal and 3D spatial information simultaneously. The stretching-lobe phase mask and other masks are fabricated in-house for proof-of-concept using multi-level light lithography and an optimized commercially sourced stretching-lobe phase mask (PM) is validated experimentally to encode 3D spatial and temporal information. The algorithms' generalizability is further demonstrated by generating a phase mask that comprises four different letters at different depths.
Collapse
|
30
|
Zelger P, Kaser K, Rossboth B, Velas L, Schütz GJ, Jesacher A. Three-dimensional localization microscopy using deep learning. OPTICS EXPRESS 2018; 26:33166-33179. [PMID: 30645473 DOI: 10.1364/oe.26.033166] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/24/2018] [Accepted: 11/11/2018] [Indexed: 05/19/2023]
Abstract
Single molecule localization microscopy (SMLM) is one of the fastest evolving and most broadly used super-resolving imaging techniques in the biosciences. While image recordings could take up to hours only ten years ago, scientists are now reaching for real-time imaging in order to follow the dynamics of biology. To this end, it is crucial to have data processing strategies available that are capable of handling the vast amounts of data produced by the microscope. In this article, we report on the use of a deep convolutional neural network (CNN) for localizing particles in three dimensions on the basis of single images. In test experiments conducted on fluorescent microbeads, we show that the precision obtained with a CNN can be comparable to that of maximum likelihood estimation (MLE), which is the accepted gold standard. Regarding speed, the CNN performs with about 22k localizations per second more than three orders of magnitude faster than the MLE algorithm of ThunderSTORM. If only five parameters are estimated (3D position, signal and background), our CNN implementation is currently slower than the fastest, recently published GPU-based MLE algorithm. However, in this comparison the CNN catches up with every additional parameter, with only a few percent extra time required per additional dimension. Thus it may become feasible to estimate further variables such as molecule orientation, aberration functions or color. We experimentally demonstrate that jointly estimating Zernike mode magnitudes for aberration modeling can significantly improve the accuracy of the estimates.
Collapse
|
31
|
Backlund MP, Shechtman Y, Walsworth RL. Fundamental Precision Bounds for Three-Dimensional Optical Localization Microscopy with Poisson Statistics. PHYSICAL REVIEW LETTERS 2018; 121:023904. [PMID: 30085695 DOI: 10.1103/physrevlett.121.023904] [Citation(s) in RCA: 27] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/06/2018] [Indexed: 05/23/2023]
Abstract
Point source localization is a problem of persistent interest in optical imaging. In particular, a number of widely used biological microscopy techniques rely on precise three-dimensional localization of single fluorophores. As emitter depth localization is more challenging than lateral localization, considerable effort has been spent on engineering the response of the microscope in a way that reveals increased depth information. Here, we prove the (sub)optimality of these approaches by deriving and comparing to the measurement-independent quantum Cramér-Rao bound (QCRB). We show that existing methods for depth localization with single-objective collection exceed the QCRB, and we gain insight into the bound by proposing an interferometer arrangement that approaches it. We also show that for light collection with two opposed objectives, an established interferometric technique globally reaches the QCRB in all three dimensions simultaneously, and so this represents an interesting case study from the point of view of quantum multiparameter estimation.
Collapse
Affiliation(s)
- Mikael P Backlund
- Harvard-Smithsonian Center for Astrophysics, Cambridge, Massachusetts 02138, USA
- Department of Physics, Harvard University, Cambridge, Massachusetts 02138, USA
| | - Yoav Shechtman
- Department of Biomedical Engineering, Technion, Israel Institute of Technology, Haifa 32000, Israel
| | - Ronald L Walsworth
- Harvard-Smithsonian Center for Astrophysics, Cambridge, Massachusetts 02138, USA
- Department of Physics, Harvard University, Cambridge, Massachusetts 02138, USA
| |
Collapse
|
32
|
Aristov A, Lelandais B, Rensen E, Zimmer C. ZOLA-3D allows flexible 3D localization microscopy over an adjustable axial range. Nat Commun 2018; 9:2409. [PMID: 29921892 PMCID: PMC6008307 DOI: 10.1038/s41467-018-04709-4] [Citation(s) in RCA: 47] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2018] [Accepted: 05/16/2018] [Indexed: 11/23/2022] Open
Abstract
Single molecule localization microscopy can generate 3D super-resolution images without scanning by leveraging the axial variations of normal or engineered point spread functions (PSF). Successful implementation of these approaches for extended axial ranges remains, however, challenging. We present Zernike Optimized Localization Approach in 3D (ZOLA-3D), an easy-to-use computational and optical solution that achieves optimal resolution over a tunable axial range. We use ZOLA-3D to demonstrate 3D super-resolution imaging of mitochondria, nuclear pores and microtubules in entire nuclei or cells up to ~5 μm deep. 3D single-molecule localization is limited in depth and often requires using a wide range of point spread functions (PSFs). Here the authors present an optical solution featuring a deformable mirror to generate different PSFs and easy-to-use software for super-resolution imaging up to 5 µm deep.
Collapse
Affiliation(s)
- Andrey Aristov
- Unité Imagerie et Modélisation, Institut Pasteur, 25-28 rue du Docteur Roux, Paris, France.,UMR 3691, CNRS; C3BI, USR 3756, IP CNRS, Paris, France
| | - Benoit Lelandais
- Unité Imagerie et Modélisation, Institut Pasteur, 25-28 rue du Docteur Roux, Paris, France.,UMR 3691, CNRS; C3BI, USR 3756, IP CNRS, Paris, France.,Hub Bioinformatique et Biostatistique, Institut Pasteur, Paris, France
| | - Elena Rensen
- Unité Imagerie et Modélisation, Institut Pasteur, 25-28 rue du Docteur Roux, Paris, France.,UMR 3691, CNRS; C3BI, USR 3756, IP CNRS, Paris, France
| | - Christophe Zimmer
- Unité Imagerie et Modélisation, Institut Pasteur, 25-28 rue du Docteur Roux, Paris, France. .,UMR 3691, CNRS; C3BI, USR 3756, IP CNRS, Paris, France.
| |
Collapse
|
33
|
Gustavsson AK, Petrov PN, Moerner WE. Light sheet approaches for improved precision in 3D localization-based super-resolution imaging in mammalian cells [Invited]. OPTICS EXPRESS 2018; 26:13122-13147. [PMID: 29801343 PMCID: PMC6005674 DOI: 10.1364/oe.26.013122] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/26/2018] [Accepted: 03/30/2018] [Indexed: 05/08/2023]
Abstract
The development of imaging techniques beyond the diffraction limit has paved the way for detailed studies of nanostructures and molecular mechanisms in biological systems. Imaging thicker samples, such as mammalian cells and tissue, in all three dimensions, is challenging due to increased background and volumes to image. Light sheet illumination is a method that allows for selective irradiation of the image plane, and its inherent optical sectioning capability allows for imaging of biological samples with reduced background, photobleaching, and photodamage. In this review, we discuss the advantage of combining single-molecule imaging with light sheet illumination. We begin by describing the principles of single-molecule localization microscopy and of light sheet illumination. Finally, we present examples of designs that successfully have married single-molecule super-resolution imaging with light sheet illumination for improved precision in mammalian cells.
Collapse
|
34
|
Zhao G, Zheng C, Kuang C, Zhou R, Kabir MM, Toussaint KC, Wang W, Xu L, Li H, Xiu P, Liu X. Nonlinear Focal Modulation Microscopy. PHYSICAL REVIEW LETTERS 2018; 120:193901. [PMID: 29799223 DOI: 10.1103/physrevlett.120.193901] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/30/2017] [Indexed: 06/08/2023]
Abstract
We demonstrate nonlinear focal modulation microscopy (NFOMM) to achieve superresolution imaging. Traditional approaches to superresolution that utilize point scanning often rely on spatially reducing the size of the emission pattern by directly narrowing (e.g., through minimizing the detection pinhole in Airyscan, Zeiss) or indirectly peeling its outer profiles [e.g., through depleting the outer emission region in stimulated emission depletion (STED) microscopy]. We show that an alternative conceptualization that focuses on maximizing the optical system's frequency shifting ability offers advantages in further improving resolution while reducing system complexity. In NFOMM, a spatial light modulator and a suitably intense laser illumination are used to implement nonlinear focal-field modulation to achieve a transverse spatial resolution of ∼60 nm (∼λ/10). We show that NFOMM is comparable with STED microscopy and suitable for fundamental biology studies, as evidenced in imaging nuclear pore complexes, tubulin and vimentin in Vero cells. Since NFOMM is readily implemented as an add-on module to a laser-scanning microscope, we anticipate wide utility of this new imaging technique.
Collapse
Affiliation(s)
- Guangyuan Zhao
- State Key Laboratory of Modern Optical Instrumentation, College of Optical Science and Engineering, Zhejiang University, Hangzhou, Zhejiang 310027, China
| | - Cheng Zheng
- State Key Laboratory of Modern Optical Instrumentation, College of Optical Science and Engineering, Zhejiang University, Hangzhou, Zhejiang 310027, China
| | - Cuifang Kuang
- State Key Laboratory of Modern Optical Instrumentation, College of Optical Science and Engineering, Zhejiang University, Hangzhou, Zhejiang 310027, China
- Collaborative Innovation Center of Extreme Optics, Shanxi University, Taiyuan 030006, China
| | - Renjie Zhou
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong, China
| | - Mohammad M Kabir
- Department of Electrical and Computer Engineering, University of Illinois Urbana-Champaign, Urbana, Illinois 61801, USA
| | - Kimani C Toussaint
- Department of Mechanical Science and Engineering, University of Illinois Urbana-Champaign, Urbana, Illinois 61801, USA
| | - Wensheng Wang
- State Key Laboratory of Modern Optical Instrumentation, College of Optical Science and Engineering, Zhejiang University, Hangzhou, Zhejiang 310027, China
| | - Liang Xu
- State Key Laboratory of Modern Optical Instrumentation, College of Optical Science and Engineering, Zhejiang University, Hangzhou, Zhejiang 310027, China
| | - Haifeng Li
- State Key Laboratory of Modern Optical Instrumentation, College of Optical Science and Engineering, Zhejiang University, Hangzhou, Zhejiang 310027, China
| | - Peng Xiu
- State Key Laboratory of Modern Optical Instrumentation, College of Optical Science and Engineering, Zhejiang University, Hangzhou, Zhejiang 310027, China
| | - Xu Liu
- State Key Laboratory of Modern Optical Instrumentation, College of Optical Science and Engineering, Zhejiang University, Hangzhou, Zhejiang 310027, China
- Collaborative Innovation Center of Extreme Optics, Shanxi University, Taiyuan 030006, China
| |
Collapse
|
35
|
Siemons M, Hulleman CN, Thorsen RØ, Smith CS, Stallinga S. High precision wavefront control in point spread function engineering for single emitter localization. OPTICS EXPRESS 2018; 26:8397-8416. [PMID: 29715807 DOI: 10.1364/oe.26.008397] [Citation(s) in RCA: 35] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
Point spread function (PSF) engineering is used in single emitter localization to measure the emitter position in 3D and possibly other parameters such as the emission color or dipole orientation as well. Advanced PSF models such as spline fits to experimental PSFs or the vectorial PSF model can be used in the corresponding localization algorithms in order to model the intricate spot shape and deformations correctly. The complexity of the optical architecture and fit model makes PSF engineering approaches particularly sensitive to optical aberrations. Here, we present a calibration and alignment protocol for fluorescence microscopes equipped with a spatial light modulator (SLM) with the goal of establishing a wavefront error well below the diffraction limit for optimum application of complex engineered PSFs. We achieve high-precision wavefront control, to a level below 20 mλ wavefront aberration over a 30 minute time window after the calibration procedure, using a separate light path for calibrating the pixel-to-pixel variations of the SLM, and alignment of the SLM with respect to the optical axis and Fourier plane within 3 μm (x/y) and 100 μm (z) error. Aberrations are retrieved from a fit of the vectorial PSF model to a bead z-stack and compensated with a residual wavefront error comparable to the error of the SLM calibration step. This well-calibrated and corrected setup makes it possible to create complex '3D+λ' PSFs that fit very well to the vectorial PSF model. Proof-of-principle bead experiments show precisions below 10 nm in x, y, and λ, and below 20 nm in z over an axial range of 1 μm with 2000 signal photons and 12 background photons.
Collapse
|
36
|
Gustavsson AK, Petrov PN, Lee MY, Shechtman Y, Moerner WE. Tilted Light Sheet Microscopy with 3D Point Spread Functions for Single-Molecule Super-Resolution Imaging in Mammalian Cells. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2018; 10500:105000M. [PMID: 29681676 PMCID: PMC5906058 DOI: 10.1117/12.2288443] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
To obtain a complete picture of subcellular nanostructures, cells must be imaged with high resolution in all three dimensions (3D). Here, we present tilted light sheet microscopy with 3D point spread functions (TILT3D), an imaging platform that combines a novel, tilted light sheet illumination strategy with engineered long axial range point spread functions (PSFs) for low-background, 3D super localization of single molecules as well as 3D super-resolution imaging in thick cells. TILT3D is built upon a standard inverted microscope and has minimal custom parts. The axial positions of the single molecules are encoded in the shape of the PSF rather than in the position or thickness of the light sheet, and the light sheet can therefore be formed using simple optics. The result is flexible and user-friendly 3D super-resolution imaging with tens of nm localization precision throughout thick mammalian cells. We validated TILT3D for 3D super-resolution imaging in mammalian cells by imaging mitochondria and the full nuclear lamina using the double-helix PSF for single-molecule detection and the recently developed Tetrapod PSF for fiducial bead tracking and live axial drift correction. We envision TILT3D to become an important tool not only for 3D super-resolution imaging, but also for live whole-cell single-particle and single-molecule tracking.
Collapse
Affiliation(s)
- Anna-Karin Gustavsson
- Dept. of Chemistry, Stanford University, 375 North-South Axis, Stanford, CA, USA 94305-4401
- Dept. of Biosciences and Nutrition, Karolinska Institutet, Stockholm, Sweden 17111
| | - Petar N. Petrov
- Dept. of Chemistry, Stanford University, 375 North-South Axis, Stanford, CA, USA 94305-4401
| | - Maurice Y. Lee
- Dept. of Chemistry, Stanford University, 375 North-South Axis, Stanford, CA, USA 94305-4401
- Biophysics Program, Stanford University, 375 North-South Axis, Stanford, CA, USA 94305-4401
| | - Yoav Shechtman
- Dept. of Chemistry, Stanford University, 375 North-South Axis, Stanford, CA, USA 94305-4401
| | - W. E. Moerner
- Dept. of Chemistry, Stanford University, 375 North-South Axis, Stanford, CA, USA 94305-4401
- Biophysics Program, Stanford University, 375 North-South Axis, Stanford, CA, USA 94305-4401
| |
Collapse
|
37
|
Gustavsson AK, Petrov PN, Lee MY, Shechtman Y, Moerner WE. 3D single-molecule super-resolution microscopy with a tilted light sheet. Nat Commun 2018; 9:123. [PMID: 29317629 PMCID: PMC5760554 DOI: 10.1038/s41467-017-02563-4] [Citation(s) in RCA: 105] [Impact Index Per Article: 17.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2017] [Accepted: 12/11/2017] [Indexed: 12/24/2022] Open
Abstract
Tilted light sheet microscopy with 3D point spread functions (TILT3D) combines a novel, tilted light sheet illumination strategy with long axial range point spread functions (PSFs) for low-background, 3D super-localization of single molecules as well as 3D super-resolution imaging in thick cells. Because the axial positions of the single emitters are encoded in the shape of each single-molecule image rather than in the position or thickness of the light sheet, the light sheet need not be extremely thin. TILT3D is built upon a standard inverted microscope and has minimal custom parts. The result is simple and flexible 3D super-resolution imaging with tens of nm localization precision throughout thick mammalian cells. We validate TILT3D for 3D super-resolution imaging in mammalian cells by imaging mitochondria and the full nuclear lamina using the double-helix PSF for single-molecule detection and the recently developed tetrapod PSFs for fiducial bead tracking and live axial drift correction.
Collapse
Affiliation(s)
- Anna-Karin Gustavsson
- Department of Chemistry, Stanford University, Stanford, CA, 94305, USA.,Department of Biosciences and Nutrition, Karolinska Institutet, Stockholm, SE-17177, Sweden
| | - Petar N Petrov
- Department of Chemistry, Stanford University, Stanford, CA, 94305, USA
| | - Maurice Y Lee
- Department of Chemistry, Stanford University, Stanford, CA, 94305, USA.,Biophysics Program, Stanford University, Stanford, CA, 94305, USA
| | - Yoav Shechtman
- Department of Chemistry, Stanford University, Stanford, CA, 94305, USA.,Biomedical Engineering Department, Technion, Israel Institute of Technology, Haifa, 3200003, Israel
| | - W E Moerner
- Department of Chemistry, Stanford University, Stanford, CA, 94305, USA. .,Biophysics Program, Stanford University, Stanford, CA, 94305, USA.
| |
Collapse
|
38
|
Shechtman Y, Gustavsson AK, Petrov PN, Dultz E, Lee MY, Weis K, Moerner WE. Observation of live chromatin dynamics in cells via 3D localization microscopy using Tetrapod point spread functions. BIOMEDICAL OPTICS EXPRESS 2017; 8:5735-5748. [PMID: 29296501 PMCID: PMC5745116 DOI: 10.1364/boe.8.005735] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/14/2017] [Revised: 11/11/2017] [Accepted: 11/11/2017] [Indexed: 05/15/2023]
Abstract
We report the observation of chromatin dynamics in living budding yeast (Saccharomyces cerevisiae) cells, in three-dimensions (3D). Using dual color localization microscopy and employing a Tetrapod point spread function, we analyze the spatio-temporal dynamics of two fluorescently labeled DNA loci surrounding the GAL locus. From the measured trajectories, we obtain different dynamical characteristics in terms of inter-loci distance and temporal variance; when the GAL locus is activated, the 3D inter-loci distance and temporal variance increase compared to the inactive state. These changes are visible in spite of the large thermally- and biologically-driven heterogeneity in the relative motion of the two loci. Our observations are consistent with current euchromatin vs. heterochromatin models.
Collapse
Affiliation(s)
- Yoav Shechtman
- Department of Chemistry, Stanford University, 375 North-South Mall, Stanford, California 94305, USA
- Currently with the Department of Biomedical Engineering, Technion-Israel Institute of Technology, Haifa, 32000 Israel
| | - Anna-Karin Gustavsson
- Department of Chemistry, Stanford University, 375 North-South Mall, Stanford, California 94305, USA
- Department of Biosciences and Nutrition, Karolinska Institutet, Stockholm, SE-171 77, Sweden
| | - Petar N Petrov
- Department of Chemistry, Stanford University, 375 North-South Mall, Stanford, California 94305, USA
| | - Elisa Dultz
- Department of Biology, Institute of Biochemistry, Eidgenössische Technische Hochschule Zurich, 8093 Zurich, Switzerland
| | - Maurice Y Lee
- Department of Chemistry, Stanford University, 375 North-South Mall, Stanford, California 94305, USA
- Biophysics Program, Stanford University, Stanford, CA 94305, USA
| | - Karsten Weis
- Department of Biology, Institute of Biochemistry, Eidgenössische Technische Hochschule Zurich, 8093 Zurich, Switzerland
| | - W E Moerner
- Department of Chemistry, Stanford University, 375 North-South Mall, Stanford, California 94305, USA
| |
Collapse
|