Simultaneous Imaging of Different Focal Planes in Fluorescence Microscopy for the Study of Cellular Dynamics in Three Dimensions

High-density volumetric super-resolution microscopy

Volumetric super-resolution microscopy typically encodes the 3D position of single-molecule fluorescence into a 2D image by changing the shape of the point spread function (PSF) as a function of depth. However, the resulting large and complex PSF spatial footprints reduce biological throughput and applicability by requiring lower labeling densities to avoid overlapping fluorescent signals. We quantitatively compare the density dependence of single-molecule light field microscopy (SMLFM) to other 3D PSFs (astigmatism, double helix and tetrapod) showing that SMLFM enables an order-of-magnitude speed improvement compared to the double helix PSF by resolving overlapping emitters through parallax. We demonstrate this optical robustness experimentally with high accuracy ( > 99.2 ± 0.1%, 0.1 locs μm-2) and sensitivity ( > 86.6 ± 0.9%, 0.1 locs μm-2) through whole-cell (scan-free) imaging and tracking of single membrane proteins in live primary B cells. We also exemplify high-density volumetric imaging (0.15 locs μm-2) in dense cytosolic tubulin datasets.

Fraunhofer diffraction at the two-dimensional quadratically distorted (QD) grating

A two-dimensional (2D) mathematical model of quadratically distorted (QD) grating is established with the principles of Fraunhofer diffraction and Fourier optics. A discrete sampling method is applied for finding a numerical solution of the diffraction pattern of QD grating. An optimized working phase term, which determines the balanced energies and high efficiency of multi-plane images, can be obtained by the bisection algorithm. To confirm the analytical approach described above, the results have been compared with those obtained using a classical numerical model based on Fraunhofer diffraction theory and a fast Fourier transform (FFT) algorithm. The results show that our analytical approach allows the precise design of QD grating and improves the optical performance of simultaneous multi-plane imaging system. An optical setup based on our well-designed QD grating has been appended to the camera port of a commercial microscope, and some preliminary microscopy images have been successfully obtained. Further upgrade of our analytical model is in progress to improve the image quality and promote the applications.

Measurement precision bounds on aberrated single molecule emission patterns

Single-Molecule Localization Microscopy (SMLM) has revolutionized the study of biological phenomena by providing exquisite nanoscale spatial resolution. However, optical aberrations induced by sample and system imperfections distort the single molecule emission patterns (i.e. PSFs), leading to reduced precision and resolution of SMLM, particularly in three-dimensional (3D) applications. While various methods, both analytical and instrumental, have been employed to mitigate these aberrations, a comprehensive analysis of how different types of commonly encountered aberrations affect single molecule experiments and their image formation remains missing. In this study, we addressed this gap by conducting a quantitative study of the theoretical precision limit for position and wavefront distortion measurements in the presence of aberrations. Leveraging Fisher information and Cramér-Rao lower bound (CRLB), we quantitively analyzed and compared the effects of different aberration types, including index mismatch aberrations, on localization precision in both biplane and astigmatism 3D modalities as well as 2D SMLM imaging. Furthermore, we studied the achievable wavefront estimation precision from aberrated single molecule emission patterns, a pivot step for successful adaptive optics in SMLM through thick specimens. This analysis lays a quantitative foundation for the development and application of SMLM in whole-cells, tissues and with large field of view, providing in-depth insights into the behavior of different aberration types in single molecule imaging and thus generating theoretical guidelines for developing highly efficient aberration correction strategies and enhancing the precision and reliability of 3D SMLM.

Imagining the future of optical microscopy: everything, everywhere, all at once

The optical microscope has revolutionized biology since at least the 17th Century. Since then, it has progressed from a largely observational tool to a powerful bioanalytical platform. However, realizing its full potential to study live specimens is hindered by a daunting array of technical challenges. Here, we delve into the current state of live imaging to explore the barriers that must be overcome and the possibilities that lie ahead. We venture to envision a future where we can visualize and study everything, everywhere, all at once - from the intricate inner workings of a single cell to the dynamic interplay across entire organisms, and a world where scientists could access the necessary microscopy technologies anywhere.

Fluorescence Microscopy: a statistics-optics perspective

Fundamental properties of light unavoidably impose features on images collected using fluorescence microscopes. Modeling these features is ever more important in quantitatively interpreting microscopy images collected at scales on par or smaller than light's wavelength. Here we review the optics responsible for generating fluorescent images, fluorophore properties, microscopy modalities leveraging properties of both light and fluorophores, in addition to the necessarily probabilistic modeling tools imposed by the stochastic nature of light and measurement.

Recent Advances in Single-Molecule Tracking and Imaging Techniques

Since the early 1990s, single-molecule detection in solution at room temperature has enabled direct observation of single biomolecules at work in real time and under physiological conditions, providing insights into complex biological systems that the traditional ensemble methods cannot offer. In particular, recent advances in single-molecule tracking techniques allow researchers to follow individual biomolecules in their native environments for a timescale of seconds to minutes, revealing not only the distinct pathways these biomolecules take for downstream signaling but also their roles in supporting life. In this review, we discuss various single-molecule tracking and imaging techniques developed to date, with an emphasis on advanced three-dimensional (3D) tracking systems that not only achieve ultrahigh spatiotemporal resolution but also provide sufficient working depths suitable for tracking single molecules in 3D tissue models. We then summarize the observables that can be extracted from the trajectory data. Methods to perform single-molecule clustering analysis and future directions are also discussed.

Design strategy for a dual-wedge prism imaging spectrometer in spectroscopic nanoscopy

Spectroscopic single-molecule localization microscopy (sSMLM, or spectroscopic nanoscopy) has been established as a key tool in functional super-resolution imaging by providing spatial and spectral information of single molecules at nanoscale resolution. A recently developed dual-wedge prism (DWP) imaging spectrometer, a monolithic optical component, has broadened the accessibility of sSMLM with an improved imaging resolution of more than 40%. It also improved the system reliability by reducing the number of discrete optical components. However, achieving its optimal performance requires the comprehensive understanding of the underlying constraints of the key system parameters, such as the refractive index of the DWP, spectral dispersion (SD), axial separation for three-dimensional (3D) biplane reconstruction, and the overall dimensional constraints. In this work, we present a generalized design principle for the DWP imaging spectrometer. Specifically, we develop the theoretical framework capturing the influence of the primary design parameters, including the achievable SD and localization performance, for different design cases. It further establishes the workflow to design and optimize the DWP imaging spectrometer for better multi-color functional imaging. This will give practical guidance for users to easily design the DWP imaging spectrometer, allowing for straightforward 3D sSMLM implementation.

Snapshot volumetric imaging with engineered point-spread functions

The biological world involves intracellular and intercellular interactions that occur at high speed, at multiple scales and in three dimensions. Acquiring 3D images, however, typically requires a compromise in either spatial or temporal resolution compared to 2D imaging. Conventional 2D fluorescence imaging provides high spatial resolution but requires plane-by-plane imaging of volumes. Conversely, snapshot methods such as light-field microscopy allow video-rate imaging, but at the cost of spatial resolution. Here we introduce 3D engineered point-spread function microscopy (3D-EPM), enabling snapshot imaging of real-world 3D extended biological structures while retaining the native resolution of the microscope in space and time. Our new computational recovery strategy is the key to volumetrically reconstructing arbitrary 3D structures from the information encapsulated in 2D raw EPM images. We validate our technique on both point-like and extended samples, and demonstrate its power by imaging the intracellular motion of chloroplasts undergoing cyclosis in a sample of Egeria densa. Our technique represents a generalised computational methodology for 3D image recovery which is readily adapted to a diverse range of existing microscopy platforms and engineered point-spread functions. We therefore expect it to find broad applicability in the study of rapid biological dynamics in 3D.

GANscan: continuous scanning microscopy using deep learning deblurring

Most whole slide imaging (WSI) systems today rely on the "stop-and-stare" approach, where, at each field of view, the scanning stage is brought to a complete stop before the camera snaps a picture. This procedure ensures that each image is free of motion blur, which comes at the expense of long acquisition times. In order to speed up the acquisition process, especially for large scanning areas, such as pathology slides, we developed an acquisition method in which the data is acquired continuously while the stage is moving at high speeds. Using generative adversarial networks (GANs), we demonstrate this ultra-fast imaging approach, referred to as GANscan, which restores sharp images from motion blurred videos. GANscan allows us to complete image acquisitions at 30x the throughput of stop-and-stare systems. This method is implemented on a Zeiss Axio Observer Z1 microscope, requires no specialized hardware, and accomplishes successful reconstructions at stage speeds of up to 5000 μm/s. We validate the proposed method by imaging H&E stained tissue sections. Our method not only retrieves crisp images from fast, continuous scans, but also adjusts for defocusing that occurs during scanning within +/- 5 μm. Using a consumer GPU, the inference runs at <20 ms/ image.

Flexible Multiplane Structured Illumination Microscope with a Four-Camera Detector

Fluorescence microscopy provides an unparalleled tool for imaging biological samples. However, producing high-quality volumetric images quickly and without excessive complexity remains a challenge. Here, we demonstrate a four-camera structured illumination microscope (SIM) capable of simultaneously imaging multiple focal planes, allowing for the capture of 3D fluorescent images without any axial movement of the sample. This setup allows for the acquisition of many different 3D imaging modes, including 3D time lapses, high-axial-resolution 3D images, and large 3D mosaics. We imaged mitochondrial motions in live cells, neuronal structure in Drosophila larvae, and imaged up to 130 µm deep into mouse brain tissue. After SIM processing, the resolution measured using one of the four cameras improved from 357 nm to 253 nm when using a 30×/1.05 NA objective.

Imaging whole-brain activity to understand behavior

The brain evolved to produce behaviors that help an animal inhabit the natural world. During natural behaviors, the brain is engaged in many levels of activity from the detection of sensory inputs to decision-making to motor planning and execution. To date, most brain studies have focused on small numbers of neurons that interact in limited circuits. This allows analyzing individual computations or steps of neural processing. During behavior, however, brain activity must integrate multiple circuits in different brain regions. The activities of different brain regions are not isolated, but may be contingent on one another. Coordinated and concurrent activity within and across brain areas is organized by (1) sensory information from the environment, (2) the animal's internal behavioral state, and (3) recurrent networks of synaptic and non-synaptic connectivity. Whole-brain recording with cellular resolution provides a new opportunity to dissect the neural basis of behavior, but whole-brain activity is also mutually contingent on behavior itself. This is especially true for natural behaviors like navigation, mating, or hunting, which require dynamic interaction between the animal, its environment, and other animals. In such behaviors, the sensory experience of an unrestrained animal is actively shaped by its movements and decisions. Many of the signaling and feedback pathways that an animal uses to guide behavior only occur in freely moving animals. Recent technological advances have enabled whole-brain recording in small behaving animals including nematodes, flies, and zebrafish. These whole-brain experiments capture neural activity with cellular resolution spanning sensory, decision-making, and motor circuits, and thereby demand new theoretical approaches that integrate brain dynamics with behavioral dynamics. Here, we review the experimental and theoretical methods that are being employed to understand animal behavior and whole-brain activity, and the opportunities for physics to contribute to this emerging field of systems neuroscience.

Answering some questions about structured illumination microscopy

Structured illumination microscopy (SIM) provides images of fluorescent objects at an enhanced resolution greater than that of conventional epifluorescence wide-field microscopy. Initially demonstrated in 1999 to enhance the lateral resolution twofold, it has since been extended to enhance axial resolution twofold (2008), applied to live-cell imaging (2009) and combined with myriad other techniques, including interferometric detection (2008), confocal microscopy (2010) and light sheet illumination (2012). Despite these impressive developments, SIM remains, perhaps, the most poorly understood 'super-resolution' method. In this article, we provide answers to the 13 questions regarding SIM proposed by Prakash et al. along with answers to a further three questions. After providing a general overview of the technique and its developments, we explain why SIM as normally used is still diffraction-limited. We then highlight the necessity for a non-polynomial, and not just nonlinear, response to the illuminating light in order to make SIM a true, diffraction-unlimited, super-resolution technique. In addition, we present a derivation of a real-space SIM reconstruction approach that can be used to process conventional SIM and image scanning microscopy (ISM) data and extended to process data with quasi-arbitrary illumination patterns. Finally, we provide a simple bibliometric analysis of SIM development over the past two decades and provide a short outlook on potential future work. This article is part of the Theo Murphy meeting issue 'Super-resolution structured illumination microscopy (part 2)'.

Monolithic dual-wedge prism-based spectroscopic single-molecule localization microscopy

By manipulating the spectral dispersion of detected photons, spectroscopic single-molecule localization microscopy (sSMLM) permits concurrent high-throughput single-molecular spectroscopic analysis and imaging. Despite its promising potential, using discrete optical components and managing the delicate balance between spectral dispersion and spatial localization compromise its performance, including non-uniform spectral dispersion, high transmission loss of grating, high optical alignment demands, and reduced precision. We designed a dual-wedge prism (DWP)-based monolithic imaging spectrometer to overcome these challenges. We optimized the DWP for spectrally dispersing focused beam without deviation and with minimal wavefront error. We integrated all components into a compact assembly, minimizing total transmission loss and significantly reducing optical alignment requirements. We show the feasibility of DWP using ray-tracing and numerical simulations. We validated our numerical simulations by experimentally imaging individual nanospheres and confirmed that DWP-sSMLM achieved much improved spatial and spectral precisions of grating-based sSMLM. We also demonstrated DWP-sSMLM in 3D multi-color imaging of cells.

Imaging Dynamic Processes in Multiple Dimensions and Length Scales

Optical microscopy has become an invaluable tool for investigating complex samples. Over the years, many advances to optical microscopes have been made that have allowed us to uncover new insights into the samples studied. Dynamic changes in biological and chemical systems are of utmost importance to study. To probe these samples, multidimensional approaches have been developed to acquire a fuller understanding of the system of interest. These dimensions include the spatial information, such as the three-dimensional coordinates and orientation of the optical probes, and additional chemical and physical properties through combining microscopy with various spectroscopic techniques. In this review, we survey the field of multidimensional microscopy and provide an outlook on the field and challenges that may arise. Expected final online publication date for the Annual Review of Physical Chemistry, Volume 73 is April 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Single-molecule tracking technologies for quantifying the dynamics of gene regulation in cells, tissue and embryos

For decades, we have relied on population and time-averaged snapshots of dynamic molecular scale events to understand how genes are regulated during development and beyond. The advent of techniques to observe single-molecule kinetics in increasingly endogenous contexts, progressing from in vitro studies to living embryos, has revealed how much we have missed. Here, we provide an accessible overview of the rapidly expanding family of technologies for single-molecule tracking (SMT), with the goal of enabling the reader to critically analyse single-molecule studies, as well as to inspire the application of SMT to their own work. We start by overviewing the basics of and motivation for SMT experiments, and the trade-offs involved when optimizing parameters. We then cover key technologies, including fluorescent labelling, excitation and detection optics, localization and tracking algorithms, and data analysis. Finally, we provide a summary of selected recent applications of SMT to study the dynamics of gene regulation.

Multifocal imaging for precise, label-free tracking of fast biological processes in 3D

Many biological processes happen on a nano- to millimeter scale and within milliseconds. Established methods such as confocal microscopy are suitable for precise 3D recordings but lack the temporal or spatial resolution to resolve fast 3D processes and require labeled samples. Multifocal imaging (MFI) allows high-speed 3D imaging but is limited by the compromise between high spatial resolution and large field-of-view (FOV), and the requirement for bright fluorescent labels. Here, we provide an open-source 3D reconstruction algorithm for multi-focal images that allows using MFI for fast, precise, label-free tracking spherical and filamentous structures in a large FOV and across a high depth. We characterize fluid flow and flagellar beating of human and sea urchin sperm with a z-precision of 0.15 µm, in a volume of 240 × 260 × 21 µm, and at high speed (500 Hz). The sampling volume allowed to follow sperm trajectories while simultaneously recording their flagellar beat. Our MFI concept is cost-effective, can be easily implemented, and does not rely on object labeling, which renders it broadly applicable.

Information-Efficient, Off-Center Sampling Results in Improved Precision in 3D Single-Particle Tracking Microscopy

In this work, we present a 3D single-particle tracking system that can apply tailored sampling patterns to selectively extract photons that yield the most information for particle localization. We demonstrate that off-center sampling at locations predicted by Fisher information utilizes photons most efficiently. When performing localization in a single dimension, optimized off-center sampling patterns gave doubled precision compared to uniform sampling. A ~20% increase in precision compared to uniform sampling can be achieved when a similar off-center pattern is used in 3D localization. Here, we systematically investigated the photon efficiency of different emission patterns in a diffraction-limited system and achieved higher precision than uniform sampling. The ability to maximize information from the limited number of photons demonstrated here is critical for particle tracking applications in biological samples, where photons may be limited.

Recurrent neural network-based volumetric fluorescence microscopy

Volumetric imaging of samples using fluorescence microscopy plays an important role in various fields including physical, medical and life sciences. Here we report a deep learning-based volumetric image inference framework that uses 2D images that are sparsely captured by a standard wide-field fluorescence microscope at arbitrary axial positions within the sample volume. Through a recurrent convolutional neural network, which we term as Recurrent-MZ, 2D fluorescence information from a few axial planes within the sample is explicitly incorporated to digitally reconstruct the sample volume over an extended depth-of-field. Using experiments on C. elegans and nanobead samples, Recurrent-MZ is demonstrated to significantly increase the depth-of-field of a 63×/1.4NA objective lens, also providing a 30-fold reduction in the number of axial scans required to image the same sample volume. We further illustrated the generalization of this recurrent network for 3D imaging by showing its resilience to varying imaging conditions, including e.g., different sequences of input images, covering various axial permutations and unknown axial positioning errors. We also demonstrated wide-field to confocal cross-modality image transformations using Recurrent-MZ framework and performed 3D image reconstruction of a sample using a few wide-field 2D fluorescence images as input, matching confocal microscopy images of the same sample volume. Recurrent-MZ demonstrates the first application of recurrent neural networks in microscopic image reconstruction and provides a flexible and rapid volumetric imaging framework, overcoming the limitations of current 3D scanning microscopy tools.

Fast, multiplane line-scan confocal microscopy using axially distributed slits

The inherent constraints on resolution, speed and field of view have hindered the development of high-speed, three-dimensional microscopy techniques over large scales. Here, we present a multiplane line-scan imaging strategy, which uses a series of axially distributed reflecting slits to probe different depths within a sample volume. Our technique enables the simultaneous imaging of an optically sectioned image stack with a single camera at frame rates of hundreds of hertz, without the need for axial scanning. We demonstrate the applicability of our system to monitor fast dynamics in biological samples by performing calcium imaging of neuronal activity in mouse brains and voltage imaging of cardiomyocytes in cardiac samples.

Analysing errors in single-molecule localisation microscopy

In single molecule localisation microscopy (SMLM) a super-resolution image of the distribution of fluorophores in the sample is built up from the localised positions of many individual molecules. It has become widely used due to its experimental simplicity and the high resolution that can be achieved. However, the factors which limit resolution in a reconstructed image, and the artefacts which can be present, are completely different to those present in standard fluorescent microscopy techniques. Artefacts may be difficult for users to identify, particularly as they can cause images to appear falsely sharp, an effect called artificial sharpening. Here we discuss the different sources of error and bias in SMLM, and the methods available for avoiding or detecting them.

Influence of FcRn binding properties on the gastrointestinal absorption and exposure profile of Fc molecules

The neonatal Fc receptor (FcRn) represents a transport system with the potential to facilitate absorption of biologics across the gastrointestinal barrier. How biologics interact with FcRn to enable their gastrointestinal absorption, and how these interactions might be optimized in a biological therapeutic are not well understood. Thus, we studied the absorption of Fc molecules from the intestine using three IgG₄-derived Fc variants with different, pH-dependent FcRn binding and release profiles. Using several different intestinal models, we consistently observed that FcRn binding affinity correlated with transcytosis. Our findings support targeting FcRn to enable intestinal absorption of biologics and highlight additional strategic considerations for future work.

Three-dimensional single molecule localization close to the coverslip: a comparison of methods exploiting supercritical angle fluorescence

The precise spatial localization of single molecules in three dimensions is an important basis for single molecule localization microscopy (SMLM) and tracking. At distances up to a few hundred nanometers from the coverslip, evanescent wave coupling into the glass, also known as supercritical angle fluorescence (SAF), can strongly improve the axial precision, thus facilitating almost isotropic localization performance. Specific detection systems, introduced as Supercritical angle localization microscopy (SALM) or Direct optical nanoscopy with axially localized detection (DONALD), have been developed to exploit SAF in modified two-channel imaging schemes. Recently, our group has shown that off-focus microscopy, i.e., imaging at an intentional slight defocus, can perform equally well, but uses only a single detection arm. Here we compare SALM, off-focus imaging and the most commonly used 3D SMLM techniques, namely cylindrical lens and biplane imaging, regarding 3D localization in close proximity to the coverslip. We show that all methods gain from SAF, which leaves a high detection NA as the only major key requirement to unlock the SAF benefit. We find parameter settings for cylindrical lens and biplane imaging for highest z-precision. Further, we compare the methods in view of robustness to aberrations, fixed dipole emission and double-emitter events. We show that biplane imaging provides the best overall performance and support our findings by DNA-PAINT experiments on DNA-nanoruler samples. Our study sheds light on the effects of SAF for SMLM and is helpful for researchers who plan to employ localization-based 3D nanoscopy close to the coverslip.

Mesoscale Modeling and Single-Nucleosome Tracking Reveal Remodeling of Clutch Folding and Dynamics in Stem Cell Differentiation

Nucleosomes form heterogeneous groups in vivo, named clutches. Clutches are smaller and less dense in mouse embryonic stem cells (ESCs) compared to neural progenitor cells (NPCs). Using coarse-grained modeling of the pluripotency Pou5f1 gene, we show that the genome-wide clutch differences between ESCs and NPCs can be reproduced at a single gene locus. Larger clutch formation in NPCs is associated with changes in the compaction and internucleosome contact probability of the Pou5f1 fiber. Using single-molecule tracking (SMT), we further show that the core histone protein H2B is dynamic, and its local mobility relates to the structural features of the chromatin fiber. H2B is less stable and explores larger areas in ESCs compared to NPCs. The amount of linker histone H1 critically affects local H2B dynamics. Our results have important implications for how nucleosome organization and H2B dynamics contribute to regulate gene activity and cell identity.

Fundamental precision bounds for three-dimensional optical localization microscopy using self-interference digital holography

Localization based microscopy using self-interference digital holography (SIDH) provides three-dimensional (3D) positional information about point sources with nanometer scale precision. To understand the performance limits of SIDH, here we calculate the theoretical limit to localization precision for SIDH when designed with two different configurations. One configuration creates the hologram using a plane wave and a spherical wave while the second configuration creates the hologram using two spherical waves. We further compare the calculated precision bounds to the 3D single molecule localization precision from different Point Spread Functions. SIDH results in almost constant localization precision in all three dimensions for a 20 µm thick depth of field. For high signal-to-background ratio (SBR), SIDH on average achieves better localization precision. For lower SBR values, the large size of the hologram on the detector becomes a problem, and PSF models perform better.

Autofocusing technologies for whole slide imaging and automated microscopy

Whole slide imaging (WSI) has moved digital pathology closer to diagnostic practice in recent years. Due to the inherent tissue topography variability, accurate autofocusing remains a critical challenge for WSI and automated microscopy systems. The traditional focus map surveying method is limited in its ability to acquire a high degree of focus points while still maintaining high throughput. Real-time approaches decouple image acquisition from focusing, thus allowing for rapid scanning while maintaining continuous accurate focus. This work reviews the traditional focus map approach and discusses the choice of focus measure for focal plane determination. It also discusses various real-time autofocusing approaches including reflective-based triangulation, confocal pinhole detection, low-coherence interferometry, tilted sensor approach, independent dual sensor scanning, beam splitter array, phase detection, dual-LED illumination and deep-learning approaches. The technical concepts, merits and limitations of these methods are explained and compared to those of a traditional WSI system. This review may provide new insights for the development of high-throughput automated microscopy imaging systems that can be made broadly available and utilizable without loss of capacity.

High-contrast multifocus microscopy with a single camera and z-splitter prism

Optical microscopy has been an indispensable tool for studying complex biological systems, but is often hampered by problems of speed and complexity when performing 3D volumetric imaging. Here, we present a multifocus imaging strategy based on the use of a simple z-splitter prism that can be assembled from off-the-shelf components. Our technique enables a widefield image stack to be distributed onto a single camera and recorded simultaneously. We exploit the volumetric nature of our image acquisition by further introducing a novel extended-volume 3D deconvolution strategy to suppress far-out-of-focus fluorescence background to significantly improve the contrast of our recorded images, conferring to our system a capacity for quasi-optical sectioning. By swapping in different z-splitter configurations, we can prioritize high speed or large 3D field-of-view imaging depending on the application of interest. Moreover, our system can be readily applied to a variety of imaging modalities in addition to fluorescence, such as phase-contrast and darkfield imaging. Because of its simplicity, versatility, and performance, we believe our system will be a useful tool for general biological or biomedical imaging applications.

Fourier DiffuserScope: single-shot 3D Fourier light field microscopy with a diffuser

Light field microscopy (LFM) uses a microlens array (MLA) near the sensor plane of a microscope to achieve single-shot 3D imaging of a sample without any moving parts. Unfortunately, the 3D capability of LFM comes with a significant loss of lateral resolution at the focal plane. Placing the MLA near the pupil plane of the microscope, instead of the image plane, can mitigate the artifacts and provide an efficient forward model, at the expense of field-of-view (FOV). Here, we demonstrate improved resolution across a large volume with Fourier DiffuserScope, which uses a diffuser in the pupil plane to encode 3D information, then computationally reconstructs the volume by solving a sparsity-constrained inverse problem. Our diffuser consists of randomly placed microlenses with varying focal lengths; the random positions provide a larger FOV compared to a conventional MLA, and the diverse focal lengths improve the axial depth range. To predict system performance based on diffuser parameters, we, for the first time, establish a theoretical framework and design guidelines, which are verified by numerical simulations, and then build an experimental system that achieves < 3 µm lateral and 4 µm axial resolution over a 1000 × 1000 × 280 µm³ volume. Our diffuser design outperforms the MLA used in LFM, providing more uniform resolution over a larger volume, both laterally and axially.

Three-Dimensional Single Particle Tracking and Its Applications in Confined Environments

Single particle tracking (SPT) has proven to be a powerful technique in studying molecular dynamics in complicated systems. We review its recent development, including three-dimensional (3D) SPT and its applications in probing nanostructures and molecule-surface interactions that are important to analytical chemical processes. Several frequently used 3D SPT techniques are introduced. Especially of interest are those based on point spread function engineering, which are simple in instrumentation and can be easily adapted and used in analytical labs. Corresponding data analysis methods are briefly discussed. We present several important case studies, with a focus on probing mass transport and molecule-surface interactions in confined environments. The presented studies demonstrate the great potential of 3D SPT for understanding fundamental phenomena in confined space, which will enable us to predict basic principles involved in chemical recognition, separation, and analysis, and to optimize mass transport and responses by structural design and optimization.

Three-dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval

Single-molecule localization microscopy is a powerful tool for visualizing subcellular structures, interactions and protein functions in biological research. However, inhomogeneous refractive indices inside cells and tissues distort the fluorescent signal emitted from single-molecule probes, which rapidly degrades resolution with increasing depth. We propose a method that enables the construction of an in situ 3D response of single emitters directly from single-molecule blinking datasets, and therefore allows their locations to be pinpointed with precision that achieves the Cramér-Rao lower bound and uncompromised fidelity. We demonstrate this method, named in situ PSF retrieval (INSPR), across a range of cellular and tissue architectures, from mitochondrial networks and nuclear pores in mammalian cells to amyloid-β plaques and dendrites in brain tissues and elastic fibers in developing cartilage of mice. This advancement expands the routine applicability of super-resolution microscopy from selected cellular targets near coverslips to intra- and extracellular targets deep inside tissues.

Snapshot multifocal light field microscopy

Light field microscopy (LFM) is an emerging technology for high-speed wide-field 3D imaging by capturing 4D light field of 3D volumes. However, its 3D imaging capability comes at a cost of lateral resolution. In addition, the lateral resolution is not uniform across depth in the light field dconvolution reconstructions. To address these problems, here, we propose a snapshot multifocal light field microscopy (MFLFM) imaging method. The underlying concept of the MFLFM is to collect multiple focal shifted light fields simultaneously. We show that by focal stacking those focal shifted light fields, the depth-of-field (DOF) of the LFM can be further improved but without sacrificing the lateral resolution. Also, if all differently focused light fields are utilized together in the deconvolution, the MFLFM could achieve a high and uniform lateral resolution within a larger DOF. We present a house-built MFLFM system by placing a diffractive optical element at the Fourier plane of a conventional LFM. The optical performance of the MFLFM are analyzed and given. Both simulations and proof-of-principle experimental results are provided to demonstrate the effectiveness and benefits of the MFLFM. We believe that the proposed snapshot MFLFM has potential to enable high-speed and high resolution 3D imaging applications.

Spatial filter and its application in three-dimensional single molecule localization microscopy

Single molecule localization microscopy (SMLM) allows the imaging of cellular structures with resolutions five to ten times below the diffraction limit of optical microscopy. It was originally introduced as a two-dimensional technique based on the localization of single emitters as projection onto the x-y imaging plane. The determination of the axial position of a fluorescent emitter is only possible by additional information. Here we report a method (spatial filter SMLM (SFSMLM)) that allows to determine the axial positions of fluorescent molecules and nanoparticles on the nanometer scale by the usage of two spatial filters, which are placed in two otherwise identical emission detection channels. SFSMLM allows axial localization in a range of ca. 1.5 μm with a localization precision of 15 - 30 nm in axial direction. The technique was utilized for localizing and imaging small cellular structures - e.g. actin filaments, vesicles and mitochondria - in three dimensions.

Defocused imaging exploits supercritical-angle fluorescence emission for precise axial single molecule localization microscopy

Single molecule localization microscopy (SMLM) is one of the key techniques that break the classical resolution limit in optical imaging. It is based on taking multiple recordings of a sample, each showing only a sparse arrangement of spatially well separated fluorescent molecules which can be localized at nanometer precision. While localizing along the lateral directions is usually straightforward, estimating axial positions at a comparable precision is known to be much harder, which is due to the relatively large depth of focus provided by the microscope optics. Whenever a molecule is sufficiently close to the coverslip, it becomes feasible to draw additional information from near field coupling effects: super-critical angle fluorescence (SAF) appears and can be exploited to boost the axial localization precision. Here we propose defocused imaging as a SMLM strategy that is capable of leveraging the information contained in SAF. We show that, regarding axial localization precision, our approach is superior to established SAF-based approaches. At the same time it is simple and can be conducted on any research-grade microscope where controlled defocusing on the order of a few hundred nanometers is possible.

3D Tracking-Free Approach for Obtaining 3D Super-Resolution Information in Rotationally Symmetric Biostructures

Currently, it is highly desirable but still challenging to obtain high-resolution (<50 nm) three-dimensional (3D) super-resolution information on structures in fixed specimens as well as for dynamic processes in live cells. Here we introduce a simple approach, without using 3D super-resolution microscopy or real-time 3D particle tracking, to estimate 3D sub-diffraction-limited structural or dynamic information in rotationally symmetric biostructures. This is a postlocalization analysis that transforms 2D super-resolution images or 2D single-molecule localization distributions into their corresponding 3D spatial probability distributions on the basis of prior known structural knowledge. This analysis is ideal in cases where the ultrastructure of a cellular structure is known but the substructural localization of a particular (usually mobile) protein is not. The method has been successfully applied to achieve 3D structural and functional sub-diffraction-limited information for 25-300 nm subcellular organelles that meet the rotational symmetry requirement, such as nuclear pore complex, primary cilium, and microtubule. In this Article, we will provide comprehensive analyses of this method by using experimental data and computational simulations. Finally, open source code of the 2D to 3D transformation algorithm (MATLAB) and simulations (Python) have also been developed.

A Light-Field Metasurface for High-Resolution Single-Particle Tracking

Three-dimensional (3D) single-particle tracking (SPT) is a key tool for studying dynamic processes in the life sciences. However, conventional optical elements utilizing light fields impose an inherent trade-off between lateral and axial resolution, preventing SPT with high spatiotemporal resolution across an extended volume. We overcome the typical loss in spatial resolution that accompanies light-field-based approaches to obtain 3D information by placing a standard microscope coverslip patterned with a multifunctional, light-field metasurface on a specimen. This approach enables an otherwise unmodified microscope to gather 3D information at an enhanced spatial resolution. We demonstrate simultaneous tracking of multiple fluorescent particles within a large 0.5 × 0.5 × 0.3 mm³ volume using a standard epi-fluorescent microscope with submicron lateral and micron-level axial resolution.

Morphological and molecular identification of a new Kudoa thyrsites isolate in Mediterranean silver scabbardfish Lepidopus caudatus

Myxozoans of the genus Kudoa (Myxosporea, Multivalvulida) infect marine and estuarine fish species worldwide. Some Kudoa species are of concern to the seafood industry since they may generate macroscopic cysts in the fish host's musculature, or cause post mortem myoliquefaction, commonly known as 'soft flesh'. One of the economically most important species is K. thyrsites, a myoliquefactive myxosporean parasite that occurs in many wild and cultured marine fish species worldwide. Here we identified a K. thyrsites isolate as the causative agent of myoliquefaction in silver scabbardfish Lepidopus caudatus from the Alboran Sea (western Mediterranean Sea). For comparative and validation purposes, the morphological and molecular characteristics of the isolate were compared with fresh spores of a K. thyrsites isolate infecting Atlantic mackerel Scomber scombrus from the Norwegian Sea. Myxospores of both isolates shared a stellate appearance and contained 4 unequal pyriform polar capsules (1 large, 1 small and 2 intermediate). These morphological traits were consistent with all other previously described K. thyrsites isolates. Moreover, the small subunit rDNA sequences of the Mediterranean and Norwegian Sea isolates revealed 100% similarity, and matched 100% with K. thyrsites isolates previously recorded in myoliquefactive Atlantic mackerel from the North Sea and off southern England. The findings suggest that K. thyrsites is the primary cause of myoliquefaction in silver scabbardfish from the Alboran Sea. This report represents the first morphological and molecular characterization of K. thyrsites in the Mediterranean Sea. A set of new allometric characters is proposed as additional descriptors for more accurate and specific description of kudoid myxospores.

Using time-lapse fluorescence microscopy to study gene regulation

Time-lapse fluorescence microscopy is a powerful tool to study gene regulation. By probing fluorescent signals in single cells over extended period of time, this method can be used to study the dynamics, noise, movement, memory, inheritance, and coordination, of gene expression during cell growth, development, and differentiation. In combination with a flow-cell device, it can also measure gene regulation by external stimuli. Due to the single cell nature and the spatial/temporal capacity, this method can often provide information that is hard to get using other methods. Here, we review the standard experimental procedures and new technical developments in this field.

Design and simulation of a snapshot multi-focal interferometric microscope

Realizing both high temporal and spatial resolution across a large volume is a key challenge for 3D fluorescent imaging. Towards achieving this objective, we introduce an interferometric multifocus microscopy (iMFM) system, a combination of multifocus microscopy (MFM) with two opposing objective lenses. We show that the proposed iMFM is capable of simultaneously producing multiple focal plane interferometry that provides axial super-resolution and hence isotropic 3D resolution with a single exposure. We design and simulate the iMFM microscope by employing two special diffractive optical elements. The point spread function of this new iMFM microscope is simulated and the image formation model is given. For reconstruction, we use the Richardson-Lucy deconvolution algorithm with total variation regularization for 3D extended object recovery, and a maximum likelihood estimator (MLE) for single molecule tracking. A method for determining an initial axial position of the molecule is also proposed to improve the convergence of the MLE. We demonstrate both theoretically and numerically that isotropic 3D nanoscopic localization accuracy is achievable with an axial imaging range of 2um when tracking a fluorescent molecule in three dimensions and that the diffraction limited axial resolution can be improved by 3-4 times in the single shot wide-field 3D extended object recovery. We believe that iMFM will be a useful tool in 3D dynamic event imaging that requires both high temporal and spatial resolution.

Fidelity test for through-focus or volumetric type of optical imaging methods

Rapid increase in interest and applications of through-focus (TF) or volumetric type of optical imaging in biology and other areas has resulted in the development of several TF image collection methods. Achieving quantitative results from images requires standardization and optimization of image acquisition protocols. Several standardization protocols are available for conventional optical microscopy where a best-focus image is used, but to date, rigorous testing protocols do not exist for TF optical imaging. In this paper, we present a method to determine the fidelity of the TF optical data using the TF scanning optical microscopy images.

Fundamental Precision Bounds for Three-Dimensional Optical Localization Microscopy with Poisson Statistics

Point source localization is a problem of persistent interest in optical imaging. In particular, a number of widely used biological microscopy techniques rely on precise three-dimensional localization of single fluorophores. As emitter depth localization is more challenging than lateral localization, considerable effort has been spent on engineering the response of the microscope in a way that reveals increased depth information. Here, we prove the (sub)optimality of these approaches by deriving and comparing to the measurement-independent quantum Cramér-Rao bound (QCRB). We show that existing methods for depth localization with single-objective collection exceed the QCRB, and we gain insight into the bound by proposing an interferometer arrangement that approaches it. We also show that for light collection with two opposed objectives, an established interferometric technique globally reaches the QCRB in all three dimensions simultaneously, and so this represents an interesting case study from the point of view of quantum multiparameter estimation.

Through-focus or volumetric type of optical imaging methods: a review

In recent years, the use of through-focus (TF) or volumetric type of optical imaging has gained momentum in several areas such as biological imaging, microscopy, adaptive optics, material processing, optical data storage, and optical inspection. We provide a review of basic TF optical methods highlighting their design, major unique characteristics, and application space.

Enhancing Analytical Separations Using Super-Resolution Microscopy

Super-resolution microscopy is becoming an invaluable tool to investigate structure and dynamics driving protein interactions at interfaces. In this review, we highlight the applications of super-resolution microscopy for quantifying the physics and chemistry that occur between target proteins and stationary-phase supports during chromatographic separations. Our discussion concentrates on the newfound ability of super-resolved single-protein spectroscopy to inform theoretical parameters via quantification of adsorption-desorption dynamics, protein unfolding, and nanoconfined transport.

Dual light-emitting diode-based multichannel microscopy for whole-slide multiplane, multispectral and phase imaging

We report the development of a multichannel microscopy for whole-slide multiplane, multispectral and phase imaging. We use trinocular heads to split the beam path into 6 independent channels and employ a camera array for parallel data acquisition, achieving a maximum data throughput of approximately 1 gigapixel per second. To perform single-frame rapid autofocusing, we place 2 near-infrared light-emitting diodes (LEDs) at the back focal plane of the condenser lens to illuminate the sample from 2 different incident angles. A hot mirror is used to direct the near-infrared light to an autofocusing camera. For multiplane whole-slide imaging (WSI), we acquire 6 different focal planes of a thick specimen simultaneously. For multispectral WSI, we relay the 6 independent image planes to the same focal position and simultaneously acquire information at 6 spectral bands. For whole-slide phase imaging, we acquire images at 3 focal positions simultaneously and use the transport-of-intensity equation to recover the phase information. We also provide an open-source design to further increase the number of channels from 6 to 15. The reported platform provides a simple solution for multiplexed fluorescence imaging and multimodal WSI. Acquiring an instant focal stack without z-scanning may also enable fast 3-dimensional dynamic tracking of various biological samples.

Multiplane and Spectrally-Resolved Single Molecule Localization Microscopy with Industrial Grade CMOS cameras

This work explores the use of industrial grade CMOS cameras for single molecule localization microscopy (SMLM). We show that industrial grade CMOS cameras approach the performance of scientific grade CMOS cameras at a fraction of the cost. This makes it more economically feasible to construct high-performance imaging systems with multiple cameras that are capable of a diversity of applications. In particular we demonstrate the use of industrial CMOS cameras for biplane, multiplane and spectrally resolved SMLM. We also provide open-source software for simultaneous control of multiple CMOS cameras and for the reduction of the movies that are acquired to super-resolution images.

Observation of live chromatin dynamics in cells via 3D localization microscopy using Tetrapod point spread functions

We report the observation of chromatin dynamics in living budding yeast (Saccharomyces cerevisiae) cells, in three-dimensions (3D). Using dual color localization microscopy and employing a Tetrapod point spread function, we analyze the spatio-temporal dynamics of two fluorescently labeled DNA loci surrounding the GAL locus. From the measured trajectories, we obtain different dynamical characteristics in terms of inter-loci distance and temporal variance; when the GAL locus is activated, the 3D inter-loci distance and temporal variance increase compared to the inactive state. These changes are visible in spite of the large thermally- and biologically-driven heterogeneity in the relative motion of the two loci. Our observations are consistent with current euchromatin vs. heterochromatin models.

Characteristic rotational behaviors of rod-shaped cargo revealed by automated five-dimensional single particle tracking

We report an automated single particle tracking technique for tracking the x, y, z coordinates, azimuthal and elevation angles of anisotropic plasmonic gold nanorod probes in live cells. These five spatial coordinates are collectively referred to as 5D. This method overcomes a long-standing challenge in distinguishing rotational motions from translational motions in the z-axis in differential interference contrast microscopy to result in full disclosure of nanoscale motions with high accuracy. Transferrin-coated endocytic gold nanorod cargoes initially undergo active rotational diffusion and display characteristic rotational motions on the membrane. Then as the cargoes being enclosed in clathrin-coated pits, they slow down the active rotation and experience a quiet period before they restore active rotational diffusion after fission and eventually being transported away from the original entry spots. Finally, the 3D trajectories and the accompanying rotational motions of the cargoes are resolved accurately to render the intracellular transport process in live cells.Distinguishing rotational motions from translational motions in the z-axis has been a long-standing challenge. Here the authors develop a five-dimensional single particle tracking method to detect rotational behaviors of nanocargos during clathrin-mediated endocytosis and intracellular transport.

Highly efficient multicolor multifocus microscopy by optimal design of diffraction binary gratings

Multifocus microscopy (MFM) allows sensitive and fast three-dimensional imaging. It relies on the efficient design of diffraction phase gratings yielding homogeneous intensities in desired diffraction orders. Such performances are however guaranteed only for a specific wavelength. Here, we discuss a novel approach for designing binary phase gratings with dual color properties and improved diffraction efficiency for MFM. We simulate binary diffraction gratings with tunable phase shifts to explore its best diffraction performances. We report the design and fabrication of a binary array generator of 3 × 3 equal-intensity diffraction orders with 74% efficiency, 95% uniformity and dual color capability. The multicolor properties of this new design are highlighted by two-color MFM imaging. Finally, we discuss the basics of extending this approach to a variety of diffraction pattern designs.

Three-Dimensional Localization of Single Molecules for Super-Resolution Imaging and Single-Particle Tracking

Single-molecule super-resolution fluorescence microscopy and single-particle tracking are two imaging modalities that illuminate the properties of cells and materials on spatial scales down to tens of nanometers or with dynamical information about nanoscale particle motion in the millisecond range, respectively. These methods generally use wide-field microscopes and two-dimensional camera detectors to localize molecules to much higher precision than the diffraction limit. Given the limited total photons available from each single-molecule label, both modalities require careful mathematical analysis and image processing. Much more information can be obtained about the system under study by extending to three-dimensional (3D) single-molecule localization: without this capability, visualization of structures or motions extending in the axial direction can easily be missed or confused, compromising scientific understanding. A variety of methods for obtaining both 3D super-resolution images and 3D tracking information have been devised, each with their own strengths and weaknesses. These include imaging of multiple focal planes, point-spread-function engineering, and interferometric detection. These methods may be compared based on their ability to provide accurate and precise position information on single-molecule emitters with limited photons. To successfully apply and further develop these methods, it is essential to consider many practical concerns, including the effects of optical aberrations, field dependence in the imaging system, fluorophore labeling density, and registration between different color channels. Selected examples of 3D super-resolution imaging and tracking are described for illustration from a variety of biological contexts and with a variety of methods, demonstrating the power of 3D localization for understanding complex systems.

Single Particle Tracking: From Theory to Biophysical Applications

After three decades of developments, single particle tracking (SPT) has become a powerful tool to interrogate dynamics in a range of materials including live cells and novel catalytic supports because of its ability to reveal dynamics in the structure-function relationships underlying the heterogeneous nature of such systems. In this review, we summarize the algorithms behind, and practical applications of, SPT. We first cover the theoretical background including particle identification, localization, and trajectory reconstruction. General instrumentation and recent developments to achieve two- and three-dimensional subdiffraction localization and SPT are discussed. We then highlight some applications of SPT to study various biological and synthetic materials systems. Finally, we provide our perspective regarding several directions for future advancements in the theory and application of SPT.

A novel dual-color bifocal imaging system for single-molecule studies

In this paper, we report the design and implementation of a dual-color bifocal imaging (DBI) system that is capable of acquiring two spectrally distinct, spatially registered images of objects located in either same or two distinct focal planes. We achieve this by separating an image into two channels with distinct chromatic properties and independently focusing both images onto a single CCD camera. The two channels in our device are registered with subpixel accuracy, and long-term stability of the registered images with nanometer-precision was accomplished by reducing the drift of the images to ∼5 nm. We demonstrate the capabilities of our DBI system by imaging biomolecules labeled with spectrally distinct dyes and micro- and nano-sized spheres located in different focal planes.

Intensity-based axial localization approaches for multifocal plane microscopy

Multifocal plane microscopy (MUM) can be used to visualize biological samples in three dimensions over large axial depths and provides for the high axial localization accuracy that is needed in applications such as the three-dimensional tracking of single particles and super-resolution microscopy. This report analyzes the performance of intensity-based axial localization approaches as applied to MUM data using Fisher information calculations. In addition, a new non-parametric intensity-based axial location estimation method, Multi-Intensity Lookup Algorithm (MILA), is introduced that, unlike typical intensity-based methods that make use of a single intensity value per data image, utilizes multiple intensity values per data image in determining the axial location of a point source. MILA is shown to be robust against potential bias induced by differences in the sub-pixel location of the imaged point source. The method's effectiveness on experimental data is also evaluated.