Optimal point spread function design for 3D imaging

Augmented 3D super-resolution of fluorescence-free nanoparticles using enhanced dark-field illumination based on wavelength-modulation and a least-cubic algorithm

Augmented three-dimensional (3D) subdiffraction-limited resolution of fluorescence-free single-nanoparticles was achieved with wavelength-dependent enhanced dark-field (EDF) illumination and a least-cubic algorithm. Various plasmonic nanoparticles on a glass slide (i.e., gold nanoparticles, GNPs; silver nanoparticles, SNPs; and gold nanorods, GNRs) were imaged and sliced in the z-direction to a thickness of 10 nm. Single-particle images were then compared with simulation data. The 3D coordinates of individual GNP, SNP, and GNR nanoparticles (x, y, z) were resolved by fitting the data with 3D point spread functions using a least-cubic algorithm and collation. Final, 3D super-resolution microscopy (SRM) images were obtained by resolving 3D coordinates and their Cramér-Rao lower bound-based localization precisions in an image space (530 nm × 530 nm × 300 nm) with a specific voxel size (2.5 nm × 2.5 nm × 5 nm). Compared with the commonly used least-square method, the least-cubic method was more useful for finding the center in asymmetric cases (i.e., nanorods) with high precision and accuracy. This novel 3D fluorescence-free SRM technique was successfully applied to resolve the positions of various nanoparticles on glass and gold nanospots (in vitro) as well as in a living single cell (in vivo) with subdiffraction limited resolution in 3D.

Tracking single fluorescent particles in three dimensions via extremum seeking

The ability to track single fluorescent particles in three-dimensions with sub-diffraction limit precision as well as sub-millisecond temporal resolution has enabled the understanding of many biophysical phenomena at the nanometer scale. While there are several techniques for achieving this, most require complicated experimental setups that are expensive to implement. These methods can offer superb performance but their complexity may be overwhelming to the end-user whose aim is only to understand the feature being imaged. In this work, we describe a method for tracking a single fluorescent particle using a standard confocal or multi-photon microscope configuration. It relies only on the assumption that the relative position of the measurement point and the particle can be actuated and that the point spread function has a global maximum that coincides with the particle's position. The method uses intensity feedback to calculate real-time position commands that "seek" the extremum of the point spread function as the particle moves through its environment. We demonstrate the method by tracking a diffusing quantum dot in a hydrogel on a standard epifluorescent confocal microscope.

Fisher information theory for parameter estimation in single molecule microscopy: tutorial

Estimation of a parameter of interest from image data represents a task that is commonly carried out in single molecule microscopy data analysis. The determination of the positional coordinates of a molecule from its image, for example, forms the basis of standard applications such as single molecule tracking and localization-based super-resolution image reconstruction. Assuming that the estimator used recovers, on average, the true value of the parameter, its accuracy, or standard deviation, is then at best equal to the square root of the Cramér-Rao lower bound. The Cramér-Rao lower bound can therefore be used as a benchmark in the evaluation of the accuracy of an estimator. Additionally, as its value can be computed and assessed for different experimental settings, it is useful as an experimental design tool. This tutorial demonstrates a mathematical framework that has been specifically developed to calculate the Cramér-Rao lower bound for estimation problems in single molecule microscopy and, more broadly, fluorescence microscopy. The material includes a presentation of the photon detection process that underlies all image data, various image data models that describe images acquired with different detector types, and Fisher information expressions that are necessary for the calculation of the lower bound. Throughout the tutorial, examples involving concrete estimation problems are used to illustrate the effects of various factors on the accuracy of parameter estimation and, more generally, to demonstrate the flexibility of the mathematical framework.

Single-Molecule Tracking and Its Application in Biomolecular Binding Detection

In the past two decades significant advances have been made in single-molecule detection, which enables the direct observation of single biomolecules at work in real time and under physiological conditions. In particular, the development of single-molecule tracking (SMT) microscopy allows us to monitor the motion paths of individual biomolecules in living systems, unveiling the localization dynamics and transport modalities of the biomolecules that support the development of life. Beyond the capabilities of traditional camera-based tracking techniques, state-of-the-art SMT microscopies developed in recent years can record fluorescence lifetime while tracking a single molecule in the 3D space. This multiparameter detection capability can open the door to a wide range of investigations at the cellular or tissue level, including identification of molecular interaction hotspots and characterization of association/dissociation kinetics between molecules. In this review, we discuss various SMT techniques developed to date, with an emphasis on our recent development of the next generation 3D tracking system that not only achieves ultrahigh spatiotemporal resolution but also provides sufficient working depth suitable for live animal imaging. We also discuss the challenges that current SMT techniques are facing and the potential strategies to tackle those challenges.

Simultaneous measurement of emission color and 3D position of single molecules

We show that the position of single molecules in all three spatial dimensions can be estimated alongside its emission color by diffractive optics based design of the Point Spread Function (PSF). The phase in a plane conjugate to the aperture stop of the objective lens is modified by a diffractive structure that splits the spot on the camera into closely spaced diffraction orders. The distance between and the size of these sub-spots are a measure of the emission color. Estimation of the axial position is enabled by imprinting aberrations such as astigmatism and defocus onto the orders. The overall spot shape is fitted with a fully vectorial PSF model. Proof-of-principle experiments on quantum dots indicate that a spectral precision of 10 to 20 nm, an axial localization precision of 25 to 50 nm, and a lateral localization precision of 10 to 30 nm can be achieved over a 1 μm range of axial positions for on average 800 signal photons and 17 background photons/pixel. The method appears to be rather sensitive to PSF model errors such as aberrations, giving in particular rise to biases in the fitted wavelength of up to 15 nm.

Optical tracking of nanoscale particles in microscale environments

The trajectories of nanoscale particles through microscale environments record useful information about both the particles and the environments. Optical microscopes provide efficient access to this information through measurements of light in the far field from nanoparticles. Such measurements necessarily involve trade-offs in tracking capabilities. This article presents a measurement framework, based on information theory, that facilitates a more systematic understanding of such trade-offs to rationally design tracking systems for diverse applications. This framework includes the degrees of freedom of optical microscopes, which determine the limitations of tracking measurements in theory. In the laboratory, tracking systems are assemblies of sources and sensors, optics and stages, and nanoparticle emitters. The combined characteristics of such systems determine the limitations of tracking measurements in practice. This article reviews this tracking hardware with a focus on the essential functions of nanoparticles as optical emitters and microenvironmental probes. Within these theoretical and practical limitations, experimentalists have implemented a variety of tracking systems with different capabilities. This article reviews a selection of apparatuses and techniques for tracking multiple and single particles by tuning illumination and detection, and by using feedback and confinement to improve the measurements. Prior information is also useful in many tracking systems and measurements, which apply across a broad spectrum of science and technology. In the context of the framework and review of apparatuses and techniques, this article reviews a selection of applications, with particle diffusion serving as a prelude to tracking measurements in biological, fluid, and material systems, fabrication and assembly processes, and engineered devices. In so doing, this review identifies trends and gaps in particle tracking that might influence future research.

Lens-based fluorescence nanoscopy

The majority of studies of the living cell rely on capturing images using fluorescence microscopy. Unfortunately, for centuries, diffraction of light was limiting the spatial resolution in the optical microscope: structural and molecular details much finer than about half the wavelength of visible light (~200 nm) could not be visualized, imposing significant limitations on this otherwise so promising method. The surpassing of this resolution limit in far-field microscopy is currently one of the most momentous developments for studying the living cell, as the move from microscopy to super-resolution microscopy or 'nanoscopy' offers opportunities to study problems in biophysical and biomedical research at a new level of detail. This review describes the principles and modalities of present fluorescence nanoscopes, as well as their potential for biophysical and cellular experiments. All the existing nanoscopy variants separate neighboring features by transiently preparing their fluorescent molecules in states of different emission characteristics in order to make the features discernible. Usually these are fluorescent 'on' and 'off' states causing the adjacent molecules to emit sequentially in time. Each of the variants can in principle reach molecular spatial resolution and has its own advantages and disadvantages. Some require specific transitions and states that can be found only in certain fluorophore subfamilies, such as photoswitchable fluorophores, while other variants can be realized with standard fluorescent labels. Similar to conventional far-field microscopy, nanoscopy can be utilized for dynamical, multi-color and three-dimensional imaging of fixed and live cells, tissues or organisms. Lens-based fluorescence nanoscopy is poised for a high impact on future developments in the life sciences, with the potential to help solve long-standing quests in different areas of scientific research.

Multicolour localization microscopy by point-spread-function engineering

Super-resolution microscopy has revolutionized cellular imaging in recent years^1-4. Methods relying on sequential localization of single point emitters enable spatial tracking at ~10-40 nm resolution. Moreover, tracking and imaging in three dimensions is made possible by various techniques, including point-spread-function (PSF) engineering^5-9 -namely, encoding the axial (z) position of a point source in the shape that it creates in the image plane. However, a remaining challenge for localization-microscopy is efficient multicolour imaging - a task of the utmost importance for contextualizing biological data. Normally, multicolour imaging requires sequential imaging^{10, 11}, multiple cameras¹², or segmented dedicated fields of view^{13, 14}. Here, we demonstrate an alternate strategy, the encoding of spectral information (colour), in addition to 3D position, directly in the image. By exploiting chromatic dispersion, we design a new class of optical phase masks that simultaneously yield controllably different PSFs for different wavelengths, enabling simultaneous multicolour tracking or super-resolution imaging in a single optical path.

Single-molecule spectroscopy and imaging over the decades

As of 2015, it has been 26 years since the first optical detection and spectroscopy of single molecules in condensed matter. This area of science has expanded far beyond the early low temperature studies in crystals to include single molecules in cells, polymers, and in solution. The early steps relied upon high-resolution spectroscopy of inhomogeneously broadened optical absorption profiles of molecular impurities in solids at low temperatures. Spectral fine structure arising directly from the position-dependent fluctuations of the number of molecules in resonance led to the attainment of the single-molecule limit in 1989 using frequency-modulation laser spectroscopy. In the early 1990s, a variety of fascinating physical effects were observed for individual molecules, including imaging of the light from single molecules as well as observations of spectral diffusion, optical switching and the ability to select different single molecules in the same focal volume simply by tuning the pumping laser frequency. In the room temperature regime, researchers showed that bursts of light from single molecules could be detected in solution, leading to imaging and microscopy by a variety of methods. Studies of single copies of the green fluorescent protein also uncovered surprises, especially the blinking and photoinduced recovery of emitters, which stimulated further development of photoswitchable fluorescent protein labels. All of these early steps provided important fundamentals underpinning the development of super-resolution microscopy based on single-molecule localization and active control of emitting concentration. Current thrust areas include extensions to three-dimensional imaging with high precision, orientational analysis of single molecules, and direct measurements of photodynamics and transport properties for single molecules trapped in solution by suppression of Brownian motion. Without question, a huge variety of studies of single molecules performed by many talented scientists all over the world have extended our knowledge of the nanoscale and many microscopic mechanisms previously hidden by ensemble averaging.

Correcting field-dependent aberrations with nanoscale accuracy in three-dimensional single-molecule localization microscopy

The localization of single fluorescent molecules enables the imaging of molecular structure and dynamics with subdiffraction precision and can be extended to three dimensions using point spread function (PSF) engineering. However, the nanoscale accuracy of localization throughout a 3D single-molecule microscope's field of view has not yet been rigorously examined. By using regularly spaced subdiffraction apertures filled with fluorescent dyes, we reveal field-dependent aberrations as large as 50-100 nm and show that they can be corrected to less than 25 nm over an extended 3D focal volume. We demonstrate the applicability of this technique for two engineered PSFs, the double-helix PSF and the astigmatic PSF. We expect these results to be broadly applicable to 3D single-molecule tracking and superresolution methods demanding high accuracy.

Method for simultaneous localization and parameter estimation in particle tracking experiments

We present a numerical method for the simultaneous localization and parameter estimation of a fluorescent particle undergoing a discrete-time continuous-state Markov process. In particular, implementation of the method proposed in this work yields an approximation to the posterior density of the particle positions over time in addition to maximum likelihood estimates of fixed, unknown parameters. The method employs sequential Monte Carlo methods and can take into account complex, potentially nonlinear noise models, including shot noise and camera-specific readout noise, as well as a wide variety of motion models and observation models, including those representing recent engineered point spread functions. We demonstrate the technique by applying it to four scenarios, including a particle undergoing free, confined, and tethered diffusions.

Real-time adaptive drift correction for super-resolution localization microscopy

Super-resolution localization microscopy involves acquiring thousands of image frames of sparse collections of single molecules in the sample. The long acquisition time makes the imaging setup prone to drift, affecting accuracy and precision. Localization accuracy is generally improved by a posteriori drift correction. However, localization precision lost due to sample drifting out of focus cannot be recovered as the signal is originally detected at a lower peak signal. Here, we demonstrate a method of stabilizing a super-resolution localization microscope in three dimensions for extended periods of time with nanometer precision. Hence, no localization correction after the experiment is required to obtain super-resolved reconstructions. The method incorporates a closed-loop with a feedback signal generated from camera images and actuation on a 3D nanopositioning stage holding the sample.

Single-Molecule Spectroscopy, Imaging, and Photocontrol: Foundations for Super-Resolution Microscopy (Nobel Lecture)

The initial steps toward optical detection and spectroscopy of single molecules in condensed matter arose out of the study of inhomogeneously broadened optical absorption profiles of molecular impurities in solids at low temperatures. Spectral signatures relating to the fluctuations of the number of molecules in resonance led to the attainment of the single-molecule limit in 1989 using frequency-modulation laser spectroscopy. In the early 90s, many fascinating physical effects were observed for individual molecules, and the imaging of single molecules as well as observations of spectral diffusion, optical switching and the ability to select different single molecules in the same focal volume simply by tuning the pumping laser frequency provided important forerunners of the later super-resolution microscopy with single molecules. In the room temperature regime, imaging of single copies of the green fluorescent protein also uncovered surprises, especially the blinking and photoinduced recovery of emitters, which stimulated further development of photoswitchable fluorescent protein labels. Because each single fluorophore acts a light source roughly 1 nm in size, microscopic observation and localization of individual fluorophores is a key ingredient to imaging beyond the optical diffraction limit. Combining this with active control of the number of emitting molecules in the pumped volume led to the super-resolution imaging of Eric Betzig and others, a new frontier for optical microscopy beyond the diffraction limit. The background leading up to these observations is described and current developments are summarized.

Frequency analysis of a self-bending point spread function for 3D localization-based optical microscopy

We developed several approaches to characterize the recently reported self-bending point spread function for 3D localization-based light microscopy. Experimentally, we generated Gaussian, astigmatic, and self-bending point spread functions. We compared the optical transfer functions, ambiguity functions, and Fisher information of these point spread functions. Our comprehensive frequency-domain analysis describes quantitative tools for the development of engineered point spread functions for 3D imaging systems.

Single-Molecule Spectroscopy, Imaging, and Photocontrol: Foundations for Super-Resolution Microscopy (Nobel Lecture)

The initial steps toward optical detection and spectroscopy of single molecules in condensed matter arose out of the study of inhomogeneously broadened optical absorption profiles of molecular impurities in solids at low temperatures. Spectral signatures relating to the fluctuations of the number of molecules in resonance led to the attainment of the single-molecule limit in 1989 using frequency-modulation laser spectroscopy. In the early 90s, many fascinating physical effects were observed for individual molecules, and the imaging of single molecules as well as observations of spectral diffusion, optical switching and the ability to select different single molecules in the same focal volume simply by tuning the pumping laser frequency provided important forerunners of the later super-resolution microscopy with single molecules. In the room temperature regime, imaging of single copies of the green fluorescent protein also uncovered surprises, especially the blinking and photoinduced recovery of emitters, which stimulated further development of photoswitchable fluorescent protein labels. Because each single fluorophore acts a light source roughly 1 nm in size, microscopic observation and localization of individual fluorophores is a key ingredient to imaging beyond the optical diffraction limit. Combining this with active control of the number of emitting molecules in the pumped volume led to the super-resolution imaging of Eric Betzig and others, a new frontier for optical microscopy beyond the diffraction limit. The background leading up to these observations is described and current developments are summarized.

Precise Three-Dimensional Scan-Free Multiple-Particle Tracking over Large Axial Ranges with Tetrapod Point Spread Functions

We employ a novel framework for information-optimal microscopy to design a family of point spread functions (PSFs), the Tetrapod PSFs, which enable high-precision localization of nanoscale emitters in three dimensions over customizable axial (z) ranges of up to 20 μm with a high numerical aperture objective lens. To illustrate, we perform flow profiling in a microfluidic channel and show scan-free tracking of single quantum-dot-labeled phospholipid molecules on the surface of living, thick mammalian cells.

Determination of localization accuracy based on experimentally acquired image sets: applications to single molecule microscopy

Fluorescence microscopy is a photon-limited imaging modality that allows the study of subcellular objects and processes with high specificity. The best possible accuracy (standard deviation) with which an object of interest can be localized when imaged using a fluorescence microscope is typically calculated using the Cramér-Rao lower bound, that is, the inverse of the Fisher information. However, the current approach for the calculation of the best possible localization accuracy relies on an analytical expression for the image of the object. This can pose practical challenges since it is often difficult to find appropriate analytical models for the images of general objects. In this study, we instead develop an approach that directly uses an experimentally collected image set to calculate the best possible localization accuracy for a general subcellular object. In this approach, we fit splines, i.e. smoothly connected piecewise polynomials, to the experimentally collected image set to provide a continuous model of the object, which can then be used for the calculation of the best possible localization accuracy. Due to its practical importance, we investigate in detail the application of the proposed approach in single molecule fluorescence microscopy. In this case, the object of interest is a point source and, therefore, the acquired image set pertains to an experimental point spread function.

Spatial information in large-scale neural recordings

To record from a given neuron, a recording technology must be able to separate the activity of that neuron from the activity of its neighbors. Here, we develop a Fisher information based framework to determine the conditions under which this is feasible for a given technology. This framework combines measurable point spread functions with measurable noise distributions to produce theoretical bounds on the precision with which a recording technology can localize neural activities. If there is sufficient information to uniquely localize neural activities, then a technology will, from an information theoretic perspective, be able to record from these neurons. We (1) describe this framework, and (2) demonstrate its application in model experiments. This method generalizes to many recording devices that resolve objects in space and should be useful in the design of next-generation scalable neural recording systems.