Beyond Rayleigh's criterion: a resolution measure with application to single-molecule microscopy

Single-molecule localization microscopy at 2.4-fold resolution improvement with optical lattice pattern illumination

Traditional camera-based single-molecule localization microscopy (SMLM), with its high imaging resolution and localization throughput, has made significant advancements in biological and chemical researches. However, due to the limitation of the fluorescence signal-to-noise ratio (SNR) of a single molecule, its resolution is difficult to reach to 5 nm. Optical lattice produces a nondiffracting beam pattern that holds the potential to enhance microscope performance through its high contrast and penetration depth. Here, we propose a new method named LatticeFLUX which utilizes the wide-field optical lattice pattern illumination for individual molecule excitation and localization. We calculated the Cramér-Rao lower bound of LatticeFLUX resolution and proved that our method can improve the single molecule localization precision by 2.4 times compared with the traditional SMLM. We propose a scheme using 9-frame localization, which solves the problem of uneven lattice light illumination. Based on the experimental single-molecule fluorescence SNR, we coded the image reconstruction software to further verify the resolution enhancement capability of LatticeFLUX on simulated punctate DNA origami, line pairs, and cytoskeleton. LatticeFLUX confirms the feasibility of using 2D structured light illumination to obtain high single-molecule localization precision under high localization throughput. It paves the way for further implementation of ultra-high resolution full 3D structured-light-illuminated SMLM.

A statistical resolution measure of fluorescence microscopy with finite photons

First discovered by Ernest Abbe in 1873, the resolution limit of a far-field microscope is considered determined by the numerical aperture and wavelength of light, approximatelyλ 2 N A . With the advent of modern fluorescence microscopy and nanoscopy methods over the last century, this definition is insufficient to fully describe a microscope's resolving power. To determine the practical resolution limit of a fluorescence microscope, photon noise remains one essential factor yet to be incorporated in a statistics-based theoretical framework. We proposed an information density measure quantifying the theoretical resolving power of a fluorescence microscope in the condition of finite photons. The developed approach not only allows us to quantify the practical resolution limit of various fluorescence and super-resolution microscopy modalities but also offers the potential to predict the achievable resolution of a microscopy design under different photon levels.

Enhancing axial localization with wavefront control

Enhancing the ability to resolve axial details is crucial in three-dimensional optical imaging. We provide experimental evidence showcasing the ultimate precision achievable in axial localization using vortex beams. For Laguerre-Gauss (LG) beams, this remarkable limit can be attained with just a single intensity scan. This proof-of-principle demonstrates that microscopy techniques based on LG vortex beams can potentially benefit from the introduced quantum-inspired superresolution protocol.

Measuring small displacements of an optical point source with digital holography

The image of an optical point source is blurred due to light diffraction so that estimating small displacements of the point source with direct imaging demands elaborate processing on the observation data of a camera. Using quantum parameter estimation, we show that for the imaging systems with a real point spread function, any measurement basis constituted by a complete set of real-valued spatial-mode functions is optimal for estimating the displacement. For small displacements, we can concentrate the information about the value of displacement to the measurement of a few spatial modes, which can be selected in terms of the Fisher information distribution. We use digital holography with a phase-only spatial light modulator to implement two simple estimation strategies that are mainly based on the projection measurement of two spatial modes and the readout of a single pixel of a camera.

Obtaining 3D super-resolution images by utilizing rotationally symmetric structures and 2D-to-3D transformation

Super-resolution imaging techniques have provided unprecedentedly detailed information by surpassing the diffraction-limited resolution of light microscopy. However, in order to derive high quality spatial resolution, many of these techniques require high laser power, extended imaging time, dedicated sample preparation, or some combination of the three. These constraints are particularly evident when considering three-dimensional (3D) super-resolution imaging. As a result, high-speed capture of 3D super-resolution information of structures and dynamic processes within live cells remains both desirable and challenging. Recently, a highly effective approach to obtain 3D super-resolution information was developed that can be employed in commonly available laboratory microscopes. This development makes it both scientifically possible and financially feasible to obtain super-resolution 3D information under certain conditions. This is accomplished by converting 2D single-molecule localization data captured at high speed within subcellular structures and rotationally symmetric organelles. Here, a high-speed 2D single-molecule tracking and post-localization technique, known as single-point edge-excitation sub-diffraction (SPEED) microcopy, along with its 2D-to-3D transformation algorithm is detailed with special emphasis on the mathematical principles and Monte Carlo simulation validation of the technique.

Bioimaging with Upconversion Nanoparticles

Bioimaging enables the spatiotemporal visualization of biological processes at various scales empowered by a range of different imaging modalities and contrast agents. Upconversion nanoparticles (UCNPs) represent a distinct type of such contrast agents with the potential to transform bioimaging due to their unique optical properties and functional design flexibilities. This review explores and discusses the opportunities, challenges, and limitations that UCNPs exhibit as bioimaging probes and highlights applications with spatial dimensions ranging from the single nanoparticle level to cellular, tissue, and whole animal imaging. We further summarized recent advancements in bioimaging applications enabled by UCNPs, including super-resolution techniques and multimodal imaging methods, and provide a perspective on the future potential of UCNP-based technologies in bioimaging research and clinical translation. This review may provide a valuable resource for researchers interested in exploring and applying UCNP-based bioimaging technologies.

Optical quantum super-resolution imaging and hypothesis testing

Estimating the angular separation between two incoherent thermal sources is a challenging task for direct imaging, especially at lengths within the diffraction limit. Moreover, detecting the presence of multiple sources of different brightness is an even more severe challenge. We experimentally demonstrate two tasks for super-resolution imaging based on hypothesis testing and quantum metrology techniques. We can significantly reduce the error probability for detecting a weak secondary source, even for small separations. We reduce the experimental complexity to a simple interferometer: we show (1) our set-up is optimal for the state discrimination task, and (2) if the two sources are equally bright, then this measurement can super-resolve their angular separation. Using a collection baseline of 5.3 mm, we resolve the angular separation of two sources placed 15 μm apart at a distance of 1.0 m with a 1.7% accuracy - an almost 3-orders-of-magnitude improvement over shot-noise limited direct imaging.

Estimating the angular separation between two incoherent sources below the diffraction limit is challenging. Hypothesis testing and quantum state discrimination techniques are used to super-resolve sources of different brightness with a simple optical interferometer.

Quantitative, in situ visualization of intracellular insulin vesicles in pancreatic beta cells

Characterizing relationships between Zn²⁺, insulin, and insulin vesicles is of vital importance to the study of pancreatic beta cells. However, the precise content of Zn²⁺ and the specific location of insulin inside insulin vesicles are not clear, which hinders a thorough understanding of the insulin secretion process and diseases caused by blood sugar dysregulation. Here, we demonstrated the colocalization of Zn²⁺ and insulin in both single extracellular insulin vesicles and pancreatic beta cells by using an X-ray scanning coherent diffraction imaging (ptychography) technique. We also analyzed the elemental Zn²⁺ and Ca²⁺ contents of insulin vesicles using electron microscopy and energy dispersive spectroscopy (EDS) mapping. We found that the presence of Zn²⁺ is an important characteristic that can be used to distinguish insulin vesicles from other types of vesicles in pancreatic beta cells and that the content of Zn²⁺ is proportional to the size of insulin vesicles. By using dual-energy contrast X-ray microscopy and scanning transmission X-ray microscopy (STXM) image stacks, we observed that insulin accumulates in the off-center position of extracellular insulin vesicles. Furthermore, the spatial distribution of insulin vesicles and their colocalization with other organelles inside pancreatic beta cells were demonstrated using three-dimensional (3D) imaging by combining X-ray ptychography and an equally sloped tomography (EST) algorithm. This study describes a powerful method to univocally describe the location and quantitative analysis of intracellular insulin, which will be of great significance to the study of diabetes and other blood sugar diseases.

LuckyProfiler: an ImageJ plug-in capable of quantifying FWHM resolution easily and effectively for super-resolution images

Quantifying the resolution of a super-resolution image is vital for biologists trying to apply super-resolution microscopy in various research fields. Among the reported image resolution estimation methods, the one that calculates the full width at half maximum (FWHM) of line profile, called FWHM resolution, continues the traditional resolution criteria and has been popularly used by many researchers. However, quantifying the FWHM resolution of a super-resolution image is a time-consuming, labor-intensive, and error-prone process because this method typically involves a manual and careful selection of one or several of the smallest structures. In this paper, we investigate the influencing factors in FWHM resolution quantification systematically and present an ImageJ plug-in called LuckyProfiler for biologists so that they can have an easy and effective way of quantifying the FWHM resolution of super-resolution images.

State-vector geometry and guided-wave physics behind optical super-resolution

We examine the state-vector geometry and guided-wave physics underpinning spatial super-resolution, which can be attained in far-field linear microscopy via a combination of statistical analysis, quantum optics, and spatial mode demultiplexing. A suitably tailored guided-wave signal pickup is shown to provide an information channel that can distill the super-resolving spatial modes, thus enabling an estimation of sub-Rayleigh space intervals ξ. We derive closed-form analytical expressions describing the distribution of the ξ-estimation Fisher information over waveguide modes, showing that this information remains nonvanishing as ξ → 0, thus preventing the variance of ξ estimation from diverging at ξ → 0. We demonstrate that the transverse refractive index profile n_Q(r) tailored to support the optimal wave function ψ_Q(r) for super-resolving ξ estimation encodes the same information about ξ as the entire manifold of waveguide modes needed to represent ψ_Q(r). Unlike ψ_Q(r), n_Q(r) does not need a representation in a lengthy manifold of eigenmodes and can be found instead via adaptive feedback-controlled learning.

Invariance Property of the Fisher Information in Scattering Media

Determining the ultimate precision limit for measurements on a subwavelength particle with coherent laser light is a goal with applications in areas as diverse as biophysics and nanotechnology. Here, we demonstrate that surrounding such a particle with a complex scattering environment does, on average, not have any influence on the mean quantum Fisher information associated with measurements on the particle. As a remarkable consequence, the average precision that can be achieved when estimating the particle's properties is the same in the ballistic and in the diffusive scattering regime, independently of the particle's position within its nonabsorbing environment. This invariance law breaks down only in the regime of Anderson localization, due to increased C_{0}-speckle correlations. Finally, we show how these results connect to the mean quantum Fisher information achievable with spatially optimized input fields.

Imaging through a scattering medium: the Fisher information and the generalized Abbe limit

Enhanced-resolution imaging in complex scattering media is revisited from a parameter estimation perspective. A suitably defined Fisher information is shown to offer useful insights into the limiting precision of parameter estimation in a scattering environment and, hence, into the limiting spatial resolution that can be achieved in imaging-through-scattering settings. The Fisher information that defines this resolution limit via the Cramér-Rao lower bound is shown to scale with the number of adaptively controlled space-time modes of the probe field, suggesting a physically intuitive generalization of the Abbe limit to the spatial resolution attainable for complex scattering systems. In a conventional, direct-imaging microscopy setting, this bound is shown to converge to the canonical Abbe limit.

Advanced Data Analysis for Fluorescence-Lifetime Single-Molecule Localization Microscopy

Fluorescence-lifetime single molecule localization microscopy (FL-SMLM) adds the lifetime dimension to the spatial super-resolution provided by SMLM. Independent of intensity and spectrum, this lifetime information can be used, for example, to quantify the energy transfer efficiency in Förster Resonance Energy Transfer (FRET) imaging, to probe the local environment with dyes that change their lifetime in an environment-sensitive manner, or to achieve image multiplexing by using dyes with different lifetimes. We present a thorough theoretical analysis of fluorescence-lifetime determination in the context of FL-SMLM and compare different lifetime-fitting approaches. In particular, we investigate the impact of background and noise, and give clear guidelines for procedures that are optimized for FL-SMLM. We do also present and discuss our public-domain software package “Fluorescence-Lifetime TrackNTrace,” which converts recorded fluorescence microscopy movies into super-resolved FL-SMLM images.

Review of bio-optical imaging systems with a high space-bandwidth product

Optical imaging has served as a primary method to collect information about biosystems across scales-from functionalities of tissues to morphological structures of cells and even at biomolecular levels. However, to adequately characterize a complex biosystem, an imaging system with a number of resolvable points, referred to as a space-bandwidth product (SBP), in excess of one billion is typically needed. Since a gigapixel-scale far exceeds the capacity of current optical imagers, compromises must be made to obtain either a low spatial resolution or a narrow field-of-view (FOV). The problem originates from constituent refractive optics-the larger the aperture, the more challenging the correction of lens aberrations. Therefore, it is impractical for a conventional optical imaging system to achieve an SBP over hundreds of millions. To address this unmet need, a variety of high-SBP imagers have emerged over the past decade, enabling an unprecedented resolution and FOV beyond the limit of conventional optics. We provide a comprehensive survey of high-SBP imaging techniques, exploring their underlying principles and applications in bioimaging.

Information-Efficient, Off-Center Sampling Results in Improved Precision in 3D Single-Particle Tracking Microscopy

In this work, we present a 3D single-particle tracking system that can apply tailored sampling patterns to selectively extract photons that yield the most information for particle localization. We demonstrate that off-center sampling at locations predicted by Fisher information utilizes photons most efficiently. When performing localization in a single dimension, optimized off-center sampling patterns gave doubled precision compared to uniform sampling. A ~20% increase in precision compared to uniform sampling can be achieved when a similar off-center pattern is used in 3D localization. Here, we systematically investigated the photon efficiency of different emission patterns in a diffraction-limited system and achieved higher precision than uniform sampling. The ability to maximize information from the limited number of photons demonstrated here is critical for particle tracking applications in biological samples, where photons may be limited.

Deep learning in electron microscopy

Deep learning is transforming most areas of science and technology, including electron microscopy. This review paper offers a practical perspective aimed at developers with limited familiarity. For context, we review popular applications of deep learning in electron microscopy. Following, we discuss hardware and software needed to get started with deep learning and interface with electron microscopes. We then review neural network components, popular architectures, and their optimization. Finally, we discuss future directions of deep learning in electron microscopy.

Localization of Nonblinking Point Sources Using Higher-Order-Mode Detection and Optical Heterodyning: Developing a Strategy for Extending the Scope of Molecular, Super-resolution Imaging

While the stochastic, "blinking" nature of fluorescent systems has enabled the super-resolution of their localization by the fitting of their point-spread functions (PSFs), this strategy cannot be exploited for similar resolution of "nonblinking" systems, such as those that might be encountered in a coherent Raman experiment. An alternative method for subdiffraction-limited imaging lies in the exploitation of optical heterodyning. For example, if a Gaussian PSF (a TEM00 mode) of a point emitter is displaced with respect to the origin of the optical system, photons in the higher-order TEM modes carry information about that displacement. Information concerning the displacement can be extracted from photons in these higher-order modes. These photons can be collected by optical heterodyning, which exploits the large gain in a detector's response to an optical signal from an emitter coupled to a local oscillator, which is prepared in the TEM of interest, e.g., TEM10. We have generalized and developed the heterodyning technique to localize point emitters via the detection of higher-order spatial modes. We have developed a theoretical approach to find a practical estimation limit of the localization parameters using a realistic model that accounts for shot noise, background noise, and Gaussian noise. To demonstrate the applicability of the method, we designed experiments in which a laser is a surrogate for one and two point emitters. Using the Fisher information and its accompanying Cramér-Rao lower bound, we demonstrate super-resolution localization in these cases: we show that objects can be localized to roughly 2-3 orders of magnitude of their point-spread function's size for a given optical system. Finally and most importantly, it is suggested that the results will ultimately be generalizable to multiple emitters and, most importantly, to "nonblinking" molecular systems, which will be essential for broadening the scope of super-resolution measurements beyond the limits of fluorescence-based techniques.

Physically Unclonable Surfaces via Dewetting of Polymer Thin Films

From anti-counterfeiting to biotechnology applications, there is a strong demand for encoded surfaces with multiple security layers that are prepared by stochastic processes and are adaptable to deterministic fabrication approaches. Here, we present dewetting instabilities in nanoscopic (thickness <100 nm) polymer films as a form of physically unclonable function (PUF). The inherent randomness involved in the dewetting process presents a highly suitable platform for fabricating unclonable surfaces. The thermal annealing-induced dewetting of poly(2-vinyl pyridine) (P2VP) on polystyrene-grafted substrates enables fabrication of randomly positioned functional features that are separated at a microscopic length scale, a requirement set by optical authentication systems. At a first level, PUFs can be simply and readily verified via reflection of visible light. Area-specific electrostatic interactions between P2VP and citrate-stabilized gold nanoparticles allow for fabrication of plasmonic PUFs. The strong surface-enhanced Raman scattering by plasmonic nanoparticles together with incorporation of taggants facilitates a molecular vibration-based security layer. The patterning of P2VP films presents opportunities for fabricating hybrid security labels, which can be resolved through both stochastic and deterministic pathways. The adaptability to a broad range of nanoscale materials, simplicity, versatility, compatibility with conventional fabrication approaches, and high levels of stability offer key opportunities in encoding applications.

Nanoscale Detection of Subcellular Nanoparticles by X-Ray Diffraction Imaging for Precise Quantitative Analysis of Whole Cancer Cells

Nanoparticles show great potential for drug delivery systems in cancer treatment and diagnosis, which mainly rely on the interaction between nanoparticles and living cells. However, there is still a lack of accurate and large field-of-view imaging techniques to reveal the aggregation and distribution behavior of nanoparticles in whole cancer cells without being destroyed. Here, we demonstrated quantitative imaging of unstained and intact mouse breast cancer cells (4T1) containing 50 nm gold nanoparticles (Au@citrate NPs) using an X-ray scanning coherent diffraction imaging (ptychography) technique in a large field-of-view. A two-dimensional spatial resolution of 17 nm was achieved on the 4T1 cell. We combine X-ray ptychography and equally sloped tomography (EST) to perform three-dimensional structural mapping, distribution, and aggregation behavior of Au@citrate NPs in cancer cells. By taking full advantage of the large field-of-view, high-resolution, and quantitative imaging technique, the single intracellular Au@citrate NPs are observed and the amount of Au@citrate NPs in aggregations can be accurately quantified. In addition, the morphological changes of lysosomes containing Au@citrate NPs can be observed in the high-contrast mass density images. This study provides an approach for exploring quantitative analysis and physiological delivery of nanomaterials in intact cancer cells at nanoscale resolution, which may greatly benefit the interdisciplinary research of material science, nanomedicine, and nanotoxicology.

Deviation from Equilibrium Thermodynamics of an Asphaltene Model Compound during Compression-Expansion Experiments at Fluid-Fluid Interfaces

Asphaltenes play a crucial role in crude oil behavior, and model compounds are often used to capture, mimic, and predict certain interfacial properties. In previous works, sorption of an asphaltene model compound (C5PeC11) was studied using surface pressure isotherms, where a deviation from the expected thermodynamic behavior of the interface during decane-water and air-water compression experiments was observed but not explained. In this work, the interfacial behavior of C5PeC11 was assessed at the decane-water and the air-water interfaces using a multiscale approach that includes: compression-expansion experiments on rectangular and radial Langmuir troughs, dynamic interfacial stress relaxation, and fluorescence microscopy imaging. Connections between molecular and microscopic phenomena strongly suggest that the nonthermodynamic response can be explained through a dynamic effect whose origin lies in the predominance of intermolecular forces in C5PeC11 molecules over the mechanical compression force applied. When aggregation begins at the air-water interface, stable structures are formed, and the nonthermodynamic phenomenon is not observed in subsequent compressions. However, at the decane-water interface, the initial aggregation is not consolidated due to the effect of the oil phase on the free energy of the interface allowing the high reproducibility of the dynamic effect in subsequent compression cycles. These results highlight the need to probe interfacial systems at various length scales to adequately separate equilibrium thermodynamics from dynamic responses.

Optimizing illumination for precise multi-parameter estimations in coherent diffractive imaging

Coherent diffractive imaging (CDI) is widely used to characterize structured samples from measurements of diffracting intensity patterns. We introduce a numerical framework to quantify the precision that can be achieved when estimating any given set of parameters characterizing the sample from measured data. The approach, based on the calculation of the Fisher information matrix, provides a clear benchmark to assess the performance of CDI methods. Moreover, by optimizing the Fisher information metric using deep learning optimization libraries, we demonstrate how to identify the optimal illumination scheme that minimizes the estimation error under specified experimental constraints. This work paves the way for an efficient characterization of structured samples at the sub-wavelength scale.

Single-molecule localization microscopy

Single-molecule localization microscopy (SMLM) describes a family of powerful imaging techniques that dramatically improve spatial resolution over standard, diffraction-limited microscopy techniques and can image biological structures at the molecular scale. In SMLM, individual fluorescent molecules are computationally localized from diffraction-limited image sequences and the localizations are used to generate a super-resolution image or a time course of super-resolution images, or to define molecular trajectories. In this Primer, we introduce the basic principles of SMLM techniques before describing the main experimental considerations when performing SMLM, including fluorescent labelling, sample preparation, hardware requirements and image acquisition in fixed and live cells. We then explain how low-resolution image sequences are computationally processed to reconstruct super-resolution images and/or extract quantitative information, and highlight a selection of biological discoveries enabled by SMLM and closely related methods. We discuss some of the main limitations and potential artefacts of SMLM, as well as ways to alleviate them. Finally, we present an outlook on advanced techniques and promising new developments in the fast-evolving field of SMLM. We hope that this Primer will be a useful reference for both newcomers and practitioners of SMLM.

Super-resolution optical classifier with high photon efficiency

We propose and demonstrate a photon-efficient optical classifier to overcome the Rayleigh limit in spatial resolution. It utilizes mode-selective sum-frequency generation and single-pixel photon detection to resolve closely spaced incoherent sources based on photon counting statistics. Super-resolving and photon efficient, this technique can find applications in microscopy, light detection and ranging, and astrophysics.

Probing Interfacial Structure and Dynamics of Model and Natural Asphaltenes at Fluid-Fluid Interfaces

Asphaltenes are largely responsible for crude oil interfacial behavior. Due to their complex molecular nature, studying connections between interfacial properties and molecular structure is challenging, and these connections remain unclear. Several groups have reported on the interfacial behavior of asphaltenes, but a unified picture of both interfacial dynamics and thermodynamics is still missing. We seek to establish connections between asphaltene interfacial morphology and interfacial dynamics by combining interfacial dilatational deformation with microscopic structural imaging analysis. Understanding the behavior of natural asphaltene samples is made difficult by the inherent molecular variability. Therefore, we have also studied the behavior of an asphaltene model compound to draw fundamental structure-property relationships. This work contains simultaneous interfacial deformation and microscopy in systems of natural and model asphaltenes at air-water and decane-water interfaces. How the dynamics of natural asphaltenes influences the morphological and thermodynamic state of the air-water and decane-water interfaces is discussed based on the deviations observed between isotropic and anisotropic deformations. Areas where model asphaltenes can help us to understand the behavior of natural asphaltenes are identified such as its high surface pressure activity and aggregation character. An aggregation mechanism for model and natural asphaltenes is proposed based on an observed relationship between microscopic and millimetric aggregates.

Noise-resolution uncertainty principle in classical and quantum systems

We show that the width of an arbitrary function and the width of the distribution of its values cannot be made arbitrarily small simultaneously. In the case of ergodic stochastic processes, an ensuing uncertainty relationship is then demonstrated for the product of correlation length and variance. A closely related uncertainty principle is also established for the average degree of fourth-order coherence and the spatial width of modes of bosonic quantum fields. However, it is shown that, in the case of stochastic and quantum observables, certain non-classical states with sub-Poissonian statistics, such as for example photon number squeezed states in quantum optics, can overcome the "classical" noise-resolution uncertainty limit. This uncertainty relationship, which is fundamentally different from the Heisenberg and related uncertainty principles, can define an upper limit for the information capacity of communication and imaging systems. It is expected to be useful in a variety of problems in classical and quantum optics and imaging.

Influence of the Local Scattering Environment on the Localization Precision of Single Particles

We study the fundamental limit on the localization precision for a subwavelength scatterer embedded in a strongly scattering environment, using the external degrees of freedom provided by wavefront shaping. For a weakly scattering target, the localization precision improves with the value of the local density of states at the target position. For a strongly scattering target, the localization precision depends on the dressed polarizability that includes the backaction of the environment. This numerical study provides new insights for the control of the information content of scattered light by wavefront shaping, with potential applications in sensing, imaging, and nanoscale engineering.

Quantum Limits to Incoherent Imaging are Achieved by Linear Interferometry

We solve the general problem of determining, through imaging, the three-dimensional positions of N weak incoherent pointlike emitters in an arbitrary spatial configuration. We show that a structured measurement strategy in which a passive linear interferometer feeds into an array of photodetectors is always optimal for this estimation problem, in the sense that it saturates the quantum Cramér-Rao bound. We provide a method for the explicit construction of the optimal interferometer. Further explicit results for the quantum Fisher information and the optimal interferometer design that attains it are obtained for the special case of one and two incoherent emitters in the paraxial regime. This work provides insights into the phenomenon of superresolution through incoherent imaging that has attracted much attention recently. Our results will find a wide range of applications over a broad spectrum of frequencies, from fluorescence microscopy to stellar interferometry.

SNR enhancement in brillouin microspectroscopy using spectrum reconstruction

Brillouin spectroscopy can suffer from low signal-to-noise ratios (SNRs). Such low SNRs can render common data analysis protocols unreliable, especially for SNRs below ∼10. In this work we exploit two denoising algorithms, namely maximum entropy reconstruction (MER) and wavelet analysis (WA), to improve the accuracy and precision in determination of Brillouin shifts and linewidth. Algorithm performance is quantified using Monte-Carlo simulations and benchmarked against the Cramér-Rao lower bound. Superior estimation results are demonstrated even at low SNRs (≥ 1). Denoising is furthermore applied to experimental Brillouin spectra of distilled water at room temperature, allowing the speed of sound in water to be extracted. Experimental and theoretical values were found to be consistent to within ±1% at unity SNR.

Intensity-Based Axial Localization at the Quantum Limit

We derive fundamental precision bounds for single-point axial localization. For Gaussian beams, this ultimate limit can be achieved with a single intensity scan, provided the camera is placed at one of two optimal transverse detection planes. Hence, for axial localization there is no need of more complicated detection schemes. The theory is verified with an experimental demonstration of axial resolution 3 orders of magnitude below the classical depth of focus.

DLBI: deep learning guided Bayesian inference for structure reconstruction of super-resolution fluorescence microscopy

Motivation

Super-resolution fluorescence microscopy with a resolution beyond the diffraction limit of light, has become an indispensable tool to directly visualize biological structures in living cells at a nanometer-scale resolution. Despite advances in high-density super-resolution fluorescent techniques, existing methods still have bottlenecks, including extremely long execution time, artificial thinning and thickening of structures, and lack of ability to capture latent structures.

Results

Here, we propose a novel deep learning guided Bayesian inference (DLBI) approach, for the time-series analysis of high-density fluorescent images. Our method combines the strength of deep learning and statistical inference, where deep learning captures the underlying distribution of the fluorophores that are consistent with the observed time-series fluorescent images by exploring local features and correlation along time-axis, and statistical inference further refines the ultrastructure extracted by deep learning and endues physical meaning to the final image. In particular, our method contains three main components. The first one is a simulator that takes a high-resolution image as the input, and simulates time-series low-resolution fluorescent images based on experimentally calibrated parameters, which provides supervised training data to the deep learning model. The second one is a multi-scale deep learning module to capture both spatial information in each input low-resolution image as well as temporal information among the time-series images. And the third one is a Bayesian inference module that takes the image from the deep learning module as the initial localization of fluorophores and removes artifacts by statistical inference. Comprehensive experimental results on both real and simulated datasets demonstrate that our method provides more accurate and realistic local patch and large-field reconstruction than the state-of-the-art method, the 3B analysis, while our method is more than two orders of magnitude faster.

Availability and implementation

The main program is available at https://github.com/lykaust15/DLBI.

Supplementary information

Supplementary data are available at Bioinformatics online.

SIMPLE: Structured illumination based point localization estimator with enhanced precision

We present a structured illumination microscopy based point localization estimator (SIMPLE) that achieves a 2-fold increase in single molecule localization precision compared to conventional centroid estimation methods. SIMPLE advances the recently introduced MINFLUX concept by using precisely phase-shifted sinusoidal wave patterns as nanometric rulers for simultaneous particle localization based on photon count variation over a 20 μm field of view. We validate SIMPLE in silico and experimentally on a TIRF-SIM setup using a digital micro-mirror device (DMD) as a spatial light modulator.

Reading out Fisher information from the zeros of the point spread function

We show that, for optical systems whose point spread functions exhibit isolated zeros, the information one can gain about the separation between two incoherent point light sources does not scale quadratically with the separation (which is the distinctive dependence causing Rayleigh's curse) but only linearly. Moreover, the dominant contribution to the separation information comes from regions in the vicinity of these zeros. We experimentally confirm this idea, demonstrating significant superresolution using natural or artificially created spectral doublets.

3D Tracking-Free Approach for Obtaining 3D Super-Resolution Information in Rotationally Symmetric Biostructures

Currently, it is highly desirable but still challenging to obtain high-resolution (<50 nm) three-dimensional (3D) super-resolution information on structures in fixed specimens as well as for dynamic processes in live cells. Here we introduce a simple approach, without using 3D super-resolution microscopy or real-time 3D particle tracking, to estimate 3D sub-diffraction-limited structural or dynamic information in rotationally symmetric biostructures. This is a postlocalization analysis that transforms 2D super-resolution images or 2D single-molecule localization distributions into their corresponding 3D spatial probability distributions on the basis of prior known structural knowledge. This analysis is ideal in cases where the ultrastructure of a cellular structure is known but the substructural localization of a particular (usually mobile) protein is not. The method has been successfully applied to achieve 3D structural and functional sub-diffraction-limited information for 25-300 nm subcellular organelles that meet the rotational symmetry requirement, such as nuclear pore complex, primary cilium, and microtubule. In this Article, we will provide comprehensive analyses of this method by using experimental data and computational simulations. Finally, open source code of the 2D to 3D transformation algorithm (MATLAB) and simulations (Python) have also been developed.

Resolution limit of image analysis algorithms

The resolution of an imaging system is a key property that, despite many advances in optical imaging methods, remains difficult to define and apply. Rayleigh’s and Abbe’s resolution criteria were developed for observations with the human eye. However, modern imaging data is typically acquired on highly sensitive cameras and often requires complex image processing algorithms to analyze. Currently, no approaches are available for evaluating the resolving capability of such image processing algorithms that are now central to the analysis of imaging data, particularly location-based imaging data. Using methods of spatial statistics, we develop a novel algorithmic resolution limit to evaluate the resolving capabilities of location-based image processing algorithms. We show how insufficient algorithmic resolution can impact the outcome of location-based image analysis and present an approach to account for algorithmic resolution in the analysis of spatial location patterns.

The resolution limitations when using the ubiquitous algorithms that process images obtained using modern techniques are not straightforward to define. Here, the authors examine the performance of localization algorithms and use spatial statistics to provide a metric for assessing the resolution limit of such algorithms.

Quenched Stochastic Optical Reconstruction Microscopy (qSTORM) with Graphene Oxide

Quenched Stochastic Optical Reconstruction Microscopy (qSTORM) was demonstrated with graphene oxide sheets, peptides and bacteria; a method of contrast enhancement with super-resolution fluorescence microscopy. Individual sheets of graphene oxide (GO) were imaged with a resolution of 16 nm using the quenching of fluorescence emission by GO via its large Resonant Energy Transfer (RET) efficiency. The method was then extended to image self-assembled peptide aggregates (resolution 19 nm) and live bacterial cells (resolution 55 nm, the capsular structure of E. coli from urinary tract infections) with extremely low backgrounds and high contrasts (between one and two orders of magnitude contrast factor improvements that depended on the thickness of the graphene oxide layer used). Graphene oxide films combined with STORM imaging thus provide an extremely convenient method to image samples with large backgrounds due to non-specifically bound fluorophores (either due to excess labelling or autofluorescent molecules), which is a common occurrence in studies of both biological cells and soft-condensed matter. The GO quenches the fluorescence across a thin layer at distances of less than 15 nm. Graphene oxide films coated with thin layers (≤15 nm) of polystyrene, polymethylmethacrylate and polylysine are shown to be effective in producing high contrast qSTORM images, providing a convenient modulation of sample/substrate interactions. The GO coatings can also provide an increased image resolution and a factor of 2.3 improvement was observed with the peptide fibres using a feature of interest metric,when there was a large non-specifically bound background.

Quantum Limited Superresolution of an Incoherent Source Pair in Three Dimensions

The error in estimating the separation of a pair of incoherent sources from radiation emitted by them and subsequently captured by an imager is fundamentally bounded below by the inverse of the corresponding quantum Fisher information (QFI) matrix. We calculate the QFI for estimating the full three-dimensional pair separation vector, extending previous work on pair separation in one and two dimensions. We also show that the pair-separation QFI is, in fact, identical to source localization QFI, which underscores the fundamental importance of photon-state localization in determining the ultimate estimation-theoretic bound for both problems. We also propose general coherent-projection bases that can attain the QFI in two special cases. We present simulations of an approximate experimental realization of such quantum limited pair superresolution using the Zernike basis, confirming the achievability of the QFI bounds.

Information-Theoretic Approach and Fundamental Limits of Resolving Two Closely Timed Neuronal Spikes in Mouse Brain Calcium Imaging

OBJECTIVE

Although optical imaging of neurons using fluorescent genetically encoded calcium sensors has enabled large-scale in vivo experiments, the sensors' slow dynamics often blur closely timed action potentials into indistinguishable transients. While several previous approaches have been proposed to estimate the timing of individual spikes, they have overlooked the important and practical problem of estimating interspike interval (ISI) for overlapping transients.

METHODS

We use statistical detection theory to find the minimum detectable ISI under different levels of signal-to-noise ratio (SNR), model complexity, and recording speed. We also derive the Cramer-Rao lower bounds (CRBs) for the problem of ISI estimation. We use Monte-Carlo simulations with biologically derived parameters to numerically obtain the minimum detectable ISI and evaluate the performance of our estimators. Furthermore, we apply our detector to distinguish overlapping transients from experimentally obtained calcium imaging data.

RESULTS

Experiments based on simulated and real data across different SNR levels and recording speeds show that our algorithms can accurately distinguish two fluorescence signals with ISI on the order of tens of milliseconds, shorter than the waveform's rise time. Our study shows that the statistically optimal ISI estimators closely approached the CRBs.

CONCLUSION

Our work suggests that full analysis using recording speed, sensor kinetics, SNR, and the sensor's stochastically distributed response to action potentials can accurately resolve ISIs much smaller than the fluorescence waveform's rise time in modern calcium imaging experiments.

SIGNIFICANCE

Such analysis aids not only in future spike detection methods, but also in future experimental design when choosing sensors of neuronal activity.

Quality of biological images, reconstructed using localization microscopy data

Motivation

Fluorescence localization microscopy is extensively used to study the details of spatial architecture of subcellular compartments. This modality relies on determination of spatial positions of fluorophores, labeling an extended biological structure, with precision exceeding the diffraction limit. Several established models describe influence of pixel size, signal-to-noise ratio and optical resolution on the localization precision. The labeling density has been also recognized as important factor affecting reconstruction fidelity of the imaged biological structure. However, quantitative data on combined influence of sampling and localization errors on the fidelity of reconstruction are scarce. It should be noted that processing localization microscopy data is similar to reconstruction of a continuous (extended) non-periodic signal from a non-uniform, noisy point samples. In two dimensions the problem may be formulated within the framework of matrix completion. However, no systematic approach has been adopted in microscopy, where images are typically rendered by representing localized molecules with Gaussian distributions (widths determined by localization precision).

Results

We analyze the process of two-dimensional reconstruction of extended biological structures as a function of the density of registered emitters, localization precision and the area occupied by the rendered localized molecule. We quantify overall reconstruction fidelity with different established image similarity measures. Furthermore, we analyze the recovered similarity measure in the frequency space for different reconstruction protocols. We compare the cut-off frequency to the limiting sampling frequency, as determined by labeling density.

Availability and implementation

The source code used in the simulations along with test images is available at https://github.com/blazi13/qbioimages.

Supplementary information

Supplementary data are available at Bioinformatics online.

Quantum-Limited Time-Frequency Estimation through Mode-Selective Photon Measurement

By projecting onto complex optical mode profiles, it is possible to estimate arbitrarily small separations between objects with quantum-limited precision, free of uncertainty arising from overlapping intensity profiles. Here we extend these techniques to the time-frequency domain using mode-selective sum-frequency generation with shaped ultrafast pulses. We experimentally resolve temporal and spectral separations between incoherent mixtures of single-photon level signals ten times smaller than their optical bandwidths with a tenfold improvement in precision over the intensity-only Cramér-Rao bound.

Fundamental Precision Bounds for Three-Dimensional Optical Localization Microscopy with Poisson Statistics

Point source localization is a problem of persistent interest in optical imaging. In particular, a number of widely used biological microscopy techniques rely on precise three-dimensional localization of single fluorophores. As emitter depth localization is more challenging than lateral localization, considerable effort has been spent on engineering the response of the microscope in a way that reveals increased depth information. Here, we prove the (sub)optimality of these approaches by deriving and comparing to the measurement-independent quantum Cramér-Rao bound (QCRB). We show that existing methods for depth localization with single-objective collection exceed the QCRB, and we gain insight into the bound by proposing an interferometer arrangement that approaches it. We also show that for light collection with two opposed objectives, an established interferometric technique globally reaches the QCRB in all three dimensions simultaneously, and so this represents an interesting case study from the point of view of quantum multiparameter estimation.

In vivo imaging and analysis of cerebrovascular hemodynamic responses and tissue oxygenation in the mouse brain

Cerebrovascular dysfunction has an important role in the pathogenesis of multiple brain disorders. Measurement of hemodynamic responses in vivo can be challenging, particularly as techniques are often not described in sufficient detail and vary between laboratories. We present a set of standardized in vivo protocols that describe high-resolution two-photon microscopy and intrinsic optical signal (IOS) imaging to evaluate capillary and arteriolar responses to a stimulus, regional hemodynamic responses, and oxygen delivery to the brain. The protocol also describes how to measure intrinsic NADH fluorescence to understand how blood O2 supply meets the metabolic demands of activated brain tissue, and to perform resting-state absolute oxygen partial pressure (pO2) measurements of brain tissue. These methods can detect cerebrovascular changes at far higher resolution than MRI techniques, although the optical nature of these techniques limits their achievable imaging depths. Each individual procedure requires 1-2 h to complete, with two to three procedures typically performed per animal at a time. These protocols are broadly applicable in studies of cerebrovascular function in healthy and diseased brain in any of the existing mouse models of neurological and vascular disorders. All these procedures can be accomplished by a competent graduate student or experienced technician, except the two-photon measurement of absolute pO2 level, which is better suited to a more experienced, postdoctoral-level researcher.

Deep learning massively accelerates super-resolution localization microscopy

The speed of super-resolution microscopy methods based on single-molecule localization, for example, PALM and STORM, is limited by the need to record many thousands of frames with a small number of observed molecules in each. Here, we present ANNA-PALM, a computational strategy that uses artificial neural networks to reconstruct super-resolution views from sparse, rapidly acquired localization images and/or widefield images. Simulations and experimental imaging of microtubules, nuclear pores, and mitochondria show that high-quality, super-resolution images can be reconstructed from up to two orders of magnitude fewer frames than usually needed, without compromising spatial resolution. Super-resolution reconstructions are even possible from widefield images alone, though adding localization data improves image quality. We demonstrate super-resolution imaging of >1,000 fields of view containing >1,000 cells in ∼3 h, yielding an image spanning spatial scales from ∼20 nm to ∼2 mm. The drastic reduction in acquisition time and sample irradiation afforded by ANNA-PALM enables faster and gentler high-throughput and live-cell super-resolution imaging.

Three-Dimensional Localization of Single Molecules for Super-Resolution Imaging and Single-Particle Tracking

Single-molecule super-resolution fluorescence microscopy and single-particle tracking are two imaging modalities that illuminate the properties of cells and materials on spatial scales down to tens of nanometers or with dynamical information about nanoscale particle motion in the millisecond range, respectively. These methods generally use wide-field microscopes and two-dimensional camera detectors to localize molecules to much higher precision than the diffraction limit. Given the limited total photons available from each single-molecule label, both modalities require careful mathematical analysis and image processing. Much more information can be obtained about the system under study by extending to three-dimensional (3D) single-molecule localization: without this capability, visualization of structures or motions extending in the axial direction can easily be missed or confused, compromising scientific understanding. A variety of methods for obtaining both 3D super-resolution images and 3D tracking information have been devised, each with their own strengths and weaknesses. These include imaging of multiple focal planes, point-spread-function engineering, and interferometric detection. These methods may be compared based on their ability to provide accurate and precise position information on single-molecule emitters with limited photons. To successfully apply and further develop these methods, it is essential to consider many practical concerns, including the effects of optical aberrations, field dependence in the imaging system, fluorophore labeling density, and registration between different color channels. Selected examples of 3D super-resolution imaging and tracking are described for illustration from a variety of biological contexts and with a variety of methods, demonstrating the power of 3D localization for understanding complex systems.

Clustered localization of EGFRvIII in glioblastoma cells as detected by high precision localization microscopy

For receptor tyrosine kinases supramolecular organization on the cell membrane is critical for their function. Super-resolution fluorescence microscopy techniques have offered new opportunities for the analysis of single receptor localization. Here, we analysed the cluster formation of the epidermal growth factor receptor variant III (EGFRvIII), a deletion variant which is expressed in glioblastoma. The constitutively activated variant EGFRvIII is expressed in cells with an egfr gene amplification and is thought to enhance the tumorigenic potential especially of glioblastoma cells. Due to the lack of an adequate model system, it is still unclear how endogenous EGFRvIII expression alters cellular signalling and if it is organized in clusters like the wild type receptor. We have recently described the establishment of two pairs of iso-genetic cell lines (BS153 and DKMG), displaying endogenous EGFRvIII expression or not. Using these cell lines we investigated single receptor localization of EGFRvIII by high precision localization microscopy. Cluster analysis revealed that EGFRvIII is present in clusters on the surface of the cells, with about 60% or even more receptor molecules being assembled in clusters of approximately 100 nm in diameter whereby the cluster definition was iteratively determined. The signal to signal distance may indicate dimer formation while signal quantification indicates 1 × 10⁶-5 × 10⁶ EGFRvIII molecules per cell. Altogether, these data give unique insights into the membrane surface localization of EGFRvIII in glioblastoma cells. These insights will help to unveil the function of this tumour associated receptor variant which might lead to a better understanding of glioblastoma and therefore could lead to improved therapy approaches.

Far-Field Superresolution of Thermal Electromagnetic Sources at the Quantum Limit

We obtain the ultimate quantum limit for estimating the transverse separation of two thermal point sources using a given imaging system with limited spatial bandwidth. We show via the quantum Cramér-Rao bound that, contrary to the Rayleigh limit in conventional direct imaging, quantum mechanics does not mandate any loss of precision in estimating even deep sub-Rayleigh separations. We propose two coherent measurement techniques, easily implementable using current linear-optics technology, that approach the quantum limit over an arbitrarily large range of separations. Our bound is valid for arbitrary source strengths, all regions of the electromagnetic spectrum, and for any imaging system with an inversion-symmetric point-spread function. The measurement schemes can be applied to microscopy, optical sensing, and astrometry at all wavelengths.

Fisher information theory for parameter estimation in single molecule microscopy: tutorial

Estimation of a parameter of interest from image data represents a task that is commonly carried out in single molecule microscopy data analysis. The determination of the positional coordinates of a molecule from its image, for example, forms the basis of standard applications such as single molecule tracking and localization-based super-resolution image reconstruction. Assuming that the estimator used recovers, on average, the true value of the parameter, its accuracy, or standard deviation, is then at best equal to the square root of the Cramér-Rao lower bound. The Cramér-Rao lower bound can therefore be used as a benchmark in the evaluation of the accuracy of an estimator. Additionally, as its value can be computed and assessed for different experimental settings, it is useful as an experimental design tool. This tutorial demonstrates a mathematical framework that has been specifically developed to calculate the Cramér-Rao lower bound for estimation problems in single molecule microscopy and, more broadly, fluorescence microscopy. The material includes a presentation of the photon detection process that underlies all image data, various image data models that describe images acquired with different detector types, and Fisher information expressions that are necessary for the calculation of the lower bound. Throughout the tutorial, examples involving concrete estimation problems are used to illustrate the effects of various factors on the accuracy of parameter estimation and, more generally, to demonstrate the flexibility of the mathematical framework.

Interferometric superlocalization of two incoherent optical point sources

A novel interferometric method - SLIVER (Super Localization by Image inVERsion interferometry) - is proposed for estimating the separation of two incoherent point sources with a mean squared error that does not deteriorate as the sources are brought closer. The essential component of the interferometer is an image inversion device that inverts the field in the transverse plane about the optical axis, assumed to pass through the centroid of the sources. The performance of the device is analyzed using the Cramér-Rao bound applied to the statistics of spatially-unresolved photon counting using photon number-resolving and on-off detectors. The analysis is supported by Monte-Carlo simulations of the maximum likelihood estimator for the source separation, demonstrating the superlocalization effect for separations well below that set by the Rayleigh criterion. Simulations indicating the robustness of SLIVER to mismatch between the optical axis and the centroid are also presented. The results are valid for any imaging system with a circularly symmetric point-spread function.

Comparison between optical-resolution photoacoustic microscopy and confocal laser scanning microscopy for turbid sample imaging

Optical-resolution photoacoustic microscopy (ORPAM) in theory provides lateral resolution equivalent to the optical diffraction limit. Scattering media, such as biological turbid media, attenuates the optical signal and also alters the diffraction-limited spot size of the focused beam. The ORPAM signal is generated only from a small voxel in scattering media with dimensions equivalent to the laser spot size after passing through scattering layers and is detected by an acoustic transducer, which is not affected by optical scattering. Thus, both ORPAM and confocal laser scanning microscopy (CLSM) reject scattered light. A multimodal optical microscopy platform that includes ORPAM and CLSM was constructed, and the lateral resolution of both modes was measured using patterned thin metal film with and without a scattering barrier. The effect of scattering media on the lateral resolution was studied using different scattering coefficients and was compared to computational results based on Monte Carlo simulations. It was found that degradation of lateral resolution due to optical scattering was not significant for either ORPAM or CLSM. The depth discrimination capability of ORPAM and CLSM was measured using microfiber embedded in a light scattering phantom material. ORPAM images demonstrated higher contrast compared to CLSM images partly due to reduced acoustic signal scattering.