Algorithmic corrections for localization microscopy with sCMOS cameras - characterisation of a computationally efficient localization approach

An integrated platform for high-throughput nanoscopy

Single-molecule localization microscopy enables three-dimensional fluorescence imaging at tens-of-nanometer resolution, but requires many camera frames to reconstruct a super-resolved image. This limits the typical throughput to tens of cells per day. While frame rates can now be increased by over an order of magnitude, the large data volumes become limiting in existing workflows. Here we present an integrated acquisition and analysis platform leveraging microscopy-specific data compression, distributed storage and distributed analysis to enable an acquisition and analysis throughput of 10,000 cells per day. The platform facilitates graphically reconfigurable analyses to be automatically initiated from the microscope during acquisition and remotely executed, and can even feed back and queue new acquisition tasks on the microscope. We demonstrate the utility of this framework by imaging hundreds of cells per well in multi-well sample formats. Our platform, implemented within the PYthon-Microscopy Environment (PYME), is easily configurable to control custom microscopes, and includes a plugin framework for user-defined extensions.

Photon-free (s)CMOS camera characterization for artifact reduction in high- and super-resolution microscopy

Modern implementations of widefield fluorescence microscopy often rely on sCMOS cameras, but this camera architecture inherently features pixel-to-pixel variations. Such variations lead to image artifacts and render quantitative image interpretation difficult. Although a variety of algorithmic corrections exists, they require a thorough characterization of the camera, which typically is not easy to access or perform. Here, we developed a fully automated pipeline for camera characterization based solely on thermally generated signal, and implemented it in the popular open-source software Micro-Manager and ImageJ/Fiji. Besides supplying the conventional camera maps of noise, offset and gain, our pipeline also gives access to dark current and thermal noise as functions of the exposure time. This allowed us to avoid structural bias in single-molecule localization microscopy (SMLM), which without correction is substantial even for scientific-grade, cooled cameras. In addition, our approach enables high-quality 3D super-resolution as well as live-cell time-lapse microscopy with cheap, industry-grade cameras. As our approach for camera characterization does not require any user interventions or additional hardware implementations, numerous correction algorithms that rely on camera characterization become easily applicable.

Fluorogenic DNA-PAINT for faster, low-background super-resolution imaging

DNA-based points accumulation for imaging in nanoscale topography (DNA-PAINT) is a powerful super-resolution microscopy method that can acquire high-fidelity images at nanometer resolution. It suffers, however, from high background and slow imaging speed, both of which can be attributed to the presence of unbound fluorophores in solution. Here we present two-color fluorogenic DNA-PAINT, which uses improved imager probe and docking strand designs to solve these problems. These self-quenching single-stranded DNA probes are conjugated with a fluorophore and quencher at the terminals, which permits an increase in fluorescence by up to 57-fold upon binding and unquenching. In addition, the engineering of base pair mismatches between the fluorogenic imager probes and docking strands allowed us to achieve both high fluorogenicity and the fast binding kinetics required for fast imaging. We demonstrate a 26-fold increase in imaging speed over regular DNA-PAINT and show that our new implementation enables three-dimensional super-resolution DNA-PAINT imaging without optical sectioning.

sCMOS Noise-Corrected Superresolution Reconstruction Algorithm for Structured Illumination Microscopy

Structured illumination microscopy (SIM) is widely applied due to its high temporal and spatial resolution imaging ability. sCMOS cameras are often used in SIM due to their superior sensitivity, resolution, field of view, and frame rates. However, the unique single-pixel-dependent readout noise of sCMOS cameras may lead to SIM reconstruction artefacts and affect the accuracy of subsequent statistical analysis. We first established a nonuniform sCMOS noise model to address this issue, which incorporates the single-pixel-dependent offset, gain, and variance based on the SIM imaging process. The simulation indicates that the sCMOS pixel-dependent readout noise causes artefacts in the reconstructed SIM superresolution (SR) image. Thus, we propose a novel sCMOS noise-corrected SIM reconstruction algorithm derived from the imaging model, which can effectively suppress the sCMOS noise-related reconstruction artefacts and improve the signal-to-noise ratio (SNR).

A gain series method for accurate EMCCD calibration

Calibration of the gain and digital conversion factor of an EMCCD is necessary for accurate photon counting. We present a new method to quickly calibrate multiple gain settings of an EMCCD camera. Acquiring gain-series calibration data and analyzing the resulting images with the EMCCD noise model more accurately estimates the gain response of the camera. Furthermore, we develop a method to compare the results from different calibration approaches. Gain-series calibration outperforms all other methods in this self-consistency test.

Expectation maximization based framework for joint localization and parameter estimation in single particle tracking from segmented images

Single Particle Tracking (SPT) is a well known class of tools for studying the dynamics of biological macromolecules moving inside living cells. In this paper, we focus on the problem of localization and parameter estimation given a sequence of segmented images. In the standard paradigm, the location of the emitter inside each frame of a sequence of camera images is estimated using, for example, Gaussian fitting (GF), and these locations are linked to provide an estimate of the trajectory. Trajectories are then analyzed by using Mean Square Displacement (MSD) or Maximum Likelihood Estimation (MLE) techniques to determine motion parameters such as diffusion coefficients. However, the problems of localization and parameter estimation are clearly coupled. Motivated by this, we have created an Expectation Maximization (EM) based framework for simultaneous localization and parameter estimation. We demonstrate this framework through two representative methods, namely, Sequential Monte Carlo combined with Expectation Maximization (SMC-EM) and Unscented Kalman Filter combined with Expectation Maximization (U-EM). Using diffusion in two-dimensions as a prototypical example, we conduct quantitative investigations on localization and parameter estimation performance across a wide range of signal to background ratios and diffusion coefficients and compare our methods to the standard techniques based on GF-MSD/MLE. To demonstrate the flexibility of the EM based framework, we do comparisons using two different camera models, an ideal camera with Poisson distributed shot noise but no readout noise, and a camera with both shot noise and the pixel-dependent readout noise that is common to scientific complementary metal-oxide semiconductor (sCMOS) camera. Our results indicate our EM based methods outperform the standard techniques, especially at low signal levels. While U-EM and SMC-EM have similar accuracy, U-EM is significantly more computationally efficient, though the use of the Unscented Kalman Filter limits U-EM to lower diffusion rates.

Characterizing and correcting camera noise in back-illuminated sCMOS cameras

With promising properties of fast imaging speed, large field-of-view, relative low cost and many others, back-illuminated sCMOS cameras have been receiving intensive attention for low light level imaging in the past several years. However, due to the pixel-to-pixel difference of camera noise (called noise non-uniformity) in sCMOS cameras, researchers may hesitate to use them in some application fields, and sometimes wonder whether they should optimize the noise non-uniformity of their sCMOS cameras before using them in a specific application scenario. In this paper, we systematically characterize the impact of different types of sCMOS noise on image quality and perform corrections to these types of sCMOS noise using three representative algorithms (PURE, NCS and MLE_sCMOS). We verify that it is possible to use appropriate correction methods to push the non-uniformity of major types of camera noise, including readout noise, offset, and photon response, to a satisfactory level for conventional microscopy and single molecule localization microscopy. We further find out that, after these corrections, global read noise becomes a major concern that limits the imaging performance of back-illuminated sCMOS cameras. We believe this study provides new insights into the understanding of camera noise in back-illuminated sCMOS cameras, and also provides useful information for future development of this promising camera technology.

Repeat DNA-PAINT suppresses background and non-specific signals in optical nanoscopy

DNA-PAINT is a versatile optical super-resolution technique relying on the transient binding of fluorescent DNA ‘imagers’ to target epitopes. Its performance in biological samples is often constrained by strong background signals and non-specific binding events, both exacerbated by high imager concentrations. Here we describe Repeat DNA-PAINT, a method that enables a substantial reduction in imager concentration, thus suppressing spurious signals. Additionally, Repeat DNA-PAINT reduces photoinduced target-site loss and can accelerate sampling, all without affecting spatial resolution.

DNA-PAINT is a super-resolution imaging technique which suffers from high background signals and non-specific binding. Here the authors report Repeat DNA-PAINT which is capable of supressing background noise and preventing photoinduced site loss, as well as decreasing the time taken for the sampling process.

Sample Preparation and Imaging Conditions Affect mEos3.2 Photophysics in Fission Yeast Cells

Photoconvertible fluorescent proteins (PCFPs) are widely used in super-resolution microscopy and studies of cellular dynamics. However, our understanding of their photophysics is still limited, hampering their quantitative application. For example, we do not know the optimal sample preparation methods or imaging conditions to count protein molecules fused to PCFPs by single-molecule localization microscopy in live and fixed cells. We also do not know how the behavior of PCFPs in live cells compares with fixed cells. Therefore, we investigated how formaldehyde fixation influences the photophysical properties of the popular green-to-red PCFP mEos3.2 in fission yeast cells under a wide range of imaging conditions. We estimated photophysical parameters by fitting a three-state model of photoconversion and photobleaching to the time course of fluorescence signal per yeast cell expressing mEos3.2. We discovered that formaldehyde fixation makes the fluorescence signal, photoconversion rate, and photobleaching rate of mEos3.2 sensitive to the buffer conditions likely by permeabilizing the yeast cell membrane. Under some imaging conditions, the time-integrated mEos3.2 signal per yeast cell is similar in live cells and fixed cells imaged in buffer at pH 8.5 with 1 mM DTT, indicating that light chemical fixation does not destroy mEos3.2 molecules. We also discovered that 405-nm irradiation drove some red-state mEos3.2 molecules to enter an intermediate dark state, which can be converted back to the red fluorescent state by 561-nm illumination. Our findings provide a guide to quantitatively compare conditions for imaging mEos3.2-tagged molecules in yeast cells. Our imaging assay and mathematical model are easy to implement and provide a simple quantitative approach to measure the time-integrated signal and the photoconversion and photobleaching rates of fluorescent proteins in cells.

A time-varying approach to single particle tracking with a nonlinear observation model

Single Particle Tracking (SPT) is a powerful class of tools for analyzing the dynamics of individual biological macromolecules moving inside living cells. The acquired data is typically in the form of a sequence of camera images that are then post-processed to reveal details about the motion. In this work, we develop a local time-varying estimation algorithm for estimating motion model parameters from the data considering nonlinear observations. Our approach uses several well-known existing tools, namely the Expectation Maximization (EM) algorithm combined with an Unscented Kalman filter (UKF) and an Unscented Rauch-Tung-Striebel smoother (URTSS), and applies them to the time-varying case through a sliding window methodology. Due to the shot noise characteristics of the photon generation process, this model uses a Poisson distribution to capture the measurement noise inherent in imaging. In order to apply our time-varying approach to the UKF, we first need to transform the measurements into a model with additive Gaussian noise. This is carried out using a variance stabilizing transform. Results from simulations show that our approach is successful in tracing time-varying diffusion constants at a range of physically relevant signal levels. We also discuss the initialization for the EM algorithm based on the available data.

Simultaneous Localization and Parameter Estimation for Single Particle Tracking via Sigma Points based EM

Single Particle Tracking (SPT) is a powerful class of tools for analyzing the dynamics of individual biological macromolecules moving inside living cells. The acquired data is typically in the form of a sequence of camera images that are then post-processed to reveal details about the motion. In this work, we develop an algorithm for jointly estimating both particle trajectory and motion model parameters from the data. Our approach uses Expectation Maximization (EM) combined with an Unscented Kalman filter (UKF) and an Unscented Rauch-Tung-Striebel smoother (URTSS), allowing us to use an accurate, nonlinear model of the observations acquired by the camera. Due to the shot noise characteristics of the photon generation process, this model uses a Poisson distribution to capture the measurement noise inherent in imaging. In order to apply a UKF, we first must transform the measurements into a model with additive Gaussian noise. We consider two approaches, one based on variance stabilizing transformations (where we compare the Anscombe and Freeman-Tukey transforms) and one on a Gaussian approximation to the Poisson distribution. Through simulations, we demonstrate efficacy of the approach and explore the differences among these measurement transformations.

Correcting Artifacts in Single Molecule Localization Microscopy Analysis Arising from Pixel Quantum Efficiency Differences in sCMOS Cameras

Optimal analysis of single molecule localization microscopy (SMLM) data acquired with a scientific Complementary Metal-Oxide-Semiconductor (sCMOS) camera relies on statistical compensation for its pixel-dependent gain, offset and readout noise. In this work we show that it is also necessary to compensate for differences in the relative quantum efficiency (RQE) of each pixel. We found differences in RQE on the order of 4% in our tested sCMOS sensors. These differences were large enough to have a noticeable effect on analysis algorithm results, as seen both in simulations and biological imaging data. We discuss how the RQE differences manifest themselves in the analysis results and present the modifications to the Poisson maximum likelihood estimation (MLE) sCMOS analysis algorithm that are needed to correct for the RQE differences.

3D super-resolution microscopy performance and quantitative analysis assessment using DNA-PAINT and DNA origami test samples

Assessment of the imaging quality in localisation-based super-resolution techniques relies on an accurate characterisation of the imaging setup and analysis procedures. Test samples can provide regular feedback on system performance and facilitate the implementation of new methods. While multiple test samples for regular, 2D imaging are available, they are not common for more specialised imaging modes. Here, we analyse robust test samples for 3D and quantitative super-resolution imaging, which are straightforward to use, are time- and cost-effective and do not require experience beyond basic laboratory and imaging skills. We present two options for assessment of 3D imaging quality, the use of microspheres functionalised for DNA-PAINT and a commercial DNA origami sample. A method to establish and assess a qPAINT workflow for quantitative imaging is demonstrated with a second, commercially available DNA origami sample.

Subnanometer localization accuracy in widefield optical microscopy

The common assumption that precision is the limit of accuracy in localization microscopy and the typical absence of comprehensive calibration of optical microscopes lead to a widespread issue-overconfidence in measurement results with nanoscale statistical uncertainties that can be invalid due to microscale systematic errors. In this article, we report a comprehensive solution to this underappreciated problem. We develop arrays of subresolution apertures into the first reference materials that enable localization errors approaching the atomic scale across a submillimeter field. We present novel methods for calibrating our microscope system using aperture arrays and develop aberration corrections that reach the precision limit of our reference materials. We correct and register localization data from multiple colors and test different sources of light emission with equal accuracy, indicating the general applicability of our reference materials and calibration methods. In a first application of our new measurement capability, we introduce the concept of critical-dimension localization microscopy, facilitating tests of nanofabrication processes and quality control of aperture arrays. In a second application, we apply these stable reference materials to answer open questions about the apparent instability of fluorescent nanoparticles that commonly serve as fiducial markers. Our study establishes a foundation for subnanometer localization accuracy in widefield optical microscopy.

Using DNA origami nanorulers as traceable distance measurement standards and nanoscopic benchmark structures

In recent years, DNA origami nanorulers for superresolution (SR) fluorescence microscopy have been developed from fundamental proof-of-principle experiments to commercially available test structures. The self-assembled nanostructures allow placing a defined number of fluorescent dye molecules in defined geometries in the nanometer range. Besides the unprecedented control over matter on the nanoscale, robust DNA origami nanorulers are reproducibly obtained in high yields. The distances between their fluorescent marks can be easily analysed yielding intermark distance histograms from many identical structures. Thus, DNA origami nanorulers have become excellent reference and training structures for superresolution microscopy. In this work, we go one step further and develop a calibration process for the measured distances between the fluorescent marks on DNA origami nanorulers. The superresolution technique DNA-PAINT is used to achieve nanometrological traceability of nanoruler distances following the guide to the expression of uncertainty in measurement (GUM). We further show two examples how these nanorulers are used to evaluate the performance of TIRF microscopes that are capable of single-molecule localization microscopy (SMLM).

True Molecular Scale Visualization of Variable Clustering Properties of Ryanodine Receptors

Signaling nanodomains rely on spatial organization of proteins to allow controlled intracellular signaling. Examples include calcium release sites of cardiomyocytes where ryanodine receptors (RyRs) are clustered with their molecular partners. Localization microscopy has been crucial to visualizing these nanodomains but has been limited by brightness of markers, restricting the resolution and quantification of individual proteins clustered within. Harnessing the remarkable localization precision of DNA-PAINT (<10 nm), we visualized punctate labeling within these nanodomains, confirmed as single RyRs. RyR positions within sub-plasmalemmal nanodomains revealed how they are organized randomly into irregular clustering patterns leaving significant gaps occupied by accessory or regulatory proteins. RyR-inhibiting protein junctophilin-2 appeared highly concentrated adjacent to RyR channels. Analyzing these molecular maps showed significant variations in the co-clustering stoichiometry between junctophilin-2 and RyR, even between nearby nanodomains. This constitutes an additional level of complexity in RyR arrangement and regulation of calcium signaling, intrinsically built into the nanodomains.

Characterization of an industry-grade CMOS camera well suited for single molecule localization microscopy - high performance super-resolution at low cost

Many commercial as well as custom-built fluorescence microscopes use scientific-grade cameras that represent a substantial share of the instrument's cost. This holds particularly true for super-resolution localization microscopy where high demands are placed especially on the detector with respect to sensitivity, noise, and also image acquisition speed. Here, we present and carefully characterize an industry-grade CMOS camera as a cost-efficient alternative to commonly used scientific cameras. Direct experimental comparison of these two detector types shows widely similar performance for imaging by single molecule localization microscopy (SMLM). Furthermore, high image acquisition speeds are demonstrated for the CMOS detector by ultra-fast SMLM imaging.