Fast and accurate sCMOS noise correction for fluorescence microscopy

Single-molecule tracking technologies for quantifying the dynamics of gene regulation in cells, tissue and embryos

For decades, we have relied on population and time-averaged snapshots of dynamic molecular scale events to understand how genes are regulated during development and beyond. The advent of techniques to observe single-molecule kinetics in increasingly endogenous contexts, progressing from in vitro studies to living embryos, has revealed how much we have missed. Here, we provide an accessible overview of the rapidly expanding family of technologies for single-molecule tracking (SMT), with the goal of enabling the reader to critically analyse single-molecule studies, as well as to inspire the application of SMT to their own work. We start by overviewing the basics of and motivation for SMT experiments, and the trade-offs involved when optimizing parameters. We then cover key technologies, including fluorescent labelling, excitation and detection optics, localization and tracking algorithms, and data analysis. Finally, we provide a summary of selected recent applications of SMT to study the dynamics of gene regulation.

Deriving high contrast fluorescence microscopy images through low contrast noisy image stacks

Contrast in fluorescence microscopy images allows for the differentiation between different structures by their difference in intensities. However, factors such as point-spread function and noise may reduce it, affecting its interpretability. We identified that fluctuation of emitters in a stack of images can be exploited to achieve increased contrast when compared to the average and Richardson-Lucy deconvolution. We tested our methods on four increasingly challenging samples including tissue, in which case results were comparable to the ones obtained by structured illumination microscopy in terms of contrast.

Spectroscopic Approach to Correction and Visualisation of Bright-Field Light Transmission Microscopy Biological Data

The most realistic information about a transparent sample such as a live cell can be obtained using bright-field light microscopy. Under high-intensity pulsing LED illumination, we captured a primary 12-bit-per-channel (bpc) response from an observed sample using a bright-field microscope equipped with a high-resolution (4872 × 3248) image sensor. In order to suppress data distortions originating from the light interactions with elements in the optical path, poor sensor reproduction (geometrical defects of the camera sensor and some peculiarities of sensor sensitivity), we propose a spectroscopic approach for the correction of these uncompressed 12 bpc data by simultaneous calibration of all parts of the experimental arrangement. Moreover, the final intensities of the corrected images are proportional to the photon fluxes detected by a camera sensor. It can be visualized in 8 bpc intensity depth after the Least Information Loss compression.

Blind Image Restoration Enhances Digital Autoradiographic Imaging of Radiopharmaceutical Tissue Distribution

Digital autoradiography (DAR) is a powerful tool to quantitatively determine the distribution of a radiopharmaceutical within a tissue section and is widely used in drug discovery and development. However, the low image resolution and significant background noise can result in poor correlation, even errors, between radiotracer distribution, anatomical structure, and molecular expression profiles. Differing from conventional optical systems, the point spread function (PSF) in DAR is determined by properties of radioisotope decay, phosphor and digitizer. Calibration of an experimental PSF a priori is difficult, prone to error, and impractical. We have developed a content-adaptive restoration algorithm to address these problems. Methods: We model the DAR imaging process using a mixed Poisson-Gaussian model, and blindly restore the image by a Penalized Maximum-Likelihood Expectation-Maximization algorithm (PG- PEM). PG-PEM implements a patch-based estimation algorithm with "Density-Based Spatial Clus- tering of Applications with Noise" to estimate noise parameters, and utilizes L2 and Hessian Frobenius (HF) norms as regularization functions to improve performance. Results: First, PG-PEM outperformed other restoration algorithms at the denoising task (p<0.01). Next, we implemented PG-PEM on pre-clinical DAR images (¹⁸F-FDG treated mice tumor and heart, ¹⁸F-NaF treated mice femur) and clinical DAR images (bone biopsy sections from ²²³RaCl2 treated castrate resistant prostate cancer patients). DAR images restored by PG-PEM of all samples achieved significantly higher effective resolution, contrast to noise ratio (CNR), and a lower standard deviation of background (STDB) (p<0.0001). Additionally, by comparing the registration results between the clinical DAR images and the segmented bone masks from the corresponding histological images, the radiopharmaceutical distribution was significantly improved (p<0.0001). Conclusion: PG-PEM is able to increase resolution and contrast while robustly accounting for DAR noise, and demonstrates the capacity to be widely implemented to improve pre- and clinical DAR imaging of radiopharmaceutical distribution.

A generic intelligent tomato classification system for practical applications using DenseNet-201 with transfer learning

A generic intelligent tomato classification system based on DenseNet-201 with transfer learning was proposed and the augmented training sets obtained by data augmentation methods were employed to train the model. The trained model achieved high classification accuracy on the images of different quality, even those containing high levels of noise. Also, the trained model could accurately and efficiently identify and classify a single tomato image with only 29 ms, indicating that the proposed model has great potential value in real-world applications. The feature visualization of the trained models shows their understanding of tomato images, i.e., the learned common and high-level features. The strongest activations of the trained models show that the correct or incorrect target recognition areas by a model during the classification process will affect its final classification accuracy. Based on this, the results obtained in this study could provide guidance and new ideas to improve the development of intelligent agriculture.

Signal and noise analysis for chiral structured illumination microscopy

Recently, chiral structured illumination microscopy has been proposed to image fluorescent chiral domains at sub-wavelength resolution. Chiral structured illumination microscopy is based on the combination of structured illumination microscopy, fluorescence-detected circular dichroism, and optical chirality engineering. Since circular dichroism of natural chiral molecules is typically weak, the differential fluorescence is also weak and can be easily buried by the noise, hampering the fidelity of the reconstructed images. In this work, we systematically study the impact of the noise on the quality and resolution of chiral domain images obtained by chiral SIM. We analytically describe the signal-to-noise ratio of the reconstructed chiral SIM image in the Fourier domain and verify our theoretical calculations with numerical demonstrations. Accordingly, we discuss the feasibility of chiral SIM in different experimental scenarios and propose possible strategies to enhance the signal-to-noise ratio for samples with weak circular dichroism.

Removing striping artifacts in light-sheet fluorescence microscopy: a review

In recent years, light-sheet fluorescence microscopy (LSFM) has found a broad application for imaging of diverse biological samples, ranging from sub-cellular structures to whole animals, both in-vivo and ex-vivo, owing to its many advantages relative to point-scanning methods. By providing the selective illumination of sample single planes, LSFM achieves an intrinsic optical sectioning and direct 2D image acquisition, with low out-of-focus fluorescence background, sample photo-damage and photo-bleaching. On the other hand, such an illumination scheme is prone to light absorption or scattering effects, which lead to uneven illumination and striping artifacts in the images, oriented along the light sheet propagation direction. Several methods have been developed to address this issue, ranging from fully optical solutions to entirely digital post-processing approaches. In this work, we present them, outlining their advantages, performance and limitations.

It's clearly the heart! Optical transparency, cardiac tissue imaging, and computer modelling

Recent developments in clearing and microscopy enable 3D imaging with cellular resolution up to the whole organ level. These methods have been used extensively in neurobiology, but their uptake in other fields has been much more limited. Application of this approach to the human heart and effective use of the data acquired present challenges of scale and complexity. Four interlinked issues need to be addressed: 1) efficient clearing and labelling of heart tissue, 2) fast microscopic imaging of human-scale samples, 3) handling and processing of multi-terabyte 3D images, and 4) extraction of structural information in computationally tractable structure-based models of cardiac function. Preliminary studies show that each of these requirements can be achieved with the appropriate application and development of existing technologies.

High-resolution Fourier light-field microscopy for volumetric multi-color live-cell imaging

Volumetric interrogation of the organization and processes of intracellular organelles and molecules in cellular systems with a high spatiotemporal resolution is essential for understanding cell physiology, development, and pathology. Here, we report high-resolution Fourier light-field microscopy (HR-FLFM) for fast and volumetric live-cell imaging. HR-FLFM transforms conventional cell microscopy and enables exploration of less accessible spatiotemporal-limiting regimes for single-cell studies. The results present a near-diffraction-limited resolution in all three dimensions, a five-fold extended focal depth to several micrometers, and a scanning-free volume acquisition time up to milliseconds. The system demonstrates instrumentation accessibility, low photo damage for continuous observation, and high compatibility with general cell assays. We anticipate HR-FLFM to offer a promising methodological pathway for investigating a wide range of intracellular processes and functions with exquisite spatiotemporal contextual details.

2.5D microscopy: Fast, high-throughput imaging via volumetric projection for quantitative subcellular analysis

Imaging-based single-cell analysis is essential to study the expression level and functions of biomolecules at subcellular resolution. However, its low throughput has prevented the measurement of numerous cellular features from multiples cells in a rapid and efficient manner. Here we report 2.5D microscopy that significantly improves the throughput of fluorescence imaging systems while maintaining high-resolution and single-molecule sensitivity. Instead of sequential z-scanning, volumetric information is projected onto a 2D image plane in a single shot by engineering the emitted fluorescence light. Our approach provides an improved imaging speed and uniform focal response within a specific imaging depth, which enabled us to perform quantitative single-molecule RNA measurements over a 2×2 mm2 region within an imaging depth of ~5 μm for mammalian cells in <10 min and immunofluorescence imaging at a >30 Hz volumetric frame rate with reduced photobleaching. Our microscope also offers the ability of multi-color imaging, depth control and super-resolution imaging.

Accurate spectroscopy with sCMOS cameras at ultra-low intensities of light

Modern image detectors with exceptionally low readout noise of about one electron (e^-) per pixel allow for applications with ultra-low levels of light intensity. In this Letter, we report a property of scientific CMOS detectors that makes accurate spectroscopy at ultra-low levels of illumination depending on a thorough calibration procedure. Our results reveal that pixel sensitivity to light may have significant nonlinearity at accumulation levels smaller than 50e^- per pixel. The sensitivity decreases by a factor of ∼0.7 at an accumulation level of ∼1e^- per pixel and photon detection rate of about 170 Hz. We demonstrate that the nature of this nonlinearity might be quite complicated: the photoelectric response of a pixel depends on both the number of accumulated electrons and the detection count rate (at rates larger than 250 Hz).

Characterizing and correcting camera noise in back-illuminated sCMOS cameras

With promising properties of fast imaging speed, large field-of-view, relative low cost and many others, back-illuminated sCMOS cameras have been receiving intensive attention for low light level imaging in the past several years. However, due to the pixel-to-pixel difference of camera noise (called noise non-uniformity) in sCMOS cameras, researchers may hesitate to use them in some application fields, and sometimes wonder whether they should optimize the noise non-uniformity of their sCMOS cameras before using them in a specific application scenario. In this paper, we systematically characterize the impact of different types of sCMOS noise on image quality and perform corrections to these types of sCMOS noise using three representative algorithms (PURE, NCS and MLE_sCMOS). We verify that it is possible to use appropriate correction methods to push the non-uniformity of major types of camera noise, including readout noise, offset, and photon response, to a satisfactory level for conventional microscopy and single molecule localization microscopy. We further find out that, after these corrections, global read noise becomes a major concern that limits the imaging performance of back-illuminated sCMOS cameras. We believe this study provides new insights into the understanding of camera noise in back-illuminated sCMOS cameras, and also provides useful information for future development of this promising camera technology.

Imaging of spine synapses using super-resolution microscopy

Neuronal circuits in the neocortex and hippocampus are essential for higher brain functions such as motor learning and spatial memory. In the mammalian forebrain, most excitatory synapses of pyramidal neurons are formed on spines, which are tiny protrusions extending from the dendritic shaft. The spine contains specialized molecular machinery that regulates synaptic transmission and plasticity. Spine size correlates with the efficacy of synaptic transmission, and spine morphology affects signal transduction at the post-synaptic compartment. Plasticity-related changes in the structural and molecular organization of spine synapses are thought to underlie the cellular basis of learning and memory. Recent advances in super-resolution microscopy have revealed the molecular mechanisms of the nanoscale synaptic structures regulating synaptic transmission and plasticity in living neurons, which are difficult to investigate using electron microscopy alone. In this review, we summarize recent advances in super-resolution imaging of spine synapses and discuss the implications of nanoscale structures in the regulation of synaptic function, learning, and memory.

Miniaturized modular-array fluorescence microscopy

Fluorescence live-cell imaging allows for continuous interrogation of cellular behaviors, and the recent development of portable live-cell imaging platforms has rapidly transformed conventional schemes with high adaptability, cost-effective functionalities and easy accessibility to cell-based assays. However, broader applications remain restrictive due to compatibility with conventional cell culture workflow and biochemical sensors, accessibility to up-right physiological imaging, or parallelization of data acquisition. Here, we introduce miniaturized modular-array fluorescence microscopy (MAM) for compact live-cell imaging in flexible formats. We advance the current miniscopy technology to devise an up-right modular architecture, each combining a gradient-index (GRIN) objective and individually-addressed illumination and acquisition components. Parallelization of an array of such modular devices allows for multi-site data acquisition in situ using conventional off-the-shelf cell chambers. Compared with existing methods, the device offers a high fluorescence sensitivity and efficiency, exquisite spatiotemporal resolution (∼3 µm and up to 60 Hz), a configuration compatible with conventional cell culture assays and physiological imaging, and an effective parallelization of data acquisition. The system has been demonstrated using various calibration and biological samples and experimental conditions, representing a promising solution to time-lapse in situ single-cell imaging and analysis.

Variance lower bound on fluorescence microscopy image denoising

The signal to noise ratio of high-speed fluorescence microscopy is heavily influenced by photon counting noise and sensor noise due to the expected low photon budget. Denoising algorithms are developed to decrease these noise fluctuations in microscopy data by incorporating additional knowledge or assumptions about imaging systems or biological specimens. One question arises: whether there exists a theoretical precision limit for the performance of a microscopy denoising algorithm. In this paper, combining Cramér-Rao Lower Bound with constraints and the low-pass-filter property of microscope systems, we develop a method to calculate a theoretical variance lower bound of microscopy image denoising. We show that this lower bound is influenced by photon count, readout noise, detection wavelength, effective pixel size and the numerical aperture of the microscope system. We demonstrate our development by comparing multiple state-of-the-art denoising algorithms to this bound. This method establishes a framework to generate theoretical performance limit, under a specific prior knowledge, or assumption, as a reference benchmark for microscopy denoising algorithms.

Structure-dependent amplification for denoising and background correction in Fourier ptychographic microscopy

Fourier Ptychographic Microscopy (FPM) allows high resolution imaging using iterative phase retrieval to recover an estimate of the complex object from a series of images captured under oblique illumination. FPM is particularly sensitive to noise and uncorrected background signals as it relies on combining information from brightfield and noisy darkfield (DF) images. In this article we consider the impact of different noise sources in FPM and show that inadequate removal of the DF background signal and associated noise are the predominant cause of artefacts in reconstructed images. We propose a simple solution to FPM background correction and denoising that outperforms existing methods in terms of image quality, speed and simplicity, whilst maintaining high spatial resolution and sharpness of the reconstructed image. Our method takes advantage of the data redundancy in real space within the acquired dataset to boost the signal-to-background ratio in the captured DF images, before optimally suppressing background signal. By incorporating differentially denoised images within the classic FPM iterative phase retrieval algorithm, we show that it is possible to achieve efficient removal of background artefacts without suppression of high frequency information. The method is tested using simulated data and experimental images of thin blood films, bone marrow and liver tissue sections. Our approach is non-parametric, requires no prior knowledge of the noise distribution and can be directly applied to other hardware platforms and reconstruction algorithms making it widely applicable in FPM.

FluoSim: simulator of single molecule dynamics for fluorescence live-cell and super-resolution imaging of membrane proteins

Fluorescence live-cell and super-resolution microscopy methods have considerably advanced our understanding of the dynamics and mesoscale organization of macro-molecular complexes that drive cellular functions. However, different imaging techniques can provide quite disparate information about protein motion and organization, owing to their respective experimental ranges and limitations. To address these issues, we present here a robust computer program, called FluoSim, which is an interactive simulator of membrane protein dynamics for live-cell imaging methods including SPT, FRAP, PAF, and FCS, and super-resolution imaging techniques such as PALM, dSTORM, and uPAINT. FluoSim integrates diffusion coefficients, binding rates, and fluorophore photo-physics to calculate in real time the localization and intensity of thousands of independent molecules in 2D cellular geometries, providing simulated data directly comparable to actual experiments. FluoSim was thoroughly validated against experimental data obtained on the canonical neurexin-neuroligin adhesion complex at cell-cell contacts. This unified software allows one to model and predict membrane protein dynamics and localization at the ensemble and single molecule level, so as to reconcile imaging paradigms and quantitatively characterize protein behavior in complex cellular environments.