Fast and accurate three-dimensional point spread function computation for fluorescence microscopy

PSF-Radon transform algorithm: Measurement of the point-spread function from the Radon transform of the line-spread function

In this article, we present a new method called point spread function (PSF)-Radon transform algorithm. This algorithm consists on recovering the instrument PSF from the Radon transform (in the line direction axis) of the line spread function (i.e., the image of a line). We present the method and tested with synthetic images, and real images from macro lens camera and microscopy. A stand-alone program along with a tutorial is available, for any interested user, in Martinez (PSF-Radon transform algorithm, standalone program). RESEARCH HIGHLIGHTS: Determining the instrument PSF is a key issue. Precise PSF determinations are mandatory if image improvement is performed numerically by deconvolution. Much less exposure time to achieve the same performance than a measurement of the PSF from a very small bead. Does not require having to adjust the PSF by an analytical function to overcome the noise uncertainties.

Dual-step reconstruction algorithm to improve microscopy resolution by deep learning

Deep learning plays an important role in the field of machine learning, which has been developed and used in a wide range of areas. Many deep-learning-based methods have been proposed to improve image resolution, most of which are based on image-to-image translation algorithms. The performance of neural networks used to achieve image translation always depends on the feature difference between input and output images. Therefore, these deep-learning-based methods sometimes do not have good performance when the feature differences between low-resolution and high-resolution images are too large. In this paper, we introduce a dual-step neural network algorithm to improve image resolution step by step. Compared with conventional deep-learning methods that use input and output images with huge differences for training, this algorithm learning from input and output images with fewer differences can improve the performance of neural networks. This method was used to reconstruct high-resolution images of fluorescence nanoparticles in cells.

Axial profiling of interferometric scattering enables an accurate determination of nanoparticle size

Interferometric scattering (iSCAT) microscopy has undergone significant development in recent years. It is a promising technique for imaging and tracking nanoscopic label-free objects with nanometer localization precision. The current iSCAT-based photometry technique allows quantitative estimation for the size of a nanoparticle by measuring iSCAT contrast and has been successfully applied to nano-objects smaller than the Rayleigh scattering limit. Here we provide an alternative method that overcomes such size limitations. We take into account the axial variation of iSCAT contrast and utilize a vectorial point spread function model to uncover the position of a scattering dipole and, consequently, the size of the scatterer, which is not limited to the Rayleigh limit. We found that our technique accurately measures the size of spherical dielectric nanoparticles in a purely optical and non-contact way. We also tested fluorescent nanodiamonds (fND) and obtained a reasonable estimate for the size of fND particles. Together with fluorescence measurement from fND, we observed a correlation between the fluorescent signal and the size of fND. Our results showed that the axial pattern of iSCAT contrast provides sufficient information for the size of spherical particles. Our method enables us to measure the size of nanoparticles from tens of nanometers and beyond the Rayleigh limit with nanometer precision, making a versatile all-optical nanometric technique.

Accurate unsupervised estimation of aberrations in digital holographic microscopy for improved quantitative reconstruction

In the context of digital in-line holographic microscopy, we describe an unsupervised methodology to estimate the aberrations of an optical microscopy system from a single hologram. The method is based on the Inverse Problems Approach reconstructions of holograms of spherical objects. The forward model is based on a Lorenz-Mie model distorted by optical aberrations described by Zernike polynomials. This methodology is thus able to characterize most varying aberrations in the field of view in order to take them into account to improve the reconstruction of any sample. We show that this approach increases the repeatability and quantitativity of the reconstructions in both simulations and experimental data. We use the Cramér-Rao lower bounds to study the accuracy of the reconstructions. Finally, we demonstrate the efficiency of this aberration calibration with image reconstructions using a phase retrieval algorithm as well as a regularized inverse problems algorithm.

Three-dimensional point spread function estimation method for mid-wave infrared microscope imaging

A three-dimensional point spread function experimental estimation method based on the system's focal plane array spatial local impulse response of a mid-wave infrared microscope is presented. The method uses several out-of-focus two-dimensional point spread function planes to achieve a single three-dimensional point spread function of the whole microscope's optical spreading, expanding the limits of infrared optical technology by one dimension. This technique includes stages of image acquisition, nonuniformity correction, filtering, and multi-planar reconstruction steps, and its effectiveness is demonstrated on biological sample image restoration by means of a multi-planar refocusing application.

Accurate 3D morphological computational model reconstruction of suspended cells imaged through stratified media by the precise depth-varying point spread function method

Accurate three-dimensional (3D) morphological computational models of cells are important in a number of biological studies. This study proposes a precise depth-varying point spread function (PDV-PSF) method for reconstructing 3D computational models of suspended cells from two-dimensional (2D) confocal image stacks. Our approach deblurs the 2D images in horizontal plane and corrects the deformation in vertical direction to overcome the refractive index mismatch problem caused by suspended cells imaging through stratified media. Standard fluorescent polystyrene spheres and Jurkat T-lymphocytes are selected to evaluate the validity and accuracy of this PDV-PSF method. Qualitative and quantitative results demonstrate that our approach has superior performance in 3D morphological computational models reconstruction of suspended cells.

Three-dimensional diffusion coefficient measurement by a large depth-of-field rotating point spread function

A prominent challenge in single-molecule localization microscopy is the real-time, fast, and accurate localization of nano-objects moving in three-dimensional (3D) samples. A well-established method for 3D single-molecule localization is the double-helix pointspread-function (DH-PSF) engineering, which uses additional optical elements to make the PSF exhibit different rotation angles with different nanoparticle depths. However, the compact main lobe size, effective detection depth, and precise conversion between rotation angle and depth are necessary, posing challenges to the DH-PSF generation method. Here we generate a more compact DH-PSF using Fresnel-zone-based spiral phases, and the pure phase mask achieves high transmission efficiency. The final generated DH-PSFs have a linear rotation rate at each axial position, showing a more accurate rotation angle and depth conversion. The Cramer-Rao lower limit calculation results show that the axial depth of DH-PSF extends to ∼11µm with an axial localization precision of ∼45nm at 3000 photons and average background noise of 15. We measured the diffusion coefficient of nanospheres in different concentrations of glycerol using the generated DH-PSF. The measured results are within 6% error from the theoretical values, indicating the superior performance of the DH-PSF for nanoparticle diffusion coefficient measurements.

The narrowing of dendrite branches across nodes follows a well-defined scaling law

The systematic variation of diameters in branched networks has tantalized biologists since the discovery of da Vinci's rule for trees. Da Vinci's rule can be formulated as a power law with exponent two: The square of the mother branch's diameter is equal to the sum of the squares of those of the daughters. Power laws, with different exponents, have been proposed for branching in circulatory systems (Murray's law with exponent 3) and in neurons (Rall's law with exponent 3/2). The laws have been derived theoretically, based on optimality arguments, but, for the most part, have not been tested rigorously. Using superresolution methods to measure the diameters of dendrites in highly branched Drosophila class IV sensory neurons, we have found that these types of power laws do not hold. In their place, we have discovered a different diameter-scaling law: The cross-sectional area is proportional to the number of dendrite tips supported by the branch plus a constant, corresponding to a minimum diameter of the terminal dendrites. The area proportionality accords with a requirement for microtubules to transport materials and nutrients for dendrite tip growth. The minimum diameter may be set by the force, on the order of a few piconewtons, required to bend membrane into the highly curved surfaces of terminal dendrites. Because the observed scaling differs from Rall's law, we propose that cell biological constraints, such as intracellular transport and protrusive forces generated by the cytoskeleton, are important in determining the branched morphology of these cells.

Artefact removal in ground truth deficient fluctuations-based nanoscopy images using deep learning

Image denoising or artefact removal using deep learning is possible in the availability of supervised training dataset acquired in real experiments or synthesized using known noise models. Neither of the conditions can be fulfilled for nanoscopy (super-resolution optical microscopy) images that are generated from microscopy videos through statistical analysis techniques. Due to several physical constraints, a supervised dataset cannot be measured. Further, the non-linear spatio-temporal mixing of data and valuable statistics of fluctuations from fluorescent molecules that compete with noise statistics. Therefore, noise or artefact models in nanoscopy images cannot be explicitly learned. Here, we propose a robust and versatile simulation-supervised training approach of deep learning auto-encoder architectures for the highly challenging nanoscopy images of sub-cellular structures inside biological samples. We show the proof of concept for one nanoscopy method and investigate the scope of generalizability across structures, and nanoscopy algorithms not included during simulation-supervised training. We also investigate a variety of loss functions and learning models and discuss the limitation of existing performance metrics for nanoscopy images. We generate valuable insights for this highly challenging and unsolved problem in nanoscopy, and set the foundation for the application of deep learning problems in nanoscopy for life sciences.

Soft thresholding schemes for multiple signal classification algorithm

Multiple signal classification algorithm (MUSICAL) exploits temporal fluctuations in fluorescence intensity to perform super-resolution microscopy by computing the value of a super-resolving indicator function across a fine sample grid. A key step in the algorithm is the separation of the measurements into signal and noise subspaces, based on a single user-specified parameter called the threshold. The resulting image is strongly sensitive to this parameter and the subjectivity arising from multiple practical factors makes it difficult to determine the right rule of selection. We address this issue by proposing soft thresholding schemes derived from a new generalized framework for indicator function design. We show that the new schemes significantly alleviate the subjectivity and sensitivity of hard thresholding while retaining the super-resolution ability. We also evaluate the trade-off between resolution and contrast and the out-of-focus light rejection using the various indicator functions. Through this, we create significant new insights into the use and further optimization of MUSICAL for a wide range of practical scenarios.

Point spread function degradation model of a polarization imaging system for wide-field subwavelength nanoparticles

We propose a comprehensive point spread function (PSF) degradation model, which considers multiple factors consisting of degradation of specimen retardant sampling and polarization angularly anamorphic sampling, to indicate the image degradation characteristics of polarization imaging systems. First, a one-layer optical coherence tomography (OCT) model was established to express the retardancy of medium-loading specimens. Then, a PSF degradation model of angularly anamorphic polarization sampling was deduced through the retrieval of Stokes parameters. Finally, maximum a posteriori probability (MAP) was adopted to assess the distribution of the proposed model. Hypothesis testing using actual data and numerical simulations demonstrated that the error of the system followed an asymmetric generalized Gaussian distribution (AGGD). Finite-difference time-domain (FDTD) simulation results and an actual imaging experiment demonstrate the consistency of the proposed model and the degradation characteristics of the PSF, which provide support for the improved accuracy and enhanced image quality of the optical field retrieval of nanoparticles.

Precise correlative method of Cryo-SXT and Cryo-FM for organelle identification

Cryogenic soft X-ray tomography (Cryo-SXT) is ideally suitable to image the 3D sub-cellular architecture and organization of cells with high resolution in the near-native preservation state. Cryogenic fluorescence microscopy (Cryo-FM) can determine the location of a molecule of interest that has been labeled with a fluorescent tag, thus revealing the function of the cells. To understand the relations between the sub-cellular architecture and the function of cells, correlative Cryo-SXT and Cryo-FM was applied. This method required the matching of images of different modalities, and the accuracy of the matching is important. Here, a precise correlative method of Cryo-SXT and Cryo-FM is introduced. The capability of matching images of different modalities with high resolution was verified by simulations and practical experiments, and the method was used to identify vacuoles and mitochondria.

Cytokit: a single-cell analysis toolkit for high dimensional fluorescent microscopy imaging

BACKGROUND

Multiplexed in-situ fluorescent imaging offers several advantages over single-cell assays that do not preserve the spatial characteristics of biological samples. This spatial information, in addition to morphological properties and extensive intracellular or surface marker profiling, comprise promising avenues for rapid advancements in the understanding of disease progression and diagnosis. As protocols for conducting such imaging experiments continue to improve, it is the intent of this study to provide and validate software for processing the large quantity of associated data in kind.

RESULTS

Cytokit offers (i) an end-to-end, GPU-accelerated image processing pipeline; (ii) efficient input/output (I/O) strategies for operations specific to high dimensional microscopy; and (iii) an interactive user interface for cross filtering of spatial, graphical, expression, and morphological cell properties within the 100+ GB image datasets common to multiplexed immunofluorescence. Image processing operations supported in Cytokit are generally sourced from existing deep learning models or are at least in part adapted from open source packages to run in a single or multi-GPU environment. The efficacy of these operations is demonstrated through several imaging experiments that pair Cytokit results with those from an independent but comparable assay. A further validation also demonstrates that previously published results can be reproduced from a publicly available multiplexed image dataset.

CONCLUSION

Cytokit is a collection of open source tools for quantifying and analyzing properties of individual cells in large fluorescent microscopy datasets that are often, but not necessarily, generated from multiplexed antibody labeling protocols over many fields of view or time periods. This project is best suited to bioinformaticians or other technical users that wish to analyze such data in a batch-oriented, high-throughput setting. All source code, documentation, and data generated for this article are available under the Apache License 2.0 at https://github.com/hammerlab/cytokit .

Single-step fabrication of surface waveguides in fused silica with few-cycle laser pulses

Direct laser writing of surface waveguides with ultrashort pulses is a crucial achievement towards all-laser manufacturing of photonic integrated circuits sensitive to their environment. In this Letter, few-cycle laser pulses (with a sub-10 fs duration) are used to produce subsurface waveguides in a non-doped, non-coated fused-silica substrate. The fabrication technique relies on laser-induced microdensification below the threshold for nanopore formation. The optical losses of the fabricated waveguides are governed by the optical properties of the superstrate. We have measured losses ranging from less than 0.1 dB/mm (air superstrate) up to 2.8 dB/mm when immersion oil is applied on top of the waveguide.

Imaging effectiveness calculator for non-design microscope samples

When attempting to integrate single-molecule fluorescence microscopy with microfabricated devices such as microfluidic channels, fabrication constraints may prevent using traditional coverslips. Instead, the fabricated devices may require imaging through material with a different thickness or index of refraction. Altering either can easily reduce the quality of the image formation (measured by the Strehl ratio) by a factor of 2 or more, reducing the signal-to-noise ratio accordingly. In such cases, successful detection of single-molecule fluorescence may prove difficult or impossible. Here we provide software to calculate the effect of non-design materials upon the Strehl ratio or ensquared energy and explore the impact of common materials used in microfabrication.

Calibration Routine for Quantitative Three-Dimensional Flow Field Measurements in Drying Polymer Solutions Subject to Marangoni Convection

Surface-tension induced flows may have a significant impact on the surface topography of thin films or small printed structures derived from polymer solution processing. Despite a century of research on Marangoni convection, the community lacks quantitative experimental flow field data, especially from within drying solutions. We utilize multifocal micro particle tracking velocimetry (µPTV) to obtain these data and show a calibration routine based on point spread function (PSF) simulations as well as experimental data. The results account for a varying sample refractive index, beneficial cover-glass correction collar settings as well as a multifocal lens system. Finally, the calibration procedure is utilized exemplarily to reconstruct a three-dimensional, transient flow field within a poly(vinyl acetate)-methanol solution dried with inhomogeneous boundary conditions.

Autonomous illumination control for localization microscopy

Super-resolution fluorescence microscopy improves spatial resolution, but this comes at a loss of image throughput and presents unique challenges in identifying optimal acquisition parameters. Microscope automation routines can offset these drawbacks, but thus far have required user inputs that presume a priori knowledge about the sample. Here, we develop a flexible illumination control system for localization microscopy comprised of two interacting components that require no sample-specific inputs: a self-tuning controller and a deep learning-based molecule density estimator that is accurate over an extended range of densities. This system obviates the need to fine-tune parameters and enables robust, autonomous illumination control for localization microscopy.

On-the-fly estimation of a microscopy point spread function

A proper estimation of realistic point-spread function (PSF) in optical microscopy can significantly improve the deconvolution performance and assist the microscope calibration process. In this work, by exemplifying 3D wide-field fluorescence microscopy, we propose an approach for estimating the spherically aberrated PSF of a microscope, directly from the observed samples. The PSF, expressed as a linear combination of 4 basis functions, is obtained directly from the acquired image by minimizing a novel criterion, which is derived from the noise statistics in the microscope. We demonstrate the effectiveness of the PSF approximation model and of our estimation method using both simulations and real experiments that were carried out on quantum dots. The principle of our PSF estimation approach is sufficiently flexible to be generalized non-spherical aberrations and other microscope modalities.

PURE-LET Image Deconvolution

We propose a non-iterative image deconvolution algorithm for data corrupted by Poisson or mixed Poisson-Gaussian noise. Many applications involve such a problem, ranging from astronomical to biological imaging. We parameterize the deconvolution process as a linear combination of elementary functions, termed as linear expansion of thresholds. This parameterization is then optimized by minimizing a robust estimate of the true mean squared error, the Poisson unbiased risk estimate. Each elementary function consists of a Wiener filtering followed by a pointwise thresholding of undecimated Haar wavelet coefficients. In contrast to existing approaches, the proposed algorithm merely amounts to solving a linear system of equations, which has a fast and exact solution. Simulation experiments over different types of convolution kernels and various noise levels indicate that the proposed method outperforms the state-of-the-art techniques, in terms of both restoration quality and computational complexity. Finally, we present some results on real confocal fluorescence microscopy images and demonstrate the potential applicability of the proposed method for improving the quality of these images.We propose a non-iterative image deconvolution algorithm for data corrupted by Poisson or mixed Poisson-Gaussian noise. Many applications involve such a problem, ranging from astronomical to biological imaging. We parameterize the deconvolution process as a linear combination of elementary functions, termed as linear expansion of thresholds. This parameterization is then optimized by minimizing a robust estimate of the true mean squared error, the Poisson unbiased risk estimate. Each elementary function consists of a Wiener filtering followed by a pointwise thresholding of undecimated Haar wavelet coefficients. In contrast to existing approaches, the proposed algorithm merely amounts to solving a linear system of equations, which has a fast and exact solution. Simulation experiments over different types of convolution kernels and various noise levels indicate that the proposed method outperforms the state-of-the-art techniques, in terms of both restoration quality and computational complexity. Finally, we present some results on real confocal fluorescence microscopy images and demonstrate the potential applicability of the proposed method for improving the quality of these images.