Wavelet-based background and noise subtraction for fluorescence microscopy images

Zero-shot denoising of microscopy images recorded at high-resolution limits

Conventional and electron microscopy visualize structures in the micrometer to nanometer range, and such visualizations contribute decisively to our understanding of biological processes. Due to different factors in recording processes, microscopy images are subject to noise. Especially at their respective resolution limits, a high degree of noise can negatively effect both image interpretation by experts and further automated processing. However, the deteriorating effects of strong noise can be alleviated to a large extend by image enhancement algorithms. Because of the inherent high noise, a requirement for such algorithms is their applicability directly to noisy images or, in the extreme case, to just a single noisy image without a priori noise level information (referred to as blind zero-shot setting). This work investigates blind zero-shot algorithms for microscopy image denoising. The strategies for denoising applied by the investigated approaches include: filtering methods, recent feed-forward neural networks which were amended to be trainable on noisy images, and recent probabilistic generative models. As datasets we consider transmission electron microscopy images including images of SARS-CoV-2 viruses and fluorescence microscopy images. A natural goal of denoising algorithms is to simultaneously reduce noise while preserving the original image features, e.g., the sharpness of structures. However, in practice, a tradeoff between both aspects often has to be found. Our performance evaluations, therefore, focus not only on noise removal but set noise removal in relation to a metric which is instructive about sharpness. For all considered approaches, we numerically investigate their performance, report their denoising/sharpness tradeoff on different images, and discuss future developments. We observe that, depending on the data, the different algorithms can provide significant advantages or disadvantages in terms of their noise removal vs. sharpness preservation capabilities, which may be very relevant for different virological applications, e.g., virological analysis or image segmentation.

Fully automatic online geometric calibration for non-circular cone-beam CT orbits using fiducials with unknown placement

BACKGROUND

Cone-beam CT (CBCT) with non-circular scanning orbits can improve image quality for 3D intraoperative image guidance. However, geometric calibration of such scans can be challenging. Existing methods typically require a prior image, specialized phantoms, presumed repeatable orbits, or long computation time.

PURPOSE

We propose a novel fully automatic online geometric calibration algorithm that does not require prior knowledge of fiducial configuration. The algorithm is fast, accurate, and can accommodate arbitrary scanning orbits and fiducial configurations.

METHODS

The algorithm uses an automatic initialization process to eliminate human intervention in fiducial localization and an iterative refinement process to ensure robustness and accuracy. We provide a detailed explanation and implementation of the proposed algorithm. Physical experiments on a lab test bench and a clinical robotic C-arm scanner were conducted to evaluate spatial resolution performance and robustness under realistic constraints.

RESULTS

Qualitative and quantitative results from the physical experiments demonstrate high accuracy, efficiency, and robustness of the proposed method. The spatial resolution performance matched that of our existing benchmark method, which used a 3D-2D registration-based geometric calibration algorithm.

CONCLUSIONS

We have demonstrated an automatic online geometric calibration method that delivers high spatial resolution and robustness performance. This methodology enables arbitrary scan trajectories and should facilitate translation of such acquisition methods in a clinical setting.

Dynamic Mode Decomposition of Multiphoton and Stimulated Emission Depletion Microscopy Data for Analysis of Fluorescent Probes in Cellular Membranes

An analysis of the membrane organization and intracellular trafficking of lipids often relies on multiphoton (MP) and super-resolution microscopy of fluorescent lipid probes. A disadvantage of particularly intrinsically fluorescent lipid probes, such as the cholesterol and ergosterol analogue, dehydroergosterol (DHE), is their low MP absorption cross-section, resulting in a low signal-to-noise ratio (SNR) in live-cell imaging. Stimulated emission depletion (STED) microscopy of membrane probes like Nile Red enables one to resolve membrane features beyond the diffraction limit but exposes the sample to a lot of excitation light and suffers from a low SNR and photobleaching. Here, dynamic mode decomposition (DMD) and its variant, higher-order DMD (HoDMD), are applied to efficiently reconstruct and denoise the MP and STED microscopy data of lipid probes, allowing for an improved visualization of the membranes in cells. HoDMD also allows us to decompose and reconstruct two-photon polarimetry images of TopFluor-cholesterol in model and cellular membranes. Finally, DMD is shown to not only reconstruct and denoise 3D-STED image stacks of Nile Red-labeled cells but also to predict unseen image frames, thereby allowing for interpolation images along the optical axis. This important feature of DMD can be used to reduce the number of image acquisitions, thereby minimizing the light exposure of biological samples without compromising image quality. Thus, DMD as a computational tool enables gentler live-cell imaging of fluorescent probes in cellular membranes by MP and STED microscopy.

Low-frequency background estimation and noise separation from high-frequency for background and noise subtraction

In fluorescence microscopy, background blur and noise are two main factors preventing the achievement of high-signal-to-noise ratio (SNR) imaging. Background blur primarily emanates from inherent factors including the spontaneous fluorescence of biological samples and out-of-focus backgrounds, while noise encompasses Gaussian and Poisson noise components. To achieve background blur subtraction and denoising simultaneously, a pioneering algorithm based on low-frequency background estimation and noise separation from high-frequency (LBNH-BNS) is presented, which effectively disentangles noise from the desired signal. Furthermore, it seamlessly integrates low-frequency features derived from background blur estimation, leading to the effective elimination of noise and background blur in wide-field fluorescence images. In comparisons with other state-of-the-art background removal algorithms, LBNH-BNS demonstrates significant advantages in key quantitative metrics such as peak signal-to-noise ratio (PSNR) and manifests substantial visual enhancements. LBNH-BNS holds immense potential for advancing the overall performance and quality of wide-field fluorescence imaging techniques.

Background-free wide-field fluorescence imaging using edge detection combined with HiLo

Fluorescence microscopy has been widely used in the field of biological imaging, but the disturbance of background noise has always been an unavoidable phenomenon. To obtain a background free image, a virtual HiLo based on edge detection (V-HiLo-ED) method for background removing is proposed, which is different from the existing popular software algorithms that obtain the background-free image by subtracting the estimated background, but the background-free image is directly reconstructed by estimating the foreground. Compared with two other popular software-based methods, the wavelet-based background and noise subtraction algorithm (WBNS) and the rolling ball algorithm (RBA), the V-HiLo-ED owns a better quality on achieving background-free imaging. Compared with hardware-based method such as HiLo method, V-HiLo-ED exhibits almost the same performance but faster speed. In combination with light sheet microscopy, the V-HiLo-ED further improves the signal-to-noise ratio of images with thick light-sheet. These experiment results indicate that the V-HiLo-ED owns the potentiality in many other image applications such as endoscopy.

3D photon counting integral imaging by using multi-level decomposition

In this paper, we propose three-dimensional (3D) photon counting integral imaging by using multi-level decomposition such as discrete wavelet transform to improve the visual quality and measurement accuracy under photon-starved conditions. Conventional 3D integral imaging can visualize 3D objects and acquire their depth information. However, the amount of irradiated light on the object causes the degradation of visual quality for 3D images under photon-starved conditions. To visualize 3D objects, photon counting integral imaging has been utilized. It can detect photons from 3D scenes by using a computational photon counting model, which is modelled by the Poisson random process. However, photons occur not only from objects but also in areas where objects do not exist. Moreover, photon fluctuation may occur in the scene through shot noise. Since these noise photons are measurement errors, it may decrease the image quality and accuracy. In contrast, our proposed method uses 2D discrete wavelet transform, which can emphasize the object photons effectively. Finally, our proposed method can enhance the visual quality of 3D images and provide more accurate depth information under photon-starved conditions. To prove the feasibility of our proposed method, we implement the optical experiment and calculate various image quality metrics.

Combining deep learning with SUPPOSe and compressed sensing for SNR-enhanced localization of overlapping emitters

We present gSUPPOSe, a novel, to the best of our knowledge, gradient-based implementation of the SUPPOSe algorithm that we have developed for the localization of single emitters. We study the performance of gSUPPOSe and compressed sensing STORM (CS-STORM) on simulations of single-molecule localization microscopy (SMLM) images at different fluorophore densities and in a wide range of signal-to-noise ratio conditions. We also study the combination of these methods with prior image denoising by means of a deep convolutional network. Our results show that gSUPPOSe can address the localization of multiple overlapping emitters even at a low number of acquired photons, outperforming CS-STORM in our quantitative analysis and having better computational times. We also demonstrate that image denoising greatly improves CS-STORM, showing the potential of deep learning enhanced localization on existing SMLM algorithms. The software developed in this work is available as open source Python libraries.

Two plus one is almost three: a fast approximation for multi-view deconvolution

Multi-view deconvolution is a powerful image-processing tool for light sheet fluorescence microscopy, providing isotropic resolution and enhancing the image content. However, performing these calculations on large datasets is computationally demanding and time-consuming even on high-end workstations. Especially in long-time measurements on developing animals, huge amounts of image data are acquired. To keep them manageable, redundancies should be removed right after image acquisition. To this end, we report a fast approximation to three-dimensional multi-view deconvolution, denoted 2D+1D multi-view deconvolution, which is able to keep up with the data flow. It first operates on the two dimensions perpendicular and subsequently on the one parallel to the rotation axis, exploiting the rotational symmetry of the point spread function along the rotation axis. We validated our algorithm and evaluated it quantitatively against two-dimensional and three-dimensional multi-view deconvolution using simulated and real image data. 2D+1D multi-view deconvolution takes similar computation time but performs markedly better than the two-dimensional approximation only. Therefore, it will be most useful for image processing in time-critical applications, where the full 3D multi-view deconvolution cannot keep up with the data flow.

Expression and Localization of AβPP in SH-SY5Y Cells Depends on Differentiation State

Neuroblastoma cell line SH-SY5Y, due to its capacity to differentiate into neurons, easy handling, and low cost, is a common experimental model to study molecular events leading to Alzheimer's disease (AD). However, it is prevalently used in its undifferentiated state, which does not resemble neurons affected by the disease. Here, we show that the expression and localization of amyloid-β protein precursor (AβPP), one of the key molecules involved in AD pathogenesis, is dramatically altered in SH-SY5Y cells fully differentiated by combined treatment with retinoic acid and BDNF. We show that insufficient differentiation of SH-SY5Y cells results in AβPP mislocalization.