Parametric blind deconvolution: a robust method for the simultaneous estimation of image and blur

On-the-fly estimation of a microscopy point spread function

A proper estimation of realistic point-spread function (PSF) in optical microscopy can significantly improve the deconvolution performance and assist the microscope calibration process. In this work, by exemplifying 3D wide-field fluorescence microscopy, we propose an approach for estimating the spherically aberrated PSF of a microscope, directly from the observed samples. The PSF, expressed as a linear combination of 4 basis functions, is obtained directly from the acquired image by minimizing a novel criterion, which is derived from the noise statistics in the microscope. We demonstrate the effectiveness of the PSF approximation model and of our estimation method using both simulations and real experiments that were carried out on quantum dots. The principle of our PSF estimation approach is sufficiently flexible to be generalized non-spherical aberrations and other microscope modalities.

DeconvolutionLab2: An open-source software for deconvolution microscopy

Images in fluorescence microscopy are inherently blurred due to the limit of diffraction of light. The purpose of deconvolution microscopy is to compensate numerically for this degradation. Deconvolution is widely used to restore fine details of 3D biological samples. Unfortunately, dealing with deconvolution tools is not straightforward. Among others, end users have to select the appropriate algorithm, calibration and parametrization, while potentially facing demanding computational tasks. To make deconvolution more accessible, we have developed a practical platform for deconvolution microscopy called DeconvolutionLab. Freely distributed, DeconvolutionLab hosts standard algorithms for 3D microscopy deconvolution and drives them through a user-oriented interface. In this paper, we take advantage of the release of DeconvolutionLab2 to provide a complete description of the software package and its built-in deconvolution algorithms. We examine several standard algorithms used in deconvolution microscopy, notably: Regularized inverse filter, Tikhonov regularization, Landweber, Tikhonov-Miller, Richardson-Lucy, and fast iterative shrinkage-thresholding. We evaluate these methods over large 3D microscopy images using simulated datasets and real experimental images. We distinguish the algorithms in terms of image quality, performance, usability and computational requirements. Our presentation is completed with a discussion of recent trends in deconvolution, inspired by the results of the Grand Challenge on deconvolution microscopy that was recently organized.

Blind deconvolution of 3D fluorescence microscopy using depth-variant asymmetric PSF

The 3D wide-field fluorescence microscopy suffers from depth-variant asymmetric blur. The depth-variance and axial asymmetry are due to refractive index mismatch between the immersion and the specimen layer. The radial asymmetry is due to lens imperfections and local refractive index inhomogeneities in the specimen. To obtain the PSF that has these characteristics, there were PSF premeasurement trials. However, they are useless since imaging conditions such as camera position and refractive index of the specimen are changed between the premeasurement and actual imaging. In this article, we focus on removing unknown depth-variant asymmetric blur in such an optical system under the assumption of refractive index homogeneities in the specimen. We propose finding few parameters in the mathematical PSF model from observed images in which the PSF model has a depth-variant asymmetric shape. After generating an initial PSF from the analysis of intensities in the observed image, the parameters are estimated based on a maximum likelihood estimator. Using the estimated PSF, we implement an accelerated GEM algorithm for image deconvolution. Deconvolution result shows the superiority of our algorithm in terms of accuracy, which quantitatively evaluated by FWHM, relative contrast, standard deviation values of intensity peaks and FWHM. Microsc. Res. Tech. 79:480-494, 2016. © 2016 Wiley Periodicals, Inc.

Blind Depth-variant Deconvolution of 3D Data in Wide-field Fluorescence Microscopy

This paper proposes a new deconvolution method for 3D fluorescence wide-field microscopy. Most previous methods are insufficient in terms of restoring a 3D cell structure, since a point spread function (PSF) is simply assumed as depth-invariant, whereas a PSF of microscopy changes significantly along the optical axis. A few methods that consider a depth-variant PSF have been proposed; however, they are impractical, since they are non-blind approaches that use a known PSF in a pre-measuring condition, whereas an imaging condition of a target image is different from that of the pre-measuring. To solve these problems, this paper proposes a blind approach to estimate depth-variant specimen-dependent PSF and restore 3D cell structure. It is shown by experiments on that the proposed method outperforms the previous ones in terms of suppressing axial blur. The proposed method is composed of the following three steps: First, a non-parametric averaged PSF is estimated by the Richardson Lucy algorithm, whose initial parameter is given by the central depth prediction from intensity analysis. Second, the estimated PSF is fitted to Gibson's parametric PSF model via optimization, and depth-variant PSFs are generated. Third, a 3D cell structure is restored by using a depth-variant version of a generalized expectation-maximization.

A novel SURE-based criterion for parametric PSF estimation

We propose an unbiased estimate of a filtered version of the mean squared error--the blur-SURE (Stein's unbiased risk estimate)--as a novel criterion for estimating an unknown point spread function (PSF) from the degraded image only. The PSF is obtained by minimizing this new objective functional over a family of Wiener processings. Based on this estimated blur kernel, we then perform nonblind deconvolution using our recently developed algorithm. The SURE-based framework is exemplified with a number of parametric PSF, involving a scaling factor that controls the blur size. A typical example of such parametrization is the Gaussian kernel. The experimental results demonstrate that minimizing the blur-SURE yields highly accurate estimates of the PSF parameters, which also result in a restoration quality that is very similar to the one obtained with the exact PSF, when plugged into our recent multi-Wiener SURE-LET deconvolution algorithm. The highly competitive results obtained outline the great potential of developing more powerful blind deconvolution algorithms based on SURE-like estimates.

A Kalman filter approach for denoising and deblurring 3-D microscopy images

This paper proposes a new method for removing noise and blurring from 3D microscopy images. The main contribution is the definition of a space-variant generating model of a 3-D signal, which is capable to stochastically describe a wide class of 3-D images. Unlike other approaches, the space-variant structure allows the model to consider the information on edge locations, if available. A suitable description of the image acquisition process, including blurring and noise, is then associated to the model. A state-space realization is finally derived, which is amenable to the application of standard Kalman filter as an image restoration algorithm. The so obtained method is able to remove, at each spatial step, both blur and noise, via a linear minimum variance recursive one-shot procedure, which does not require the simultaneous processing of the whole image. Numerical results on synthetic and real microscopy images confirm the merit of the approach.

Depth-variant deconvolution of 3D widefield fluorescence microscopy using the penalized maximum likelihood estimation method

We investigated the deconvolution of 3D widefield fluorescence microscopy using the penalized maximum likelihood estimation method and the depth-variant point spread function (DV-PSF). We build the DV-PSF by fitting a parameterized theoretical PSF model to an experimental microbead image. On the basis of the constructed DV-PSF, we restore the 3D widefield microscopy by minimizing an objective function consisting of a negative Poisson likelihood function and a total variation regularization function. In simulations and experiments, the proposed method showed better performance than existing methods.

Quantitative in vivo imaging of embryonic development: opportunities and challenges

Animal models are critically important for a mechanistic understanding of embryonic morphogenesis. For decades, visualizing these rapid and complex multidimensional events has relied on projection images and thin section reconstructions. While much insight has been gained, fixed tissue specimens offer limited information on dynamic processes that are essential for tissue assembly and organ patterning. Quantitative imaging is required to unlock the important basic science and clinically relevant secrets that remain hidden. Recent advances in live imaging technology have enabled quantitative longitudinal analysis of embryonic morphogenesis at multiple length and time scales. Four different imaging modalities are currently being used to monitor embryonic morphogenesis: optical, ultrasound, magnetic resonance imaging (MRI), and micro-computed tomography (micro-CT). Each has its advantages and limitations with respect to spatial resolution, depth of field, scanning speed, and tissue contrast. In addition, new processing tools have been developed to enhance live imaging capabilities. In this review, we analyze each type of imaging source and its use in quantitative study of embryonic morphogenesis in small animal models. We describe the physics behind their function, identify some examples in which the modality has revealed new quantitative insights, and then conclude with a discussion of new research directions with live imaging.

Automated clarity assessment of retinal images using regionally based structural and statistical measures

An automated image analysis system for application in mass medical screening must assess the clarity of the images before analysing their content. This is the case in grading for diabetic retinopathy screening where the failure to assess clarity could result in retinal images of people with retinopathy being erroneously classed as normal. This paper compares methods of clarity assessment based on the degradation of visible structures and based on the deviation of image properties outside expected norms caused by clarity loss. Vessel visibility measures and statistical measures were determined at locations in the image which have high saliency and these were used to obtain an image clarity assessment using supervised classification. The usefulness of the measures as indicators of image clarity was assessed. Tests were performed on 987 disc-centred and macula-centred retinal photographs (347 with inadequate clarity) obtained from the English National Screening Programme. Images with inadequate clarity were detected with 92.6% sensitivity at 90% specificity. In a set of 2000 macula-centred images (200 with inadequate clarity) from the Scottish Screening Programme, inadequate clarity was detected with 96.7% sensitivity at 90% specificity. This study has shown that structural and statistical measures are equally useful for retinal image clarity assessment.

Application of regularized Richardson-Lucy algorithm for deconvolution of confocal microscopy images

Although confocal microscopes have considerably smaller contribution of out-of-focus light than widefield microscopes, the confocal images can still be enhanced mathematically if the optical and data acquisition effects are accounted for. For that, several deconvolution algorithms have been proposed. As a practical solution, maximum-likelihood algorithms with regularization have been used. However, the choice of regularization parameters is often unknown although it has considerable effect on the result of deconvolution process. The aims of this work were: to find good estimates of deconvolution parameters; and to develop an open source software package that would allow testing different deconvolution algorithms and that would be easy to use in practice. Here, Richardson-Lucy algorithm has been implemented together with the total variation regularization in an open source software package IOCBio Microscope. The influence of total variation regularization on deconvolution process is determined by one parameter. We derived a formula to estimate this regularization parameter automatically from the images as the algorithm progresses. To assess the effectiveness of this algorithm, synthetic images were composed on the basis of confocal images of rat cardiomyocytes. From the analysis of deconvolved results, we have determined under which conditions our estimation of total variation regularization parameter gives good results. The estimated total variation regularization parameter can be monitored during deconvolution process and used as a stopping criterion. An inverse relation between the optimal regularization parameter and the peak signal-to-noise ratio of an image is shown. Finally, we demonstrate the use of the developed software by deconvolving images of rat cardiomyocytes with stained mitochondria and sarcolemma obtained by confocal and widefield microscopes.

Fast nonnegative deconvolution for spike train inference from population calcium imaging

Fluorescent calcium indicators are becoming increasingly popular as a means for observing the spiking activity of large neuronal populations. Unfortunately, extracting the spike train of each neuron from a raw fluorescence movie is a nontrivial problem. This work presents a fast nonnegative deconvolution filter to infer the approximately most likely spike train of each neuron, given the fluorescence observations. This algorithm outperforms optimal linear deconvolution (Wiener filtering) on both simulated and biological data. The performance gains come from restricting the inferred spike trains to be positive (using an interior-point method), unlike the Wiener filter. The algorithm runs in linear time, and is fast enough that even when simultaneously imaging >100 neurons, inference can be performed on the set of all observed traces faster than real time. Performing optimal spatial filtering on the images further refines the inferred spike train estimates. Importantly, all the parameters required to perform the inference can be estimated using only the fluorescence data, obviating the need to perform joint electrophysiological and imaging calibration experiments.

Fidelity optimization for aberration-tolerant hybrid imaging systems

Several phase-modulation functions have been reported to decrease the aberration variance of the modulation-transfer-function (MTF) in aberration-tolerant hybrid imaging systems. The choice of this phase-modulation function is crucial for optimization of the overall system performance. To prevent a significant loss in signal-to-noise ratio, it is common to enforce restorability constraints on the MTF, requiring trade of aberration-tolerance and noise-gain. Instead of optimizing specific MTF characteristics, we directly minimize the expected imaging-error of the joint design. This method is used to compare commonly used phase-modulation functions: the antisymmetric generalized cubic polynomial and fourth-degree rotational symmetric phase-modulation. The analysis shows how optimal imaging performance is obtained using moderate phase-modulation, and more importantly, the relative merits of the above functions.

An open-source deconvolution software package for 3-D quantitative fluorescence microscopy imaging

Deconvolution techniques have been widely used for restoring the 3-D quantitative information of an unknown specimen observed using a wide-field fluorescence microscope. Deconv, an open-source deconvolution software package, was developed for 3-D quantitative fluorescence microscopy imaging and was released under the GNU Public License. Deconv provides numerical routines for simulation of a 3-D point spread function and deconvolution routines implemented three constrained iterative deconvolution algorithms: one based on a Poisson noise model and two others based on a Gaussian noise model. These algorithms are presented and evaluated using synthetic images and experimentally obtained microscope images, and the use of the library is explained. Deconv allows users to assess the utility of these deconvolution algorithms and to determine which are suited for a particular imaging application. The design of Deconv makes it easy for deconvolution capabilities to be incorporated into existing imaging applications.

Blind and semi-blind deblurring of natural images

A method for blind image deblurring is presented. The method only makes weak assumptions about the blurring filter and is able to undo a wide variety of blurring degradations. To overcome the ill-posedness of the blind image deblurring problem, the method includes a learning technique which initially focuses on the main edges of the image and gradually takes details into account. A new image prior, which includes a new edge detector, is used. The method is able to handle unconstrained blurs, but also allows the use of constraints or of prior information on the blurring filter, as well as the use of filters defined in a parametric manner. Furthermore, it works in both single-frame and multiframe scenarios. The use of constrained blur models appropriate to the problem at hand, and/or of multiframe scenarios, generally improves the deblurring results. Tests performed on monochrome and color images, with various synthetic and real-life degradations, without and with noise, in single-frame and multiframe scenarios, showed good results, both in subjective terms and in terms of the increase of signal to noise ratio (ISNR) measure. In comparisons with other state of the art methods, our method yields better results, and shows to be applicable to a much wider range of blurs.

Automatic deconvolution of 4Pi-microscopy data with arbitrary phase

We propose a maximum a posteriori-based method that solves an important practical problem in the deconvolution of 4Pi images by simultaneously delivering an estimate of both the object and the unknown phase. The method was tested in simulations and on data from both test samples and biological specimen. It generates object estimates that are free from interference artifacts and reliably recovers arbitrary relative phases. Based on vectorial focusing theory, our theoretical analysis allowed for a simple and efficient implementation of the algorithm. Taking several 4Pi images at different relative phases of the interfering beams is shown to improve the robustness of the approach.

Blind deconvolution for thin-layered confocal imaging

We propose an alternate minimization algorithm for estimating the point-spread function (PSF) of a confocal laser scanning microscope and the specimen fluorescence distribution. A three-dimensional separable Gaussian model is used to restrict the PSF solution space and a constraint on the specimen is used so as to favor the stabilization and convergence of the algorithm. The results obtained from the simulation show that the PSF can be estimated to a high degree of accuracy, and those on real data show better deconvolution as compared to a full theoretical PSF model.

Depth-variant maximum-likelihood restoration for three-dimensional fluorescence microscopy

We derive an algorithm for maximum-likelihood image estimation on the basis of the expectation-maximization (EM) formalism by using a new approximate model for depth-varying image formation for optical sectioning microscopy. This new strata-based model incorporates spherical aberration that worsens as the microscope is focused deeper under the cover slip and is the result of the refractive-index mismatch between the immersion medium and the mounting medium of the specimen. Images of a specimen with known geometry and refractive index show that the model captures the main features of the image. We analyze the performance of the depth-variant EM algorithm with simulations, which show that the algorithm can compensate for image degradation changing with depth.

Numerical evaluation of Hankel transforms for oscillating functions

Six methods for the numerical calculation of zero-order Hankel transforms of oscillating functions were evaluated. One method based on Filon quadrature philosophy, two published projection-slice methods, and a third projection-slice method based on a new approach to computation of the Abel transform were implemented; alternative versions of two of the projection-slice methods were derived for more accurate approximations in the projection step. These six algorithms were tested with an oscillating sweep signal and with the calculation of a three-dimensional diffraction-limited point-spread function of a fluorescence microscope. We found that the Filon quadrature method is highly accurate but also computationally demanding. The projection-slice methods, in particular the new one that we derived, offer an excellent compromise between accuracy and computational efficiency.

Relaxed ordered-subset algorithm for penalized-likelihood image restoration

The expectation-maximization (EM) algorithm for maximum-likelihood image recovery is guaranteed to converge, but it converges slowly. Its ordered-subset version (OS-EM) is used widely in tomographic image reconstruction because of its order-of-magnitude acceleration compared with the EM algorithm, but it does not guarantee convergence. Recently the ordered-subset, separable-paraboloidal-surrogate (OS-SPS) algorithm with relaxation has been shown to converge to the optimal point while providing fast convergence. We adapt the relaxed OS-SPS algorithm to the problem of image restoration. Because data acquisition in image restoration is different from that in tomography, we employ a different strategy for choosing subsets, using pixel locations rather than projection angles. Simulation results show that the relaxed OS-SPS algorithm can provide an order-of-magnitude acceleration over the EM algorithm for image restoration. This new algorithm now provides the speed and guaranteed convergence necessary for efficient image restoration.

Image recovery using partitioned-separable paraboloidal surrogate coordinate ascent algorithms

Iterative coordinate ascent algorithms have been shown to be useful for image recovery, but are poorly suited to parallel computing due to their sequential nature. This paper presents a new fast converging parallelizable algorithm for image recovery that can be applied to a very broad class of objective functions. This method is based on paraboloidal surrogate functions and a concavity technique. The paraboloidal surrogates simplify the optimization problem. The idea of the concavity technique is to partition pixels into subsets that can be updated in parallel to reduce the computation time. For fast convergence, pixels within each subset are updated sequentially using a coordinate ascent algorithm. The proposed algorithm is guaranteed to monotonically increase the objective function and intrinsically accommodates nonnegativity constraints. A global convergence proof is summarized. Simulation results show that the proposed algorithm requires less elapsed time for convergence than iterative coordinate ascent algorithms. With four parallel processors, the proposed algorithm yields a speedup factor of 3.77 relative to single processor coordinate ascent algorithms for a three-dimensional (3-D) confocal image restoration problem.

Iterative statistical approach to blind image deconvolution

Image deblurring has long been modeled as a deconvolution problem. In the literature, the point-spread function (PSF) is often assumed to be known exactly. However, in practical situations such as image acquisition in cameras, we may have incomplete knowledge of the PSF. This deblurring problem is referred to as blind deconvolution. We employ a statistical point of view of the data and use a modified maximum a posteriori approach to identify the most probable object and blur given the observed image. To facilitate computation we use an iterative method, which is an extension of the traditional expectation-maximization method, instead of direct optimization. We derive separate formulas for the updates of the estimates in each iteration to enhance the deconvolution results, which are based on the specific nature of our a priori knowledge available about the object and the blur.