Evaluation of noise and blur effects with SIRT-FISTA-TV reconstruction algorithm: Application to fast environmental transmission electron tomography

Autonomous Electron Tomography Reconstruction with Machine Learning

Modern electron tomography has progressed to higher resolution at lower doses by leveraging compressed sensing (CS) methods that minimize total variation (TV). However, these sparsity-emphasized reconstruction algorithms introduce tunable parameters that greatly influence the reconstruction quality. Here, Pareto front analysis shows that high-quality tomograms are reproducibly achieved when TV minimization is heavily weighted. However, in excess, CS tomography creates overly smoothed three-dimensional (3D) reconstructions. Adding momentum to the gradient descent during reconstruction reduces the risk of over-smoothing and better ensures that CS is well behaved. For simulated data, the tedious process of tomography parameter selection is efficiently solved using Bayesian optimization with Gaussian processes. In combination, Bayesian optimization with momentum-based CS greatly reduces the required compute time-an 80% reduction was observed for the 3D reconstruction of SrTiO3 nanocubes. Automated parameter selection is necessary for large-scale tomographic simulations that enable the 3D characterization of a wider range of inorganic and biological materials.

Machine learning in electron microscopy for advanced nanocharacterization: current developments, available tools and future outlook

In the last few years, electron microscopy has experienced a new methodological paradigm aimed to fix the bottlenecks and overcome the challenges of its analytical workflow. Machine learning and artificial intelligence are answering this call providing powerful resources towards automation, exploration, and development. In this review, we evaluate the state-of-the-art of machine learning applied to electron microscopy (and obliquely, to materials and nano-sciences). We start from the traditional imaging techniques to reach the newest higher-dimensionality ones, also covering the recent advances in spectroscopy and tomography. Additionally, the present review provides a practical guide for microscopists, and in general for material scientists, but not necessarily advanced machine learning practitioners, to straightforwardly apply the offered set of tools to their own research. To conclude, we explore the state-of-the-art of other disciplines with a broader experience in applying artificial intelligence methods to their research (e.g., high-energy physics, astronomy, Earth sciences, and even robotics, videogames, or marketing and finances), in order to narrow down the incoming future of electron microscopy, its challenges and outlook.

Iterative tomographic reconstruction with TV prior for low-dose CBCT dental imaging

Objective.Cone-beam computed tomography is becoming more and more popular in applications such as 3D dental imaging. Iterative methods compared to the standard Feldkamp algorithm have shown improvements in image quality of reconstruction of low-dose acquired data despite their long computing time. An interesting aspect of iterative methods is their ability to include prior information such as sparsity-constraint. While a large panel of optimization algorithms along with their adaptation to tomographic problems are available, they are mainly studied on 2D parallel or fan-beam data. The issues raised by 3D CBCT and moreover by truncated projections are still poorly understood.Approach.We compare different carefully designed optimization schemes in the context of realistic 3D dental imaging. Besides some known algorithms, SIRT-TV and MLEM, we investigate the primal-dual hybrid gradient (PDHG) approach and a newly proposed MLEM-TV optimizer. The last one is alternating EM steps and TV-denoising, combination not yet investigated for CBCT. Experiments are performed on both simulated data from a 3D jaw phantom and data acquired with a dental clinical scanner.Main results.With some adaptations to the specificities of CBCT operators, PDHG and MLEM-TV algorithms provide the best reconstruction quality. These results were obtained by comparing the full-dose image with a low-dose image and an ultra low-dose image.Significance.The convergence speed of the original iterative methods is hampered by the conical geometry and significantly reduced compared to parallel geometries. We promote the pre-conditioned version of PDHG and we propose a pre-conditioned version of the MLEM-TV algorithm. To the best of our knowledge, this is the first time PDHG and convergent MLEM-TV algorithms are evaluated on experimental dental CBCT data, where constraints such as projection truncation and presence of metal have to be jointly overcome.

Noise-robust phase-space deconvolution for light-field microscopy

SIGNIFICANCE

Light-field microscopy has achieved success in various applications of life sciences that require high-speed volumetric imaging. However, existing light-field reconstruction algorithms degrade severely in low-light conditions, and the deconvolution process is time-consuming.

AIM

This study aims to develop a noise robustness phase-space deconvolution method with low computational costs.

APPROACH

We reformulate the light-field phase-space deconvolution model into the Fourier domain with random-subset ordering and total-variation (TV) regularization. Additionally, we build a time-division-based multicolor light-field microscopy and conduct the three-dimensional (3D) imaging of the heart beating in zebrafish larva at over 95 Hz with a low light dose.

RESULTS

We demonstrate that this approach reduces computational resources, brings a tenfold speedup, and achieves a tenfold improvement for the noise robustness in terms of SSIM over the state-of-the-art approach.

CONCLUSIONS

We proposed a phase-space deconvolution algorithm for 3D reconstructions in fluorescence imaging. Compared with the state-of-the-art method, we show significant improvement in both computational effectiveness and noise robustness; we further demonstrated practical application on zebrafish larva with low exposure and low light dose.

Deep Learning-Based Image Reconstruction for Different Medical Imaging Modalities

Image reconstruction in magnetic resonance imaging (MRI) and computed tomography (CT) is a mathematical process that generates images at many different angles around the patient. Image reconstruction has a fundamental impact on image quality. In recent years, the literature has focused on deep learning and its applications in medical imaging, particularly image reconstruction. Due to the performance of deep learning models in a wide variety of vision applications, a considerable amount of work has recently been carried out using image reconstruction in medical images. MRI and CT appear as the ultimate scientifically appropriate imaging mode for identifying and diagnosing different diseases in this ascension age of technology. This study demonstrates a number of deep learning image reconstruction approaches and a comprehensive review of the most widely used different databases. We also give the challenges and promising future directions for medical image reconstruction.

Impact of the training loss in deep learning based CT reconstruction of bone microarchitecture

PURPOSE

Computed tomography (CT) is a technique of choice to image bone structure at different scales. Methods to enhance the quality of degraded reconstructions obtained from low-dose CT data have shown impressive results recently, especially in the realm of supervised deep learning. As the choice of the loss function affects the reconstruction quality, it is necessary to focus on the way neural networks evaluate the correspondence between predicted and target images during the training stage. This is even more true in the case of bone microarchitecture imaging at high spatial resolution where both the quantitative analysis of Bone Mineral Density (BMD) and bone microstructure are essential for assessing diseases such as osteoporosis. Our aim is thus to evaluate the quality of reconstruction on key metrics for diagnosis depending on the loss function that has been used for training the neural network.

METHODS

We compare and analyze volumes that are reconstructed with neural networks trained with pixelwise, structural and adversarial loss functions or with a combination of them. We perform realistic simulations of various low-dose acquisitions of bone microarchitecture. Our comparative study is performed with metrics that have an interest regarding the diagnosis of bone diseases. We therefore focus on bone-specific metrics such as BV/TV, resolution, connectivity assessed with the Euler number and quantitative analysis of BMD to evaluate the quality of reconstruction obtained with networks trained with the different loss functions.

RESULTS

We find that using L₁ norm as the pixelwise loss is the best choice compared to L₂ or no pixelwise loss since it improves resolution without deteriorating other metrics. VGG perceptual loss, especially when combined with an adversarial loss, allows to better retrieve topological and morphological parameters of bone microarchitecture compared to SSIM. This however leads to a decreased resolution performance. The adversarial loss enchances the reconstruction performance in terms of BMD distribution accuracy.

CONCLUSIONS

In order to retrieve the quantitative and structural characteristics of bone microarchitecture that are essential for post-reconstruction diagnosis, our results suggest to use L₁ norm as part of the loss function. Then, trade-offs should be made depending on the application: VGG perceptual loss improves accuracy in terms of connectivity at the cost of a deteriorated resolution, and adversarial losses help better retrieve BMD distribution while significantly increasing the training time. This article is protected by copyright. All rights reserved.

Morphology and topology assessment in hierarchical zeolite materials: adsorption hysteresis, scanning behavior, and domain theory

Using a prototypical family of hierarchical zeolites, we show how adsorption-based characterization can be extended to provide morphological and topological assessment beyond state-of-the-art tools. The well-controlled materials under study consist...

An improved deep learning-based algorithm for 3D reconstruction of vacuum arcs

Extensive attempts have been made to enable the application of deep learning to 3D plasma reconstruction. However, due to the limitation on the number of available training samples, deep learning-based methods have insufficient generalization ability compared to the traditional iterative methods. This paper proposes an improved algorithm named convolutional neural network-maximum likelihood expectation maximization-split-Bergman (CNN-MLEM-SB) based on the combination of the deep learning CNN and an iterative algorithm known as MLEM-SB. This method uses the prediction result of a CNN as the initial value and then corrects it using the MLEM-SB to obtain the final results. The proposed method is verified experimentally by reconstructing two types of vacuum arcs with and without transverse magnetic field (TMF) control. In addition, the CNN and the proposed algorithm are compared with respect to accuracy and generalization ability. The results show that the CNN can effectively reconstruct the arcs between a pair of disk contacts, which has specific distribution patterns: its structural similarity index measurement (SSIM) can reach 0.952. However, the SSIM decreases to 0.868 for the arc between a pair of TMF contacts, which is controlled by the TMF and has complex distribution patterns. Compared with the CNN reconstruction method, the proposed algorithm can achieve a higher reconstruction accuracy for any arc shape. Compared with the iterative algorithm, the proposed algorithm's reconstruction efficiency is higher by 38.24% and 35.36% for the vacuum arc between the disk and the TMF contacts, respectively.

3D reconstruction of dynamic behaviors of vacuum arcs under transverse magnetic fields via computer tomography

The transverse magnetic field (TMF) contacts make the vacuum arcs deviate from the axisymmetric structure, so complete spatiotemporal evolution information of the plasma cannot be obtained by adopting one- or two-dimensional (2D) diagnostic methods. To address the issues, computer tomography was introduced in this paper. First, a multi-angle diagnostic imaging system based on split fiber bundles was proposed, which used a high-speed camera to simultaneously acquire eight angles of the arc image over time. In addition, a tomography algorithm called the maximum likelihood expectation maximum with Split Bregman denoising was proposed to reconstruct the dynamic spatiotemporal characteristics of the arc under complex conditions. Then, the three-dimensional (3D) distribution of Cu i and Cr i particles inside the contact gap was obtained by adopting optical filters. The 3D distribution of the vacuum arc had shown an obvious asymmetrical pattern under the TMF contacts, and there was a ring-like aggregation zone inside the arc, which can cause severe ablation on the anode contacts. According to the reconstructed 3D distribution of Cu i and Cr i, it is found that the metal vapor was mainly concentrated near the electrode surface and showed a clear distribution of non-uniform aggregates, while the concentration of particles in the gap was low. Moreover, on the cathode surface, the cathode spots moved in the form of groups driven by the TMF, while the anode surface was ablated by the electric arc, and the metal vapor existed in the form of bands.

Dynamic compressed sensing for real-time tomographic reconstruction

Electron tomography has achieved higher resolution and quality at reduced doses with recent advances in compressed sensing. Compressed sensing (CS) exploits the inherent sparse signal structure to efficiently reconstruct three-dimensional (3D) volumes at the nanoscale from undersampled measurements. However, the process bottlenecks 3D reconstruction with computation times that run from hours to days. Here we demonstrate a framework for dynamic compressed sensing that produces a 3D specimen structure that updates in real-time as new specimen projections are collected. Researchers can begin interpreting 3D specimens as data is collected to facilitate high-throughput and interactive analysis. Using scanning transmission electron microscopy (STEM), we show that dynamic compressed sensing accelerates the convergence speed by ~3-fold while also reducing its error by 27% for a Au/SrTiO₃ nanoparticle specimen. Before a tomography experiment is completed, the 3D tomogram has interpretable structure within ~33% of completion and fine details are visible as early as ~66%. Upon completion of an experiment, a high-fidelity 3D visualization is produced without further delay. Additionally, reconstruction parameters that tune data fidelity can be manipulated throughout the computation without re-running the entire process.

Electron tomography imaging methods with diffraction contrast for materials research

Transmission electron microscopy (TEM) and scanning transmission electron microscopy (STEM) enable the visualization of three-dimensional (3D) microstructures ranging from atomic to micrometer scales using 3D reconstruction techniques based on computed tomography algorithms. This 3D microscopy method is called electron tomography (ET) and has been utilized in the fields of materials science and engineering for more than two decades. Although atomic resolution is one of the current topics in ET research, the development and deployment of intermediate-resolution (non-atomic-resolution) ET imaging methods have garnered considerable attention from researchers. This research trend is probably not irrelevant due to the fact that the spatial resolution and functionality of 3D imaging methods of scanning electron microscopy (SEM) and X-ray microscopy have come to overlap with those of ET. In other words, there may be multiple ways to carry out 3D visualization using different microscopy methods for nanometer-scale objects in materials. From the above standpoint, this review paper aims to (i) describe the current status and issues of intermediate-resolution ET with regard to enhancing the effectiveness of TEM/STEM imaging and (ii) discuss promising applications of state-of-the-art intermediate-resolution ET for materials research with a particular focus on diffraction contrast ET for crystalline microstructures (superlattice domains and dislocations) including a demonstration of in situ dislocation tomography.

Dynamic Fuzzy Adjustment Algorithm for Web Information Acquisition and Data Transmission

In order to improve the transmission dynamic fuzzy adjustment ability of web information acquisition data, a dynamic fuzzy adjustment method of web information acquisition data transmission based on auto correlation feature matching is proposed. This paper constructs the key transfer protocol of the dynamic fuzzy adjustment of web information acquisition data transmission, uses a chaotic sequence structure reconstruction design method to carry out vector quantization and coding processing in the process of the dynamic fuzzy adjustment of web information acquisition data transmission, extracts nonlinear associated feature quantities of web information acquisition data, adopts a statistical feature detection method to select features in dynamic fuzzy adjustment process of web information acquisition data transmission, constructs a feature selection model of the dynamic fuzzy adjustment of web information acquisition data transmission, dynamically adjusts the fuzzy data with a fuzzy information clustering analysis method, and dynamically adjusts the fuzzy data transmission through fuzzy design and fuzzy encryption. The simulation results show that the dynamic fuzzy adjustment of web information acquisition data transmission when using this method is better, and the accurate transmission ability is stronger.