Super-resolution for asymmetric resolution of FIB-SEM 3D imaging using AI with deep learning

The Applications of Artificial Intelligence (AI)-Driven Tools in Virus-Like Particles (VLPs) Research

Viral-like particles (VLPs) represent versatile nanoscale structures mimicking the morphology and antigenic characteristics of viruses, devoid of genetic material, making them promising candidates for various biomedical applications. The integration of artificial intelligence (AI) into VLP research has catalyzed significant advancements in understanding, production, and therapeutic applications of these nanostructures. This comprehensive review explores the collaborative utilization of AI tools, computational methodologies, and state-of-the-art technologies within the VLP domain. AI's involvement in bioinformatics facilitates sequencing and structure prediction, unraveling genetic intricacies and three-dimensional configurations of VLPs. Furthermore, AI-enabled drug discovery enables virtual screening, demonstrating promise in identifying compounds to inhibit VLP activity. In VLP production, AI optimizes processes by providing strategies for culture conditions, nutrient concentrations, and growth kinetics. AI's utilization in image analysis and electron microscopy expedites VLP recognition and quantification. Moreover, network analysis of protein-protein interactions through AI tools offers an understanding of VLP interactions. The integration of multi-omics data via AI analytics provides a comprehensive view of VLP behavior. Predictive modeling utilizing machine learning algorithms aids in forecasting VLP stability, guiding optimization efforts. Literature mining facilitated by text mining algorithms assists in summarizing information from the VLP knowledge corpus. Additionally, AI's role in laboratory automation enhances experimental efficiency. Addressing data security concerns, AI ensures the protection of sensitive information in the digital era of VLP research. This review serves as a roadmap, providing insights into AI's current and future applications in VLP research, thereby guiding innovative directions in medicine and beyond.

The application and development of electron microscopy for three-dimensional reconstruction in life science: a review

Imaging technologies have played a pivotal role in advancing biological research by enabling visualization of biological structures and processes. While traditional electron microscopy (EM) produces two-dimensional images, emerging techniques now allow high-resolution three-dimensional (3D) characterization of specimens in situ, meeting growing needs in molecular and cellular biology. Combining transmission electron microscopy (TEM) with serial sectioning inaugurated 3D imaging, attracting biologists seeking to explore cell ultrastructure and driving advancement of 3D EM reconstruction. By comprehensively and precisely rendering internal structure and distribution, 3D TEM reconstruction provides unparalleled ultrastructural insights into cells and molecules, holding tremendous value for elucidating structure-function relationships and broadly propelling structural biology. Here, we first introduce the principle of 3D reconstruction of cells and tissues by classical approaches in TEM and then discuss modern technologies utilizing TEM and on new SEM-based as well as cryo-electron microscope (cryo-EM) techniques. 3D reconstruction techniques from serial sections, electron tomography (ET), and the recent single-particle analysis (SPA) are examined; the focused ion beam scanning electron microscopy (FIB-SEM), the serial block-face scanning electron microscopy (SBF-SEM), and automatic tape-collecting lathe ultramicrotome (ATUM-SEM) for 3D reconstruction of large volumes are discussed. Finally, we review the challenges and development prospects of these technologies in life science. It aims to provide an informative reference for biological researchers.

Revealing the 3D Structure of Block Copolymers with Electron Microscopy: Current Status and Future Directions

Block copolymers (BCPs) are considered model systems for understanding and utilizing self-assembly in soft matter. Their tunable nanometric structure and composition enable comprehensive studies of self-assembly processes as well as make them relevant materials in diverse applications. A key step in developing and controlling BCP nanostructures is a full understanding of their three-dimensional (3D) structure and how this structure is affected by the BCP chemistry, confinement, boundary conditions, and the self-assembly evolution and dynamics. Electron microscopy (EM) is a leading method in BCP 3D characterization owing to its high resolution in imaging nanosized structures. Here we discuss the two main 3D EM methods: namely, transmission EM tomography and slice and view scanning EM tomography. We present each method's principles, examine their strengths and weaknesses, and discuss ways researchers have devised to overcome some of the challenges in BCP 3D characterization with EM- from specimen preparation to imaging radiation-sensitive materials. Importantly, we review current and new cutting-edge EM methods such as direct electron detectors, energy dispersive X-ray spectroscopy of soft matter, high temporal rate imaging, and single-particle analysis that have great potential for expanding the BCP understanding through EM in the future.

Quantitative 3D Characterization of Functionally Relevant Parameters in Heavy-Oxide-Supported 4d Metal Nanocatalysts

Accurate 3D nanometrology of catalysts with small nanometer-sized particles of light 3d or 4d metals supported on high-atomic-number oxides is crucial for understanding their functionality. However, performing quantitative 3D electron tomography analysis on systems involving metals like Pd, Ru, or Rh supported on heavy oxides (e.g., CeO2) poses significant challenges. The low atomic number (Z) of the metal complicates discrimination, especially for very small nanoparticles (1-3 nm). Conventional reconstruction methods successful for catalysts with 5d metals (e.g., Au, Pt, or Ir) fail to detect 4d metal particles in electron tomography reconstructions, as their contrasts cannot be effectively separated from those of the underlying support crystallites. To address this complex 3D characterization challenge, we have developed a full deep learning (DL) pipeline that combines multiple neural networks, each one optimized for a specific image-processing task. In particular, single-image super-resolution (SR) techniques are used to intelligently denoise and enhance the quality of the tomographic tilt series. U-net generative adversarial network algorithms are employed for image restoration and correcting alignment-related artifacts in the tilt series. Finally, semantic segmentation, utilizing a U-net-based convolutional neural network, splits the 3D volumes into their components (metal and support). This approach enables the visualization of subnanometer-sized 4d metal particles and allows for the quantitative extraction of catalytically relevant structural information, such as particle size, sphericity, and truncation, from compressed sensing electron tomography volume reconstructions. We demonstrate the potential of this approach by characterizing nanoparticles of a metal widely used in catalysis, Pd (Z = 46), supported on CeO2, a very high density (7.22 g/cm3) oxide involving a quite high-atomic-number element, Ce (Z = 58).

Deep Learning to Analyze Sliding Drops

State-of-the-art contact angle measurements usually involve image analysis of sessile drops. The drops are symmetric and images can be taken at high resolution. The analysis of videos of drops sliding down a tilted plate is hampered due to the low resolution of the cutout area where the drop is visible. The challenge is to analyze all video images automatically, while the drops are not symmetric anymore and contact angles change while sliding down the tilted plate. To increase the accuracy of contact angles, we present a 4-segment super-resolution optimized-fitting (4S-SROF) method. We developed a deep learning-based super-resolution model with an upscale ratio of 3; i.e., the trained model is able to enlarge drop images 9 times accurately (PSNR = 36.39). In addition, a systematic experiment using synthetic images was conducted to determine the best parameters for polynomial fitting of contact angles. Our method improved the accuracy by 21% for contact angles lower than 90° and by 33% for contact angles higher than 90°.

Digital Innovation Enabled Nanomaterial Manufacturing; Machine Learning Strategies and Green Perspectives

Machine learning has been an emerging scientific field serving the modern multidisciplinary needs in the Materials Science and Manufacturing sector. The taxonomy and mapping of nanomaterial properties based on data analytics is going to ensure safe and green manufacturing with consciousness raised on effective resource management. The utilization of predictive modelling tools empowered with artificial intelligence (AI) has proposed novel paths in materials discovery and optimization, while it can further stimulate the cutting-edge and data-driven design of a tailored behavioral profile of nanomaterials to serve the special needs of application environments. The previous knowledge of the physics and mathematical representation of material behaviors, as well as the utilization of already generated testing data, received specific attention by scientists. However, the exploration of available information is not always manageable, and machine intelligence can efficiently (computational resources, time) meet this challenge via high-throughput multidimensional search exploration capabilities. Moreover, the modelling of bio-chemical interactions with the environment and living organisms has been demonstrated to connect chemical structure with acute or tolerable effects upon exposure. Thus, in this review, a summary of recent computational developments is provided with the aim to cover excelling research and present challenges towards unbiased, decentralized, and data-driven decision-making, in relation to increased impact in the field of advanced nanomaterials manufacturing and nanoinformatics, and to indicate the steps required to realize rapid, safe, and circular-by-design nanomaterials.

Applications of scanning electron microscopy and focused ion beam milling in dental research

The abilities of scanning electron microscopy (SEM) and focused ion beam (FIB) milling for obtaining high-resolution images from top surfaces, cross-sectional surfaces, and even in three dimensions, are becoming increasingly important for imaging and analyzing tooth structures such as enamel and dentin. FIB was originally developed for material research in the semiconductor industry. However, use of SEM/FIB has been growing recently in dental research due to the versatility of dual platform instruments that can be used as a milling device to obtain low-artifact cross-sections of samples combined with high-resolution images. The advent of the SEM/FIB system and accessories may offer access to previously inaccessible length scales for characterizing tooth structures for dental research, opening exciting opportunities to address many central questions in dental research. New discoveries and fundamental breakthroughs in understanding are likely to follow. This review covers the applications, key findings, and future direction of SEM/FIB in dental research in morphology imaging, specimen preparation for transmission electron microscopy (TEM) analysis, and three-dimensional volume imaging using SEM/FIB tomography.

An Adversarial Learning Approach for Super-Resolution Enhancement Based on AgCl@Ag Nanoparticles in Scanning Electron Microscopy Images

Scanning electron microscopy (SEM) plays a crucial role in the characterization of nanoparticles. Unfortunately, due to the limited resolution, existing imaging techniques are insufficient to display all detailed characteristics at the nanoscale. Hardware-oriented techniques are troubled with costs and material properties. Computational approaches often prefer blurry results or produce a less meaningful high-frequency noise. Therefore, we present a staged loss-driven neural networks model architecture to transform low-resolution SEM images into super-resolved ones. Our approach consists of two stages: first, residual channel attention network (RCAN) with mean absolute error (MAE) loss was used to get a better peak signal-to-noise ratio (PSNR). Then, discriminators with adversarial losses were activated to reconstruct high-frequency texture features. The quantitative and qualitative evaluation results indicate that compared with other advanced approaches, our model achieves satisfactory results. The experiment in AgCl@Ag for photocatalytic degradation confirms that our proposed method can bring realistic high-frequency structural detailed information rather than meaningless noise. With this approach, high-resolution SEM images can be acquired immediately without sample damage. Moreover, it provides an enhanced characterization method for further directing the preparation of nanoparticles.

Electron Microscopy Techniques for Investigating Structure and Composition of Hair-Cell Stereociliary Bundles

Hair cells—the sensory cells of the vertebrate inner ear—bear at their apical surfaces a bundle of actin-filled protrusions called stereocilia, which mediate the cells’ mechanosensitivity. Hereditary deafness is often associated with morphological disorganization of stereocilia bundles, with the absence or mislocalization within stereocilia of specific proteins. Thus, stereocilia bundles are closely examined to understand most animal models of hereditary hearing loss. Because stereocilia have a diameter less than a wavelength of light, light microscopy is not adequate to reveal subtle changes in morphology or protein localization. Instead, electron microscopy (EM) has proven essential for understanding stereocilia bundle development, maintenance, normal function, and dysfunction in disease. Here we review a set of EM imaging techniques commonly used to study stereocilia, including optimal sample preparation and best imaging practices. These include conventional and immunogold transmission electron microscopy (TEM) and scanning electron microscopy (SEM), as well as focused-ion-beam scanning electron microscopy (FIB-SEM), which enables 3-D serial reconstruction of resin-embedded biological structures at a resolution of a few nanometers. Parameters for optimal sample preparation, fixation, immunogold labeling, metal coating and imaging are discussed. Special attention is given to protein localization in stereocilia using immunogold labeling. Finally, we describe the advantages and limitations of these EM techniques and their suitability for different types of studies.

Hierarchical ultrastructure: An overview of what is known about tendons and future perspective for tendon engineering

Abnormal tendons are rarely ever repaired to the natural structure and morphology of normal tendons. To better guide the repair and regeneration of injured tendons through a tissue engineering method, it is necessary to have insights into the internal morphology, organization, and composition of natural tendons. This review summarized recent researches on the structure and function of the extracellular matrix (ECM) components of tendons and highlight the application of multiple detection methodologies concerning the structure of ECMs. In addition, we look forward to the future of multi-dimensional biomaterial design methods and the potential of structural repair for tendon ECM components. In addition, focus is placed on the macro to micro detection methods for tendons, and current techniques for evaluating the extracellular matrix of tendons at the micro level are introduced in detail. Finally, emphasis is given to future extracellular matrix detection methods, as well as to how future efforts could concentrate on fabricating the biomimetic tendons.

•

Summarize recent research on the structure and function of the extracellular matrix (ECM) components of tendons.

•

Comments on current research methods concerning the structure of ECMs.

•

Perspective on the future of multi-dimensional detection techniques and structural repair of tendon ECM components.

A Close Look at Molecular Self-Assembly with the Transmission Electron Microscope

Molecular self-assembly is pervasive in the formation of living and synthetic materials. Knowledge gained from research into the principles of molecular self-assembly drives innovation in the biological, chemical, and materials sciences. Self-assembly processes span a wide range of temporal and spatial domains and are often unintuitive and complex. Studying such complex processes requires an arsenal of analytical and computational tools. Within this arsenal, the transmission electron microscope stands out for its unique ability to visualize and quantify self-assembly structures and processes. This review describes the contribution that the transmission electron microscope has made to the field of molecular self-assembly. An emphasis is placed on which TEM methods are applicable to different structures and processes and how TEM can be used in combination with other experimental or computational methods. Finally, we provide an outlook on the current challenges to, and opportunities for, increasing the impact that the transmission electron microscope can have on molecular self-assembly.

Deep learning-based estimation of Flory-Huggins parameter of A-B block copolymers from cross-sectional images of phase-separated structures

In this study, deep learning (DL)-based estimation of the Flory-Huggins χ parameter of A-B diblock copolymers from two-dimensional cross-sectional images of three-dimensional (3D) phase-separated structures were investigated. 3D structures with random networks of phase-separated domains were generated from real-space self-consistent field simulations in the 25-40 χN range for chain lengths (N) of 20 and 40. To confirm that the prepared data can be discriminated using DL, image classification was performed using the VGG-16 network. We comprehensively investigated the performances of the learned networks in the regression problem. The generalization ability was evaluated from independent images with the unlearned χN. We found that, except for large χN values, the standard deviation values were approximately 0.1 and 0.5 for A-component fractions of 0.2 and 0.35, respectively. The images for larger χN values were more difficult to distinguish. In addition, the learning performances for the 4-class problem were comparable to those for the 8-class problem, except when the χN values were large. This information is useful for the analysis of real experimental image data, where the variation of samples is limited.

Vertical distance from shading in the SEM

Vertical data collected by Scanning Electron Microscopy (SEM) are important for sample characterization, 3D reconstruction, and flex manipulation. Traditional methods are limited by the extent to which the probe obstructs the view of the sample along the vertical axis. Herein, we propose a novel SEM microprobe for measuring the vertical distance between the probe and substrate. To form a semi-transparent hole that is set as the objective regions in processing of the SEM images, an epoxy film was embedded in the through-hole at the tip of the microforce probe with 3D printing. The film can be modified with a focused ion beam (FIB) system. The motion of the modified probe along the vertical axis is controlled by a nanopositioner and the process is recorded by taking a real-time SEM video. The change in gray contrast caused by the semi-transparent epoxy is corrected during the SEM image processing of the video. By comparing the gray contrast with the nanopositioner motion data, we find that the change in gray contrast can provide feedback for adjusting the displacement between the probe and the substrate, and the resolution can be up to 100 nm. We propose a novel and simple method for measuring vertical distances in the SEM, which is useful for in-situ measurements and nanomanipulations.

High-Speed Piezoresponse Force Microscopy and Machine Learning Approaches for Dynamic Domain Growth in Ferroelectric Materials

Domain dynamics has been one of the hottest research topics for ferroelectric materials in order to understand the ferroelectric mechanisms and to develop the related applications. By using high-speed piezoresponse force microscopy (HSPFM), it is possible to observe the dynamic domain evolution in an ultrashort time increment. This paper combines the HSPFM experiments and machine learning to study the domain growth under a weak AC field in ferroelectric materials. Here, the Bayesian optimized support vector machine is employed to classify the switching domain and stable domain. The results indicate that the machine learning classifier is capable of discerning the switching area. In addition, the domain associated characteristics, such as domain pinning and domain wall pinning, can also be observed and analyzed by combining experiments and machine learning. The machine learning approach can fast and deeply extract the complicated features related to free energy from the multidimensional signals obtained by HSPFM.

Enhancing images of shale formations by a hybrid stochastic and deep learning algorithm

Accounting for the morphology of shale formations, which represent highly heterogeneous porous media, is a difficult problem. Although two- or three-dimensional images of such formations may be obtained and analyzed, they either do not capture the nanoscale features of the porous media, or they are too small to be an accurate representative of the media, or both. Increasing the resolution of such images is also costly. While high-resolution images may be used to train a deep-learning network in order to increase the quality of low-resolution images, an important obstacle is the lack of a large number of images for the training, as the accuracy of the network's predictions depends on the extent of the training data. Generating a large number of high-resolution images by experimental means is, however, very time consuming and costly, hence limiting the application of deep-learning algorithms to such an important class of problems. To address the issue we propose a novel hybrid algorithm by which a stochastic reconstruction method is used to generate a large number of plausible images of a shale formation, using very few input images at very low cost, and then train a deep-learning convolutional network by the stochastic realizations. We refer to the method as hybrid stochastic deep-learning (HSDL) algorithm. The results indicate promising improvement in the quality of the images, the accuracy of which is confirmed by visual, as well as quantitative comparison between several of their statistical properties. The results are also compared with those obtained by the regular deep learning algorithm without using an enriched and large dataset for training, as well as with those generated by bicubic interpolation.

Study of Commodity VR for Computational Material Sciences

Recent advancements in virtual reality (VR) devices and software environments make it possible to easily incorporate this technology for many applications, including computational materials science. For studying three-dimensional (3D) structure models and related chemical information, we focused on using a commodity VR device (VIVE) and an authoring tool (Unity). To visualize 3D chemical structures, disturbances like judder due to dropped frames should be eliminated from the VR experience to improve simulations. We propose a simple evaluation method that is straightforward for the nonexpert or novice VR user. We examine the major visualization representations including ball, ball and stick, and isosurface systems. For systematic benchmark measurements, a pendulum from the VR device was used to generate periodic oscillatory motion during measurements of a time series in frames per second (fps). For VIVE with a refresh rate of 90 Hz, judder occurred when less than 90 fps. We demonstrated the system size limitations for the results of molecular dynamics simulations of phase separation of ABA block copolymers and experimental observations of filler morphologies in rubber.