1
|
Eliasson H, Erni R. Localization and segmentation of atomic columns in supported nanoparticles for fast scanning transmission electron microscopy. NPJ COMPUTATIONAL MATERIALS 2024; 10:168. [PMID: 39104782 PMCID: PMC11297796 DOI: 10.1038/s41524-024-01360-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Accepted: 07/21/2024] [Indexed: 08/07/2024]
Abstract
To accurately capture the dynamic behavior of small nanoparticles in scanning transmission electron microscopy, high-quality data and advanced data processing is needed. The fast scan rate required to observe structural dynamics inherently leads to very noisy data where machine learning tools are essential for unbiased analysis. In this study, we develop a workflow based on two U-Net architectures to automatically localize and classify atomic columns at particle-support interfaces. The model is trained on non-physical image simulations, achieves sub-pixel localization precision, high classification accuracy, and generalizes well to experimental data. We test our model on both in situ and ex situ experimental time series recorded at 5 frames per second of small Pt nanoparticles supported on CeO2(111). The processed movies show sub-second dynamics of the nanoparticles and reveal site-specific movement patterns of individual atomic columns.
Collapse
Affiliation(s)
- Henrik Eliasson
- Electron Microscopy Center, Empa – Swiss Federal Laboratories for Materials Science and Technology, Überlandstrasse 129, 8600 Dübendorf, Switzerland
| | - Rolf Erni
- Electron Microscopy Center, Empa – Swiss Federal Laboratories for Materials Science and Technology, Überlandstrasse 129, 8600 Dübendorf, Switzerland
- Department of Materials, ETH Zürich, CH-8093 Zürich, Switzerland
| |
Collapse
|
2
|
Zhu Q, Zhao Q, Huang Q, Chen Y, Suresh S, Yang W, Zhang Z, Zhou H, Gao H, Wang J. Grain boundary plasticity initiated by excess volume. Proc Natl Acad Sci U S A 2024; 121:e2400161121. [PMID: 38478685 PMCID: PMC10963002 DOI: 10.1073/pnas.2400161121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Accepted: 02/13/2024] [Indexed: 03/27/2024] Open
Abstract
Grain boundaries (GBs) serve not only as strong barriers to dislocation motion, but also as important carriers to accommodate plastic deformation in crystalline solids. During deformation, the inherent excess volume associated with loose atomic packing in GBs brings about a microscopic degree of freedom that can initiate GB plasticity, which is beyond the classic geometric description of GBs. However, identification of this atomistic process has long remained elusive due to its transient nature. Here, we use Au polycrystals to unveil a general and inherent route to initiating GB plasticity via a transient topological transition process triggered by the excess volume. This route underscores the general impact of a microscopic degree of freedom which is governed by a stress-triaxiality-based criterion. Our findings provide a missing perspective for developing a more comprehensive understanding of the role of GBs in plastic deformation.
Collapse
Affiliation(s)
- Qi Zhu
- Center of Electron Microscopy, State Key Laboratory of Silicon and Advanced Semiconductor Materials, School of Materials Science and Engineering, Zhejiang University, Hangzhou310027, People’s Republic of China
- School of Mechanical and Aerospace Engineering, College of Engineering, Nanyang Technological University, Singapore639798, Singapore
| | - Qingkun Zhao
- Department of Engineering Mechanics, Center for X-Mechanics, Zhejiang University, Hangzhou310027, People’s Republic of China
| | - Qishan Huang
- Department of Engineering Mechanics, Center for X-Mechanics, Zhejiang University, Hangzhou310027, People’s Republic of China
| | - Yingbin Chen
- Center of Electron Microscopy, State Key Laboratory of Silicon and Advanced Semiconductor Materials, School of Materials Science and Engineering, Zhejiang University, Hangzhou310027, People’s Republic of China
| | - Subra Suresh
- School of Mechanical and Aerospace Engineering, College of Engineering, Nanyang Technological University, Singapore639798, Singapore
- Department of Materials Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA02139
| | - Wei Yang
- Department of Engineering Mechanics, Center for X-Mechanics, Zhejiang University, Hangzhou310027, People’s Republic of China
| | - Ze Zhang
- Center of Electron Microscopy, State Key Laboratory of Silicon and Advanced Semiconductor Materials, School of Materials Science and Engineering, Zhejiang University, Hangzhou310027, People’s Republic of China
| | - Haofei Zhou
- Department of Engineering Mechanics, Center for X-Mechanics, Zhejiang University, Hangzhou310027, People’s Republic of China
| | - Huajian Gao
- School of Mechanical and Aerospace Engineering, College of Engineering, Nanyang Technological University, Singapore639798, Singapore
- Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR), Singapore138632, Singapore
| | - Jiangwei Wang
- Center of Electron Microscopy, State Key Laboratory of Silicon and Advanced Semiconductor Materials, School of Materials Science and Engineering, Zhejiang University, Hangzhou310027, People’s Republic of China
- Wenzhou Key Laboratory of Novel Optoelectronic and Nano Materials, Institute of Wenzhou, Zhejiang University, Wenzhou325006, People’s Republic of China
| |
Collapse
|
3
|
Rossi K, Ruiz-Ferrando A, Akl DF, Abalos VG, Heras-Domingo J, Graux R, Hai X, Lu J, Garcia-Gasulla D, López N, Pérez-Ramírez J, Mitchell S. Quantitative Description of Metal Center Organization and Interactions in Single-Atom Catalysts. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2024; 36:e2307991. [PMID: 37757786 DOI: 10.1002/adma.202307991] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/08/2023] [Revised: 09/22/2023] [Indexed: 09/29/2023]
Abstract
Ultra-high-density single-atom catalysts (UHD-SACs) present unique opportunities for harnessing cooperative effects between neighboring metal centers. However, the lack of tools to establish correlations between the density, types, and arrangements of isolated metal atoms and the support surface properties hinders efforts to engineer advanced material architectures. Here, this work precisely describes the metal center organization in various mono- and multimetallic UHD-SACs based on nitrogen-doped carbon (NC) supports by coupling transmission electron microscopy with tailored machine-learning methods (released as a user-friendly web app) and density functional theory simulations. This approach quantifies the non-negligible presence of multimers with increasing atom density, characterizes the size and shape of these low-nuclearity clusters, and identifies surface atom density criteria to ensure isolation. Further, it provides previously inaccessible experimental insights into coordination site arrangements in the NC host, uncovering a repulsive interaction that influences the disordered distribution of metal centers in UHD-SACs. This observation holds in multimetallic systems, where chemically-specific analysis quantifies the degree of intermixing. These fundamental insights into the materials chemistry of single-atom catalysts are crucial for designing catalytic systems with superior reactivity.
Collapse
Affiliation(s)
- Kevin Rossi
- Institute for Chemical and Bioengineering, Department of Chemistry and Applied Biosciences, ETH Zurich, Vladimir-Prelog-Weg 1, Zurich, 8093, Switzerland
| | - Andrea Ruiz-Ferrando
- Institute of Chemical Research of Catalonia, Avenida Països Catalans 16, Tarragona, 43007, Spain
- Departament de Química Física i Inorgànica, Universitat Rovira i Virgili, Carrer de Marcellí Domingo 1, Tarragona, 43007, Spain
| | - Dario Faust Akl
- Institute for Chemical and Bioengineering, Department of Chemistry and Applied Biosciences, ETH Zurich, Vladimir-Prelog-Weg 1, Zurich, 8093, Switzerland
| | | | - Javier Heras-Domingo
- Institute of Chemical Research of Catalonia, Avenida Països Catalans 16, Tarragona, 43007, Spain
| | - Romain Graux
- Institute of Chemical Sciences and Engineering, Ecole Polytechnique Fédérale de Lausanne, Route Cantonale, Lausanne, 1015, Switzerland
| | - Xiao Hai
- Department of Chemistry, National University of Singapore, Science Drive 3, Singapore, 117543, Singapore
| | - Jiong Lu
- Department of Chemistry, National University of Singapore, Science Drive 3, Singapore, 117543, Singapore
- Centre for Advanced 2D Materials and Graphene Research Centre, National University of Singapore, Science Drive 2, Singapore, 117546, Singapore
- Institute for Functional Intelligent Materials, National University of Singapore, Science Drive 2, Singapore, 117544, Singapore
| | - Dario Garcia-Gasulla
- Barcelona Supercomputing Center, Plaça d'Eusebi Güell 1-3, Barcelona, 08034, Spain
| | - Nuria López
- Institute of Chemical Research of Catalonia, Avenida Països Catalans 16, Tarragona, 43007, Spain
| | - Javier Pérez-Ramírez
- Institute for Chemical and Bioengineering, Department of Chemistry and Applied Biosciences, ETH Zurich, Vladimir-Prelog-Weg 1, Zurich, 8093, Switzerland
| | - Sharon Mitchell
- Institute for Chemical and Bioengineering, Department of Chemistry and Applied Biosciences, ETH Zurich, Vladimir-Prelog-Weg 1, Zurich, 8093, Switzerland
| |
Collapse
|
4
|
Chee SW, Lunkenbein T, Schlögl R, Roldán Cuenya B. Operando Electron Microscopy of Catalysts: The Missing Cornerstone in Heterogeneous Catalysis Research? Chem Rev 2023; 123:13374-13418. [PMID: 37967448 PMCID: PMC10722467 DOI: 10.1021/acs.chemrev.3c00352] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Revised: 10/14/2023] [Accepted: 10/20/2023] [Indexed: 11/17/2023]
Abstract
Heterogeneous catalysis in thermal gas-phase and electrochemical liquid-phase chemical conversion plays an important role in our modern energy landscape. However, many of the structural features that drive efficient chemical energy conversion are still unknown. These features are, in general, highly distinct on the local scale and lack translational symmetry, and thus, they are difficult to capture without the required spatial and temporal resolution. Correlating these structures to their function will, conversely, allow us to disentangle irrelevant and relevant features, explore the entanglement of different local structures, and provide us with the necessary understanding to tailor novel catalyst systems with improved productivity. This critical review provides a summary of the still immature field of operando electron microscopy for thermal gas-phase and electrochemical liquid-phase reactions. It focuses on the complexity of investigating catalytic reactions and catalysts, progress in the field, and analysis. The forthcoming advances are discussed in view of correlative techniques, artificial intelligence in analysis, and novel reactor designs.
Collapse
Affiliation(s)
- See Wee Chee
- Department
of Interface Science, Fritz-Haber Institute
of the Max-Planck Society, 14195 Berlin, Germany
| | - Thomas Lunkenbein
- Department
of Inorganic Chemistry, Fritz-Haber Institute
of the Max-Planck Society, 14195 Berlin, Germany
| | - Robert Schlögl
- Department
of Interface Science, Fritz-Haber Institute
of the Max-Planck Society, 14195 Berlin, Germany
| | - Beatriz Roldán Cuenya
- Department
of Interface Science, Fritz-Haber Institute
of the Max-Planck Society, 14195 Berlin, Germany
| |
Collapse
|
5
|
Del-Pozo-Bueno D, Kepaptsoglou D, Peiró F, Estradé S. Comparative of machine learning classification strategies for electron energy loss spectroscopy: Support vector machines and artificial neural networks. Ultramicroscopy 2023; 253:113828. [PMID: 37556961 DOI: 10.1016/j.ultramic.2023.113828] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2023] [Revised: 07/26/2023] [Accepted: 08/02/2023] [Indexed: 08/11/2023]
Abstract
Machine Learning (ML) strategies applied to Scanning and conventional Transmission Electron Microscopy have become a valuable tool for analyzing the large volumes of data generated by various S/TEM techniques. In this work, we focus on Electron Energy Loss Spectroscopy (EELS) and study two ML techniques for classifying spectra in detail: Support Vector Machines (SVM) and Artificial Neural Networks (ANN). Firstly, we systematically analyze the optimal configurations and architectures for ANN classifiers using random search and the tree-structured Parzen estimator methods. Secondly, a new kernel strategy is introduced for the soft-margin SVMs, the cosine kernel, which offers a significant advantage over the previously studied kernels and other ML classification strategies. This kernel allows us to bypass the normalization of EEL spectra, achieving accurate classification. This result is highly relevant for the EELS community since we also assess the impact of common normalization techniques on our spectra using Uniform Manifold Approximation and Projection (UMAP), revealing a strong bias introduced in the spectra once normalized. In order to evaluate and study both classification strategies, we focus on determining the oxidation state of transition metals through their EEL spectra, examining which feature is more suitable for oxidation state classification: the oxygen K peak or the transition metal white lines. Subsequently, we compare the resistance to energy loss shifts for both classifiers and present a strategy to improve their resistance. The results of this study suggest the use of soft-margin SVMs for simpler EELS classification tasks with a limited number of spectra, as they provide performance comparable to ANNs while requiring lower computational resources and reduced training times. Conversely, ANNs are better suited for handling complex classification problems with extensive training data.
Collapse
Affiliation(s)
- Daniel Del-Pozo-Bueno
- Departament d'Enginyeria Electrònica i Biomèdica, LENS-MIND, Universitat de Barcelona, Barcelona 08028, Spain; Institute of Nanoscience and Nanotechnology (IN2UB), Universitat de Barcelona, Barcelona 08028, Spain.
| | - Demie Kepaptsoglou
- SuperSTEM, Sci-Tech Daresbury Campus, Daresbury WA4 4AD, UK; School of Physics, Engineering and Technology, University of York, Heslington YO10 5DD, UK
| | - Francesca Peiró
- Departament d'Enginyeria Electrònica i Biomèdica, LENS-MIND, Universitat de Barcelona, Barcelona 08028, Spain; Institute of Nanoscience and Nanotechnology (IN2UB), Universitat de Barcelona, Barcelona 08028, Spain
| | - Sònia Estradé
- Departament d'Enginyeria Electrònica i Biomèdica, LENS-MIND, Universitat de Barcelona, Barcelona 08028, Spain; Institute of Nanoscience and Nanotechnology (IN2UB), Universitat de Barcelona, Barcelona 08028, Spain
| |
Collapse
|
6
|
Leth Larsen MH, Lomholdt WB, Nuñez Valencia C, Hansen TW, Schiøtz J. Quantifying noise limitations of neural network segmentations in high-resolution transmission electron microscopy. Ultramicroscopy 2023; 253:113803. [PMID: 37499574 DOI: 10.1016/j.ultramic.2023.113803] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Revised: 05/29/2023] [Accepted: 06/30/2023] [Indexed: 07/29/2023]
Abstract
Motivated by the need for low electron dose transmission electron microscopy imaging, we report the optimal frame dose (i.e.e-/Å2) range for object detection and segmentation tasks with neural networks. The MSD-net architecture shows promising abilities over the industry standard U-net architecture in generalising to frame doses below the range included in the training set, for both simulated and experimental images. It also presents a heightened ability to learn from lower dose images. The MSD-net displays mild visibility of a Au nanoparticle at 20-30 e-/Å2, and converges at 200 e-/Å2 where a full segmentation of the nanoparticle is achieved. Between 30 and 200 e-/Å2 object detection applications are still possible. This work also highlights the importance of modelling the modulation transfer function when training with simulated images for applications on images acquired with scintillator based detectors such as the Gatan Oneview camera. A parametric form of the modulation transfer function is applied with varying ranges of parameters, and the effects on low electron dose segmentation is presented.
Collapse
Affiliation(s)
- Matthew Helmi Leth Larsen
- Computational Atomic-scale Materials Design (CAMD), Department of Physics, Technical University of Denmark, DK-2800 Kgs. Lyngby, Denmark
| | - William Bang Lomholdt
- National Center for Nano Fabrication and Characterization, Technical University of Denmark, DK-2800 Kgs. Lyngby, Denmark
| | - Cuauhtemoc Nuñez Valencia
- Computational Atomic-scale Materials Design (CAMD), Department of Physics, Technical University of Denmark, DK-2800 Kgs. Lyngby, Denmark
| | - Thomas W Hansen
- National Center for Nano Fabrication and Characterization, Technical University of Denmark, DK-2800 Kgs. Lyngby, Denmark
| | - Jakob Schiøtz
- Computational Atomic-scale Materials Design (CAMD), Department of Physics, Technical University of Denmark, DK-2800 Kgs. Lyngby, Denmark.
| |
Collapse
|
7
|
Gui C, Zhang Z, Li Z, Luo C, Xia J, Wu X, Chu J. Deep learning analysis on transmission electron microscope imaging of atomic defects in two-dimensional materials. iScience 2023; 26:107982. [PMID: 37810254 PMCID: PMC10551659 DOI: 10.1016/j.isci.2023.107982] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/10/2023] Open
Abstract
Defects are prevalent in two-dimensional (2D) materials due to thermal equilibrium and processing kinetics. The presence of various defect types can affect material properties significantly. With the development of the advanced transmission electron microscopy (TEM), the property-related structures could be investigated in multiple dimensions. It produces TEM datasets containing a large amount of information. Traditional data analysis is influenced by the subjectivity of researchers, and manual analysis is inefficient and imprecise. Recent developments in deep learning provide robust methods for the quantitative identification of defects in 2D materials efficiently and precisely. Taking advantage of big data, it breaks the limitations of TEM as a local characterization tool, making TEM an intelligent macroscopic analysis method. In this review, the recent developments in the TEM data analysis of defects in 2D materials using deep learning technology are summarized. Initially, an in-depth examination of the distinctions between TEM and natural images is presented. Subsequently, a comprehensive exploration of TEM data analysis ensues, encompassing denoising, point defects, line defects, planar defects, quantitative analysis, and applications. Furthermore, an exhaustive assessment of the significant obstacles encountered in the accurate identification of distinct structures is also provided.
Collapse
Affiliation(s)
- Chen Gui
- Shanghai Key Laboratory of Multidimensional Information Processing, School of Communication and Electronic Engineering, East China Normal University, Shanghai 200241, China
| | - Zhihao Zhang
- Shanghai Key Laboratory of Multidimensional Information Processing, School of Communication and Electronic Engineering, East China Normal University, Shanghai 200241, China
| | - Zongyi Li
- Shanghai Key Laboratory of Multidimensional Information Processing, School of Communication and Electronic Engineering, East China Normal University, Shanghai 200241, China
- JCET SEMICONDUCTOR INTEGRATION (SHAOXING) CO, LTD, Shaoxing, Zhejiang 312000, China
| | - Chen Luo
- Shanghai Key Laboratory of Multidimensional Information Processing, School of Communication and Electronic Engineering, East China Normal University, Shanghai 200241, China
- Institute of Optoelectronics, Fudan University, Shanghai 200433, China. Frontier Institute of Chip and System, Fudan University, Shanghai 200433, China
| | - Jiang Xia
- JCET SEMICONDUCTOR INTEGRATION (SHAOXING) CO, LTD, Shaoxing, Zhejiang 312000, China
| | - Xing Wu
- Shanghai Key Laboratory of Multidimensional Information Processing, School of Communication and Electronic Engineering, East China Normal University, Shanghai 200241, China
- State Key Laboratory of Infrared Physics, Shanghai Institute of Technical Physics, Chinese Academy of Sciences, Shanghai 200083, China
| | - Junhao Chu
- Shanghai Key Laboratory of Multidimensional Information Processing, School of Communication and Electronic Engineering, East China Normal University, Shanghai 200241, China
- State Key Laboratory of Infrared Physics, Shanghai Institute of Technical Physics, Chinese Academy of Sciences, Shanghai 200083, China
- Institute of Optoelectronics, Fudan University, Shanghai 200433, China. Frontier Institute of Chip and System, Fudan University, Shanghai 200433, China
| |
Collapse
|
8
|
Zhu D, Wang C, Zou P, Zhang R, Wang S, Song B, Yang X, Low KB, Xin HL. Deep-Learning Aided Atomic-Scale Phase Segmentation toward Diagnosing Complex Oxide Cathodes for Lithium-Ion Batteries. NANO LETTERS 2023; 23:8272-8279. [PMID: 37643420 DOI: 10.1021/acs.nanolett.3c02441] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/31/2023]
Abstract
Phase transformation─a universal phenomenon in materials─plays a key role in determining their properties. Resolving complex phase domains in materials is critical to fostering a new fundamental understanding that facilitates new material development. So far, although conventional classification strategies such as order-parameter methods have been developed to distinguish remarkably disparate phases, highly accurate and efficient phase segmentation for material systems composed of multiphases remains unavailable. Here, by coupling hard-attention-enhanced U-Net network and geometry simulation with atomic-resolution transmission electron microscopy, we successfully developed a deep-learning tool enabling automated atom-by-atom phase segmentation of intertwined phase domains in technologically important cathode materials for lithium-ion batteries. The new strategy outperforms traditional methods and quantitatively elucidates the correlation between the multiple phases formed during battery operation. Our work demonstrates how deep learning can be employed to foster an in-depth understanding of phase transformation-related key issues in complex materials.
Collapse
Affiliation(s)
- Dong Zhu
- Department of Physics and Astronomy, University of California Irvine, Irvine, California 92697, United States
- Computer Network Information Centre, Chinese Academy of Sciences, Beijing, 100190, P. R. China
- University of Chinese Academy of Sciences, Beijing, 100049, P. R. China
| | - Chunyang Wang
- Department of Physics and Astronomy, University of California Irvine, Irvine, California 92697, United States
| | - Peichao Zou
- Department of Physics and Astronomy, University of California Irvine, Irvine, California 92697, United States
| | - Rui Zhang
- Department of Physics and Astronomy, University of California Irvine, Irvine, California 92697, United States
| | - Shefang Wang
- BASF Corporation, Iselin, New Jersey 08830, United States
| | - Bohang Song
- BASF Corporation, Beachwood, Ohio 44122, United States
| | - Xiaoyu Yang
- Computer Network Information Centre, Chinese Academy of Sciences, Beijing, 100190, P. R. China
- University of Chinese Academy of Sciences, Beijing, 100049, P. R. China
| | - Ke-Bin Low
- BASF Corporation, Iselin, New Jersey 08830, United States
| | - Huolin L Xin
- Department of Physics and Astronomy, University of California Irvine, Irvine, California 92697, United States
| |
Collapse
|
9
|
Ni H, Wu Z, Wu X, Smith JG, Zachman MJ, Zuo JM, Ju L, Zhang G, Chi M. Quantifying Atomically Dispersed Catalysts Using Deep Learning Assisted Microscopy. NANO LETTERS 2023; 23:7442-7448. [PMID: 37566785 DOI: 10.1021/acs.nanolett.3c01892] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/13/2023]
Abstract
The catalytic performance of atomically dispersed catalysts (ADCs) is greatly influenced by their atomic configurations, such as atom-atom distances, clustering of atoms into dimers and trimers, and their distributions. Scanning transmission electron microscopy (STEM) is a powerful technique for imaging ADCs at the atomic scale; however, most STEM analyses of ADCs thus far have relied on human labeling, making it difficult to analyze large data sets. Here, we introduce a convolutional neural network (CNN)-based algorithm capable of quantifying the spatial arrangement of different adatom configurations. The algorithm was tested on different ADCs with varying support crystallinity and homogeneity. Results show that our algorithm can accurately identify atom positions and effectively analyze large data sets. This work provides a robust method to overcome a major bottleneck in STEM analysis for ADC catalyst research. We highlight the potential of this method to serve as an on-the-fly analysis tool for catalysts in future in situ microscopy experiments.
Collapse
Affiliation(s)
- Haoyang Ni
- Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, Oak Ridge, Tennessee 37831, United States
- Department of Materials Science and Engineering, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801, United States
| | - Zhenyao Wu
- Department of Mathematics, University of South Carolina, Columbia, South Carolina 29208, United States
| | - Xinyi Wu
- Department of Mathematics, University of South Carolina, Columbia, South Carolina 29208, United States
| | - Jacob G Smith
- Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, Oak Ridge, Tennessee 37831, United States
| | - Michael J Zachman
- Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, Oak Ridge, Tennessee 37831, United States
| | - Jian-Min Zuo
- Department of Materials Science and Engineering, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801, United States
| | - Lili Ju
- Department of Mathematics, University of South Carolina, Columbia, South Carolina 29208, United States
| | - Guannan Zhang
- Computer Science and Mathematics Division, Oak Ridge National Laboratory, Oak Ridge, Tennessee 37831, United States
| | - Miaofang Chi
- Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, Oak Ridge, Tennessee 37831, United States
| |
Collapse
|
10
|
Prifti E, Buban JP, Thind AS, Klie RF. Variational Convolutional Autoencoders for Anomaly Detection in Scanning Transmission Electron Microscopy. SMALL (WEINHEIM AN DER BERGSTRASSE, GERMANY) 2023; 19:e2205977. [PMID: 36651114 DOI: 10.1002/smll.202205977] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Revised: 12/08/2022] [Indexed: 06/17/2023]
Abstract
Identifying point defects and other structural anomalies using scanning transmission electron microscopy (STEM) is important to understand a material's properties caused by the disruption of the regular pattern of crystal lattice. Due to improvements in instrumentation stability and electron optics, atomic-resolution images with a field of view of several hundred nanometers can now be routinely acquired at 1-10 Hz frame rates and such data, which often contain thousands of atomic columns, need to be analyzed. To date, image analysis is performed largely manually, but recent developments in computer vision (CV) and machine learning (ML) now enable automated analysis of atomic structures and associated defects. Here, the authors report on how a Convolutional Variational Autoencoder (CVAE) can be utilized to detect structural anomalies in atomic-resolution STEM images. Specifically, the training set is limited to perfect crystal images , and the performance of a CVAE in differentiating between single-crystal bulk data or point defects is demonstrated. It is found that the CVAE can reproduce the perfect crystal data but not the defect input data. The disagreesments between the CVAE-predicted data for defects allows for a clear and automatic distinction and differentiation of several point defect types.
Collapse
Affiliation(s)
- Enea Prifti
- Department of Physics, University of Illinois Chicago, 845 W Taylor Street, Chicago, IL, 60607, USA
| | - James P Buban
- Department of Physics, University of Illinois Chicago, 845 W Taylor Street, Chicago, IL, 60607, USA
| | - Arashdeep Singh Thind
- Department of Physics, University of Illinois Chicago, 845 W Taylor Street, Chicago, IL, 60607, USA
| | - Robert F Klie
- Department of Physics, University of Illinois Chicago, 845 W Taylor Street, Chicago, IL, 60607, USA
| |
Collapse
|
11
|
A small-dataset-trained deep learning framework for identifying atoms on transmission electron microscopy images. Sci Rep 2023; 13:2631. [PMID: 36788257 PMCID: PMC9929221 DOI: 10.1038/s41598-023-29606-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2022] [Accepted: 02/07/2023] [Indexed: 02/16/2023] Open
Abstract
To accurately identify atoms on noisy transmission electron microscope images, a deep learning (DL) approach is employed to estimate the map of probabilities at each pixel for being an atom with element discernment. Thanks to a delicately-designed loss function and the ability to extract features, the proposed DL networks can be trained by a small dataset created from approximately 30 experimental images, each with a size of 256 × 256 pixels2. The accuracy and robustness of the network were verified by resolving the structural defects of graphene and polar structures in PbTiO3/SrTiO3 multilayers from both the general TEM images and their imitated images on which intensities of some pixels lost randomly. Such a network has the potential to identify atoms from very few images of beam-sensitive material and explosive images recorded in a dynamical atomic process. The idea of using a small-dataset-trained DL framework to resolve a specific problem may prove instructive for practical DL applications in various fields.
Collapse
|
12
|
Wang C, Wang X, Zhang R, Lei T, Kisslinger K, Xin HL. Resolving complex intralayer transition motifs in high-Ni-content layered cathode materials for lithium-ion batteries. NATURE MATERIALS 2023; 22:235-241. [PMID: 36702885 DOI: 10.1038/s41563-022-01461-5] [Citation(s) in RCA: 21] [Impact Index Per Article: 21.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Accepted: 12/09/2022] [Indexed: 06/18/2023]
Abstract
High-Ni-content layered materials are promising cathodes for next-generation lithium-ion batteries. However, investigating the atomic configurations of the delithiation-induced complex phase boundaries and their transitions remains challenging. Here, by using deep-learning-aided super-resolution electron microscopy, we resolve the intralayer transition motifs at complex phase boundaries in high-Ni cathodes. We reveal that an O3 → O1 transformation driven by delithiation leads to the formation of two types of O1-O3 interface, the continuous- and abrupt-transition interfaces. The interfacial misfit is accommodated by a continuous shear-transition zone and an abrupt structural unit, respectively. Atomic-scale simulations show that uneven in-plane Li+ distribution contributes to the formation of both types of interface, and the abrupt transition is energetically more favourable in a delithiated state where O1 is dominant, or when there is an uneven in-plane Li+ distribution in a delithiated O3 lattice. Moreover, a twin-like motif that introduces structural units analogous to the abrupt-type O1-O3 interface is also uncovered. The structural transition motifs resolved in this study provide further understanding of shear-induced phase transformations and phase boundaries in high-Ni layered cathodes.
Collapse
Affiliation(s)
- Chunyang Wang
- Department of Physics and Astronomy, University of California, Irvine, Irvine, CA, USA
| | - Xuelong Wang
- Chemistry Division, Brookhaven National Laboratory, Upton, NY, USA
| | - Rui Zhang
- Department of Physics and Astronomy, University of California, Irvine, Irvine, CA, USA
| | - Tianjiao Lei
- Department of Materials Science and Engineering, University of California, Irvine, Irvine, CA, USA
| | - Kim Kisslinger
- Center for Functional Nanomaterials, Brookhaven National Laboratory, Upton, NY, USA
| | - Huolin L Xin
- Department of Physics and Astronomy, University of California, Irvine, Irvine, CA, USA.
| |
Collapse
|
13
|
Botifoll M, Pinto-Huguet I, Arbiol J. Machine learning in electron microscopy for advanced nanocharacterization: current developments, available tools and future outlook. NANOSCALE HORIZONS 2022; 7:1427-1477. [PMID: 36239693 DOI: 10.1039/d2nh00377e] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
In the last few years, electron microscopy has experienced a new methodological paradigm aimed to fix the bottlenecks and overcome the challenges of its analytical workflow. Machine learning and artificial intelligence are answering this call providing powerful resources towards automation, exploration, and development. In this review, we evaluate the state-of-the-art of machine learning applied to electron microscopy (and obliquely, to materials and nano-sciences). We start from the traditional imaging techniques to reach the newest higher-dimensionality ones, also covering the recent advances in spectroscopy and tomography. Additionally, the present review provides a practical guide for microscopists, and in general for material scientists, but not necessarily advanced machine learning practitioners, to straightforwardly apply the offered set of tools to their own research. To conclude, we explore the state-of-the-art of other disciplines with a broader experience in applying artificial intelligence methods to their research (e.g., high-energy physics, astronomy, Earth sciences, and even robotics, videogames, or marketing and finances), in order to narrow down the incoming future of electron microscopy, its challenges and outlook.
Collapse
Affiliation(s)
- Marc Botifoll
- Catalan Institute of Nanoscience and Nanotechnology (ICN2), CSIC and BIST, Campus UAB, Bellaterra, 08193 Barcelona, Catalonia, Spain.
| | - Ivan Pinto-Huguet
- Catalan Institute of Nanoscience and Nanotechnology (ICN2), CSIC and BIST, Campus UAB, Bellaterra, 08193 Barcelona, Catalonia, Spain.
| | - Jordi Arbiol
- Catalan Institute of Nanoscience and Nanotechnology (ICN2), CSIC and BIST, Campus UAB, Bellaterra, 08193 Barcelona, Catalonia, Spain.
- ICREA, Pg. Lluís Companys 23, 08010 Barcelona, Catalonia, Spain
| |
Collapse
|
14
|
Larsen MHL, Dahl F, Hansen LP, Barton B, Kisielowski C, Helveg S, Winther O, Hansen TW, Schiøtz J. Reconstructing the exit wave of 2D materials in high-resolution transmission electron microscopy using machine learning. Ultramicroscopy 2022; 243:113641. [DOI: 10.1016/j.ultramic.2022.113641] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2021] [Revised: 09/23/2022] [Accepted: 10/30/2022] [Indexed: 11/06/2022]
|
15
|
Xu M, Kumar A, LeBeau JM. Towards Augmented Microscopy with Reinforcement Learning-Enhanced Workflows. MICROSCOPY AND MICROANALYSIS : THE OFFICIAL JOURNAL OF MICROSCOPY SOCIETY OF AMERICA, MICROBEAM ANALYSIS SOCIETY, MICROSCOPICAL SOCIETY OF CANADA 2022; 28:1-9. [PMID: 36062363 DOI: 10.1017/s1431927622012193] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Here, we report a case study implementation of reinforcement learning (RL) to automate operations in the scanning transmission electron microscopy workflow. To do so, we design a virtual, prototypical RL environment to test and develop a network to autonomously align the electron beam position without prior knowledge. Using this simulator, we evaluate the impact of environment design and algorithm hyperparameters on alignment accuracy and learning convergence, showing robust convergence across a wide hyperparameter space. Additionally, we deploy a successful model on the microscope to validate the approach and demonstrate the value of designing appropriate virtual environments. Consistent with simulated results, the on-microscope RL model achieves convergence to the goal alignment after minimal training. Overall, the results highlight that by taking advantage of RL, microscope operations can be automated without the need for extensive algorithm design, taking another step toward augmenting electron microscopy with machine learning methods.
Collapse
Affiliation(s)
- Michael Xu
- Department of Materials Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Abinash Kumar
- Department of Materials Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - James M LeBeau
- Department of Materials Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| |
Collapse
|
16
|
Lee K, Park J, Choi S, Lee Y, Lee S, Jung J, Lee JY, Ullah F, Tahir Z, Kim YS, Lee GH, Kim K. STEM Image Analysis Based on Deep Learning: Identification of Vacancy Defects and Polymorphs of MoS 2. NANO LETTERS 2022; 22:4677-4685. [PMID: 35674452 DOI: 10.1021/acs.nanolett.2c00550] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Scanning transmission electron microscopy (STEM) is an indispensable tool for atomic-resolution structural analysis for a wide range of materials. The conventional analysis of STEM images is an extensive hands-on process, which limits efficient handling of high-throughput data. Here, we apply a fully convolutional network (FCN) for identification of important structural features of two-dimensional crystals. ResUNet, a type of FCN, is utilized in identifying sulfur vacancies and polymorph types of MoS2 from atomic resolution STEM images. Efficient models are achieved based on training with simulated images in the presence of different levels of noise, aberrations, and carbon contamination. The accuracy of the FCN models toward extensive experimental STEM images is comparable to that of careful hands-on analysis. Our work provides a guideline on best practices to train a deep learning model for STEM image analysis and demonstrates FCN's application for efficient processing of a large volume of STEM data.
Collapse
Affiliation(s)
- Kihyun Lee
- Department of Physics, Yonsei University, Seoul 03722, Korea
| | - Jinsub Park
- Department of Physics, Yonsei University, Seoul 03722, Korea
| | - Soyeon Choi
- Department of Physics, Yonsei University, Seoul 03722, Korea
| | - Yangjin Lee
- Department of Physics, Yonsei University, Seoul 03722, Korea
- Center for Nanomedicine, Institute for Basic Science (IBS), Seoul 03722, Korea
| | - Sol Lee
- Department of Physics, Yonsei University, Seoul 03722, Korea
- Center for Nanomedicine, Institute for Basic Science (IBS), Seoul 03722, Korea
| | - Joowon Jung
- Department of Physics, Yonsei University, Seoul 03722, Korea
| | - Jong-Young Lee
- Department of Material Science and Engineering, Seoul National University, Seoul 08826, Korea
| | - Farman Ullah
- Department of Physics and Energy Harvest Storage Research Center, University of Ulsan, Ulsan 44610, Korea
| | - Zeeshan Tahir
- Department of Physics and Energy Harvest Storage Research Center, University of Ulsan, Ulsan 44610, Korea
| | - Yong Soo Kim
- Department of Physics and Energy Harvest Storage Research Center, University of Ulsan, Ulsan 44610, Korea
| | - Gwan-Hyoung Lee
- Department of Material Science and Engineering, Seoul National University, Seoul 08826, Korea
- Research Institute of Advanced Materials, Seoul National University, Seoul 08826, Korea
- Institute of Engineering Research, Seoul National University, Seoul 08826, Korea
- Institute of Applied Physics, Seoul National University, Seoul 08826, Korea
| | - Kwanpyo Kim
- Department of Physics, Yonsei University, Seoul 03722, Korea
- Center for Nanomedicine, Institute for Basic Science (IBS), Seoul 03722, Korea
| |
Collapse
|
17
|
Kawahara K, Ishikawa R, Sasano S, Shibata N, Ikuhara Y. Atomic-Resolution STEM Image Denoising by Total Variation Regularization. Microscopy (Oxf) 2022; 71:302-310. [PMID: 35713554 DOI: 10.1093/jmicro/dfac032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2021] [Revised: 05/31/2022] [Accepted: 06/16/2022] [Indexed: 11/13/2022] Open
Abstract
Atomic-resolution electron microscopy imaging of solid state material is a powerful method for structural analysis. Scanning transmission electron microscopy (STEM) is one of the actively used techniques to directly observe atoms in materials. However, some materials are easily damaged by the electron beam irradiation, and only noisy images are available when we decrease the electron dose to avoid beam damages. Therefore, a denoising process is necessary for precise structural analysis in low-dose STEM. In this study, we propose total variation (TV) denoising algorithm to remove quantum noise in a STEM image. We defined an entropy of STEM image that corresponds to the image contrast to determine a hyperparameter and we found that there is a hyperparameter that maximize the entropy. We acquired atomic resolution STEM image of CaF2 viewed along the [001] direction, and executed TV denoising. The atomic columns of Ca and F are clearly visualized by the TV denoising, and atomic position of Ca and F are determined with the error of ± 1 pm and ± 4 pm, respectively.
Collapse
Affiliation(s)
- Kazuaki Kawahara
- Institute of Engineering Innovation, The University of Tokyo, Bunkyo, Tokyo 113-8656, Japan
| | - Ryo Ishikawa
- Institute of Engineering Innovation, The University of Tokyo, Bunkyo, Tokyo 113-8656, Japan
| | - Shun Sasano
- Institute of Engineering Innovation, The University of Tokyo, Bunkyo, Tokyo 113-8656, Japan
| | - Naoya Shibata
- Institute of Engineering Innovation, The University of Tokyo, Bunkyo, Tokyo 113-8656, Japan.,Nanostructures Research Laboratory, Japan Fine Ceramics Center, Atsuta, Nagoya 456-8587, Japan
| | - Yuichi Ikuhara
- Institute of Engineering Innovation, The University of Tokyo, Bunkyo, Tokyo 113-8656, Japan.,Nanostructures Research Laboratory, Japan Fine Ceramics Center, Atsuta, Nagoya 456-8587, Japan
| |
Collapse
|
18
|
Zhang R, Wang C, Ge M, Xin HL. Accelerated Degradation in a Quasi-Single-Crystalline Layered Oxide Cathode for Lithium-Ion Batteries Caused by Residual Grain Boundaries. NANO LETTERS 2022; 22:3818-3824. [PMID: 35471058 DOI: 10.1021/acs.nanolett.2c01103] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
The rapidly growing demand of electrical vehicles (EVs) requires high-energy-density lithium-ion batteries (LIBs) with excellent cycling stability and safety performance. However, conventional polycrystalline high-Ni cathodes severely suffer from intrinsic chemomechanical degradation and fast capacity fade. The emerging single-crystallization strategy offers a promising pathway to improve the cathode's chemomechanical stability; however, the single-crystallinity of the cathode is not always guaranteed, and residual grain boundaries (GBs) could persist in nonideal synthesis conditions, leading to the formation of "quasi-single-crystalline" (QSC) cathodes. So far, there has been a lack of understanding of the influence of these residual GBs on the electrochemical performance and structural stability. Herein, we investigate the degradation pathway of a QSC high-Ni cathode through transmission electron microscopy and X-ray techniques. The residual GBs caused by insufficient calcination time dramatically exacerbate the cathode's chemomechanical instability and cycling performance. Our work offers important guidance for next-generation cathodes for long-life LIBs.
Collapse
Affiliation(s)
- Rui Zhang
- Department of Physics and Astronomy, University of California, Irvine, California 92697, United States
| | - Chunyang Wang
- Department of Physics and Astronomy, University of California, Irvine, California 92697, United States
| | - Mingyuan Ge
- National Synchrotron Light Source II, Brookhaven National Laboratory, Upton, New York 11973, United States
| | - Huolin L Xin
- Department of Physics and Astronomy, University of California, Irvine, California 92697, United States
| |
Collapse
|
19
|
Manzorro R, Xu Y, Vincent JL, Rivera R, Matteson DS, Crozier PA. Exploring Blob Detection to Determine Atomic Column Positions and Intensities in Time-Resolved TEM Images with Ultra-Low Signal-to-Noise. MICROSCOPY AND MICROANALYSIS : THE OFFICIAL JOURNAL OF MICROSCOPY SOCIETY OF AMERICA, MICROBEAM ANALYSIS SOCIETY, MICROSCOPICAL SOCIETY OF CANADA 2022; 28:1-14. [PMID: 35343415 DOI: 10.1017/s1431927622000356] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Spatially resolved in situ transmission electron microscopy (TEM), equipped with direct electron detection systems, is a suitable technique to record information about the atom-scale dynamics with millisecond temporal resolution from materials. However, characterizing dynamics or fluxional behavior requires processing short time exposure images which usually have severely degraded signal-to-noise ratios. The poor signal-to-noise associated with high temporal resolution makes it challenging to determine the position and intensity of atomic columns in materials undergoing structural dynamics. To address this challenge, we propose a noise-robust, processing approach based on blob detection, which has been previously established for identifying objects in images in the community of computer vision. In particular, a blob detection algorithm has been tailored to deal with noisy TEM image series from nanoparticle systems. In the presence of high noise content, our blob detection approach is demonstrated to outperform the results of other algorithms, enabling the determination of atomic column position and its intensity with a higher degree of precision.
Collapse
Affiliation(s)
- Ramon Manzorro
- School for the Engineering of Matter, Transport, and Energy, Arizona State University, Engineering G Wing #301, 501 E Tyler Mall, Tempe, AZ85287, USA
| | - Yuchen Xu
- Department of Statistics and Data Science, Cornell University, Ithaca, NY, USA
| | - Joshua L Vincent
- School for the Engineering of Matter, Transport, and Energy, Arizona State University, Engineering G Wing #301, 501 E Tyler Mall, Tempe, AZ85287, USA
| | - Roberto Rivera
- Department of Mathematical Sciences, University of Puerto Rico-Mayaguez, Mayaguez, Puerto Rico
| | - David S Matteson
- Department of Statistics and Data Science, Cornell University, Ithaca, NY, USA
| | - Peter A Crozier
- School for the Engineering of Matter, Transport, and Energy, Arizona State University, Engineering G Wing #301, 501 E Tyler Mall, Tempe, AZ85287, USA
| |
Collapse
|
20
|
Mitchell S, Parés F, Faust Akl D, Collins SM, Kepaptsoglou DM, Ramasse QM, Garcia-Gasulla D, Pérez-Ramírez J, López N. Automated Image Analysis for Single-Atom Detection in Catalytic Materials by Transmission Electron Microscopy. J Am Chem Soc 2022; 144:8018-8029. [PMID: 35333043 DOI: 10.1021/jacs.1c12466] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
Abstract
Single-atom catalytic sites may have existed in all supported transition metal catalysts since their first application. Yet, interest in the design of single-atom heterogeneous catalysts (SACs) only really grew when advances in transmission electron microscopy (TEM) permitted direct confirmation of metal site isolation. While atomic-resolution imaging remains a central characterization tool, poor statistical significance, reproducibility, and interoperability limit its scope for deriving robust characteristics about these frontier catalytic materials. Here, we introduce a customized deep-learning method for automated atom detection in image analysis, a rate-limiting step toward high-throughput TEM. Platinum atoms stabilized on a functionalized carbon support with a challenging irregular three-dimensional morphology serve as a practically relevant test system with promising scope in thermo- and electrochemical applications. The model detects over 20,000 atomic positions for the statistical analysis of important properties for establishing structure-performance relations over nanostructured catalysts, like the surface density, proximity, clustering extent, and dispersion uniformity of supported metal species. Good performance obtained on direct application of the model to an iron SAC based on carbon nitride demonstrates its generalizability for single-atom detection on carbon-related materials. The approach establishes a route to integrate artificial intelligence into routine TEM workflows. It accelerates image processing times by orders of magnitude and reduces human bias by providing an uncertainty analysis that is not readily quantifiable in manual atom identification, improving standardization and scalability.
Collapse
Affiliation(s)
- Sharon Mitchell
- Department of Chemistry and Applied Biosciences, Institute for Chemical and Bioengineering, ETH Zurich, Vladimir-Prelog-Weg 1, 8093 Zurich, Switzerland
| | - Ferran Parés
- Barcelona Supercomputing Center (BSC), Plaça d'Eusebi Güell 1-3, 08034 Barcelona, Spain
| | - Dario Faust Akl
- Department of Chemistry and Applied Biosciences, Institute for Chemical and Bioengineering, ETH Zurich, Vladimir-Prelog-Weg 1, 8093 Zurich, Switzerland
| | - Sean M Collins
- School of Chemical and Process Engineering and School of Chemistry, University of Leeds, Leeds LS2 9JT, U.K
| | - Demie M Kepaptsoglou
- SuperSTEM Laboratory, SciTech Daresbury Campus, Daresbury WA4 4AD, U.K.,Department of Physics, University of York, Heslington, York YO10 5DD, U.K
| | - Quentin M Ramasse
- SuperSTEM Laboratory, SciTech Daresbury Campus, Daresbury WA4 4AD, U.K.,School of Chemical and Process Engineering and School of Physics, University of Leeds, Leeds LS2 9JT, U.K
| | - Dario Garcia-Gasulla
- Barcelona Supercomputing Center (BSC), Plaça d'Eusebi Güell 1-3, 08034 Barcelona, Spain
| | - Javier Pérez-Ramírez
- Department of Chemistry and Applied Biosciences, Institute for Chemical and Bioengineering, ETH Zurich, Vladimir-Prelog-Weg 1, 8093 Zurich, Switzerland
| | - Núria López
- Institute of Chemical Research of Catalonia and The Barcelona Institute of Science and Technology, 43007 Tarragona, Spain
| |
Collapse
|
21
|
Adaptive Image Denoising Method Based on Diffusion Equation and Deep Learning. JOURNAL OF ROBOTICS 2022. [DOI: 10.1155/2022/7115551] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Effective noise removal has become a hot topic in image denoising research while preserving important details of an image. An adaptive threshold image denoising algorithm based on fitting diffusion is proposed. Firstly, the diffusion coefficient in the diffusion equation is improved, and the fitting diffusion coefficient is established to overcome the defects of texture detail loss and edge degradation caused by excessive diffusion intensity. Then, the threshold function is adaptively designed and improved so that it can automatically control the threshold of the function according to the maximum gray value of the image and the number of iterations, so as to further preserve the important details of the image such as edge and texture. A neural network is used to realize image denoising because of its good learning ability of image statistical characteristics, mainly by the diffusion equation and deep learning (CNN) algorithm as the foundation, focus on the effects of activation function of network optimization, using multiple feature extraction technology in-depth networks to study the characteristics of the input image richer, and how to better use the adaptive algorithm on the depth of diffusion equation and optimization backpropagation learning. The training speed of the model is accelerated and the convergence of the algorithm is improved. Combined with batch standardization and residual learning technology, the image denoising network model based on deep residual learning of the convolutional network is designed with better denoising performance. Finally, the algorithm is compared with other excellent denoising algorithms. From the comparison results, it can be seen that the improved denoising algorithm in this paper can also improve the detail restoration of denoised images without losing the sharpness. Moreover, it has better PSNR than other excellent denoising algorithms at different noise standard deviations. The PSNR of the new algorithm is greatly improved compared with the classical algorithm, which can effectively suppress the noise and protect the image edge and detail information.
Collapse
|
22
|
Wardini JL, Vahidi H, Guo H, Bowman WJ. Probing Multiscale Disorder in Pyrochlore and Related Complex Oxides in the Transmission Electron Microscope: A Review. Front Chem 2021; 9:743025. [PMID: 34917587 PMCID: PMC8668443 DOI: 10.3389/fchem.2021.743025] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2021] [Accepted: 10/15/2021] [Indexed: 11/13/2022] Open
Abstract
Transmission electron microscopy (TEM), and its counterpart, scanning TEM (STEM), are powerful materials characterization tools capable of probing crystal structure, composition, charge distribution, electronic structure, and bonding down to the atomic scale. Recent (S)TEM instrumentation developments such as electron beam aberration-correction as well as faster and more efficient signal detection systems have given rise to new and more powerful experimental methods, some of which (e.g., 4D-STEM, spectrum-imaging, in situ/operando (S)TEM)) facilitate the capture of high-dimensional datasets that contain spatially-resolved structural, spectroscopic, time- and/or stimulus-dependent information across the sub-angstrom to several micrometer length scale. Thus, through the variety of analysis methods available in the modern (S)TEM and its continual development towards high-dimensional data capture, it is well-suited to the challenge of characterizing isometric mixed-metal oxides such as pyrochlores, fluorites, and other complex oxides that reside on a continuum of chemical and spatial ordering. In this review, we present a suite of imaging and diffraction (S)TEM techniques that are uniquely suited to probe the many types, length-scales, and degrees of disorder in complex oxides, with a focus on disorder common to pyrochlores, fluorites and the expansive library of intermediate structures they may adopt. The application of these techniques to various complex oxides will be reviewed to demonstrate their capabilities and limitations in resolving the continuum of structural and chemical ordering in these systems.
Collapse
Affiliation(s)
- Jenna L. Wardini
- Materials Science and Engineering, University of California, Irvine, Irvine, CA, United States
| | - Hasti Vahidi
- Materials Science and Engineering, University of California, Irvine, Irvine, CA, United States
| | - Huiming Guo
- Materials Science and Engineering, University of California, Irvine, Irvine, CA, United States
| | - William J. Bowman
- Materials Science and Engineering, University of California, Irvine, Irvine, CA, United States
- Irvine Materials Research Institute, Irvine, CA, United States
| |
Collapse
|
23
|
Wang C, Zhang R, Siu C, Ge M, Kisslinger K, Shin Y, Xin HL. Chemomechanically Stable Ultrahigh-Ni Single-Crystalline Cathodes with Improved Oxygen Retention and Delayed Phase Degradations. NANO LETTERS 2021; 21:9797-9804. [PMID: 34752113 DOI: 10.1021/acs.nanolett.1c03852] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
The pressing demand in electrical vehicle (EV) markets for high-energy-density lithium-ion batteries (LIBs) requires further increasing the Ni content in high-Ni and low-Co cathodes. However, the commercialization of high-Ni cathodes is hindered by their intrinsic chemomechanical instabilities and fast capacity fade. The emerging single-crystalline strategy offers a promising solution, yet the operation and degradation mechanism of single-crystalline cathodes remain elusive, especially in the extremely challenging ultrahigh-Ni (Ni > 90%) regime whereby the phase transformation, oxygen loss, and mechanical instability are exacerbated with increased Ni content. Herein, we decipher the atomic-scale stabilization mechanism controlling the enhanced cycling performance of an ultrahigh-Ni single-crystalline cathode. We find that the charge/discharge inhomogeneity, the intergranular cracking, and oxygen-loss-related phase degradations that are prominent in ultrahigh-Ni polycrystalline cathodes are considerably suppressed in their single-crystalline counterparts, leading to improved chemomechanical and cycling stabilities of the single-crystalline cathodes. Our work offers important guidance for designing next-generation single-crystalline cathodes for high-capacity, long-life LIBs.
Collapse
Affiliation(s)
- Chunyang Wang
- Department of Physics and Astronomy, University of California, Irvine, California 92697, United States
| | - Rui Zhang
- Department of Physics and Astronomy, University of California, Irvine, California 92697, United States
| | - Carrie Siu
- Materials Engineering Research Facility, Applied Materials Division, Argonne National Laboratory, Lemont, Illinois 60439, United States
| | - Mingyuan Ge
- National Synchrotron Light Source II, Brookhaven National Laboratory, Upton, New York 11973, United States
| | - Kim Kisslinger
- Center for Functional Nanomaterials, Brookhaven National Laboratory, Upton, New York 11973, United States
| | - Youngho Shin
- Materials Engineering Research Facility, Applied Materials Division, Argonne National Laboratory, Lemont, Illinois 60439, United States
| | - Huolin L Xin
- Department of Physics and Astronomy, University of California, Irvine, California 92697, United States
| |
Collapse
|
24
|
Pate CM, Hart JL, Taheri ML. RapidEELS: machine learning for denoising and classification in rapid acquisition electron energy loss spectroscopy. Sci Rep 2021; 11:19515. [PMID: 34593833 PMCID: PMC8484590 DOI: 10.1038/s41598-021-97668-8] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Accepted: 08/24/2021] [Indexed: 11/08/2022] Open
Abstract
Recent advances in detectors for imaging and spectroscopy have afforded in situ, rapid acquisition of hyperspectral data. While electron energy loss spectroscopy (EELS) data acquisition speeds with electron counting are regularly reaching 400 frames per second with near-zero read noise, signal to noise ratio (SNR) remains a challenge owing to fundamental counting statistics. In order to advance understanding of transient materials phenomena during rapid acquisition EELS, trustworthy analysis of noisy spectra must be demonstrated. In this study, we applied machine learning techniques to denoise high frame rate spectra, benchmarking with slower frame rate "ground truths". The results provide a foundation for reliable use of low SNR data acquired in rapid, in-situ spectroscopy experiments. Such a tool-set is a first step toward both automation in microscopy as well as use of these methods to interrogate otherwise poorly understood transformations.
Collapse
Affiliation(s)
- Cassandra M Pate
- Department of Materials Science & Engineering, Johns Hopkins University, Baltimore, MD, 21218, USA
| | - James L Hart
- Department of Materials Science & Engineering, Johns Hopkins University, Baltimore, MD, 21218, USA
- Currently at Department of Mechanical Engineering & Materials Science, Yale University, New Haven, CT, 06511, USA
| | - Mitra L Taheri
- Department of Materials Science & Engineering, Johns Hopkins University, Baltimore, MD, 21218, USA.
| |
Collapse
|
25
|
Zheng S, Wang C, Yuan X, Xin HL. Super-compression of large electron microscopy time series by deep compressive sensing learning. PATTERNS (NEW YORK, N.Y.) 2021; 2:100292. [PMID: 34286306 PMCID: PMC8276025 DOI: 10.1016/j.patter.2021.100292] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/25/2021] [Revised: 04/21/2021] [Accepted: 05/26/2021] [Indexed: 11/14/2022]
Abstract
The development of ultrafast detectors for electron microscopy (EM) opens a new door to exploring dynamics of nanomaterials; however, it raises grand challenges for big data processing and storage. Here, we combine deep learning and temporal compressive sensing (TCS) to propose a novel EM big data compression strategy. Specifically, TCS is employed to compress sequential EM images into a single compressed measurement; an end-to-end deep learning network is leveraged to reconstruct the original images. Owing to the significantly improved compression efficiency and built-in denoising capability of the deep learning framework over conventional JPEG compression, compressed videos with a compression ratio of up to 30 can be reconstructed with high fidelity. Using this approach, considerable encoding power, memory, and transmission bandwidth can be saved, allowing it to be deployed to existing detectors. We anticipate the proposed technique will have far-reaching applications in edge computing for EM and other imaging techniques.
Collapse
Affiliation(s)
- Siming Zheng
- Department of Physics and Astronomy, University of California, Irvine, Irvine, CA, USA
| | - Chunyang Wang
- Department of Physics and Astronomy, University of California, Irvine, Irvine, CA, USA
| | - Xin Yuan
- Bell Labs, 600 Mountain Avenue, Murray Hill, NJ 07974, USA
| | - Huolin L. Xin
- Department of Physics and Astronomy, University of California, Irvine, Irvine, CA, USA
| |
Collapse
|
26
|
Lang S, Xu Y, Li L, Wang B, Yang Y, Xue Y, Shi K. Joint Detection of Tap and CEA Based on Deep Learning Medical Image Segmentation: Risk Prediction of Thyroid Cancer. JOURNAL OF HEALTHCARE ENGINEERING 2021; 2021:5920035. [PMID: 34158913 PMCID: PMC8187068 DOI: 10.1155/2021/5920035] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/22/2021] [Accepted: 05/12/2021] [Indexed: 02/06/2023]
Abstract
In recent years, the incidence of thyroid nodules has shown an increasing trend year by year and has become one of the important diseases that endanger human health. Ultrasound medical images based on deep learning are widely used in clinical diagnosis due to their cheapness, no radiation, and low cost. The use of image processing technology to accurately segment the nodule area provides important auxiliary information for the doctor's diagnosis, which is of great value for guiding clinical treatment. The purpose of this article is to explore the application value of combined detection of abnormal sugar-chain glycoprotein (TAP) and carcinoembryonic antigen (CEA) in the risk estimation of thyroid cancer in patients with thyroid nodules of type IV and above based on deep learning medical images. In this paper, ultrasound thyroid images are used as the research content, and the active contour level set method is used as the segmentation basis, and a segmentation algorithm for thyroid nodules is proposed. This paper takes ultrasound thyroid images as the research content, uses the active contour level set method as the basis of segmentation, and proposes an image segmentation algorithm Fast-SegNet based on deep learning, which extends the network model that was mainly used for thyroid medical image segmentation to more scenarios of the segmentation task. From January 2019 to October 2020, 400 patients with thyroid nodules of type IV and above were selected for physical examination and screening at the Health Management Center of our hospital, and they were diagnosed as thyroid cancer by pathological examination of thyroid nodules under B-ultrasound positioning. The detection rates of thyroid cancer in patients with thyroid nodules of type IV and above are compared; serum TAP and CEA levels are detected; PT-PCR is used to detect TTF-1, PTEN, and NIS expression; the detection, missed diagnosis, misdiagnosis rate, and diagnostic efficiency of the three detection methods are compared. This article uses the thyroid nodule region segmented based on deep learning medical images and compares experiments with CV model, LBF model, and DRLSE model. The experimental results show that the segmentation overlap rate of this method is as high as 98.4%, indicating that the algorithm proposed in this paper can more accurately extract the thyroid nodule area.
Collapse
Affiliation(s)
- Shaolei Lang
- Sanmenxia Central Hospital of Henan Province, Sanmenxia, Henan 472000, China
| | - Yinxia Xu
- Sanmenxia Central Hospital of Henan Province, Sanmenxia, Henan 472000, China
| | - Liang Li
- Sanmenxia Central Hospital of Henan Province, Sanmenxia, Henan 472000, China
| | - Bin Wang
- Sanmenxia Central Hospital of Henan Province, Sanmenxia, Henan 472000, China
| | - Yang Yang
- Sanmenxia Central Hospital of Henan Province, Sanmenxia, Henan 472000, China
| | - Yan Xue
- Sanmenxia Central Hospital of Henan Province, Sanmenxia, Henan 472000, China
| | - Kexin Shi
- Shaanxi Provincial People's Hospital, Taiyuan, Shaanxi 710068, China
| |
Collapse
|
27
|
Wang C, Zhang R, Kisslinger K, Xin HL. Atomic-Scale Observation of O1 Faulted Phase-Induced Deactivation of LiNiO 2 at High Voltage. NANO LETTERS 2021; 21:3657-3663. [PMID: 33821650 DOI: 10.1021/acs.nanolett.1c00862] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
LiNiO2 and cobalt-free ultrahigh-Ni content cathodes suffer from rapid capacity loss and severe chemomechanical degradation, especially when operated at high voltages. Here, by cycling LiNiO2 up to 4.7 V, we report the atomic-scale observation of O1 faulted phase-induced deactivation of LiNiO2. We find that, although a thin layer of the O3 phase forms on the particle surface by reversible O3 → O1 transformation during discharge, the bulk interior still maintains the O1 faulted phase, leading to rapid capacity loss of LiNiO2. Moreover, the atomic configuration of the O1/O3 interface is investigated comprehensively. We reveal that the misfit along the c axes of the O1 and O3 phases results in the formation of misfit dislocations, whereby cation mixing is promoted at the dislocation cores. A transition zone with continuous shear along the a-b plane is uncovered between the O1 and O3 phases for the first time. Besides, severe oxygen loss-induced pore formation and concurrent rock salt transformation are also identified.
Collapse
Affiliation(s)
- Chunyang Wang
- Department of Physics and Astronomy, University of California, Irvine, Irvine, California 92697, United States
| | - Rui Zhang
- Department of Physics and Astronomy, University of California, Irvine, Irvine, California 92697, United States
| | - Kim Kisslinger
- Center for Functional Nanomaterials, Brookhaven National Laboratory, Upton, New York 11973, United States
| | - Huolin L Xin
- Department of Physics and Astronomy, University of California, Irvine, Irvine, California 92697, United States
| |
Collapse
|