1
|
Kimoto K, Kikkawa J, Harano K, Cretu O, Shibazaki Y, Uesugi F. Unsupervised machine learning combined with 4D scanning transmission electron microscopy for bimodal nanostructural analysis. Sci Rep 2024; 14:2901. [PMID: 38316959 PMCID: PMC11303778 DOI: 10.1038/s41598-024-53289-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2023] [Accepted: 01/30/2024] [Indexed: 02/07/2024] Open
Abstract
Unsupervised machine learning techniques have been combined with scanning transmission electron microscopy (STEM) to enable comprehensive crystal structure analysis with nanometer spatial resolution. In this study, we investigated large-scale data obtained by four-dimensional (4D) STEM using dimensionality reduction techniques such as non-negative matrix factorization (NMF) and hierarchical clustering with various optimization methods. We developed software scripts incorporating knowledge of electron diffraction and STEM imaging for data preprocessing, NMF, and hierarchical clustering. Hierarchical clustering was performed using cross-correlation instead of conventional Euclidean distances, resulting in rotation-corrected diffractions and shift-corrected maps of major components. An experimental analysis was conducted on a high-pressure-annealed metallic glass, Zr-Cu-Al, revealing an amorphous matrix and crystalline precipitates with an average diameter of approximately 7 nm, which were challenging to detect using conventional STEM techniques. Combining 4D-STEM and optimized unsupervised machine learning enables comprehensive bimodal (i.e., spatial and reciprocal) analyses of material nanostructures.
Collapse
Affiliation(s)
- Koji Kimoto
- Center for Basic Research On Materials, National Institute for Materials Science (NIMS), 1-1 Namiki, Tsukuba, Ibaraki, 305-0044, Japan.
| | - Jun Kikkawa
- Center for Basic Research On Materials, National Institute for Materials Science (NIMS), 1-1 Namiki, Tsukuba, Ibaraki, 305-0044, Japan
| | - Koji Harano
- Center for Basic Research On Materials, National Institute for Materials Science (NIMS), 1-1 Namiki, Tsukuba, Ibaraki, 305-0044, Japan
| | - Ovidiu Cretu
- Center for Basic Research On Materials, National Institute for Materials Science (NIMS), 1-1 Namiki, Tsukuba, Ibaraki, 305-0044, Japan
| | - Yuki Shibazaki
- Institute of Materials Structure Science, High Energy Accelerator Research Organization, Tsukuba, Japan
| | - Fumihiko Uesugi
- Research Network and Facility Service Division, National Institute for Materials Science, Tsukuba, Japan
| |
Collapse
|
2
|
Terzoudis-Lumsden EWC, Petersen TC, Brown HG, Pelz PM, Ophus C, Findlay SD. Resolution of Virtual Depth Sectioning from Four-Dimensional Scanning Transmission Electron Microscopy. MICROSCOPY AND MICROANALYSIS : THE OFFICIAL JOURNAL OF MICROSCOPY SOCIETY OF AMERICA, MICROBEAM ANALYSIS SOCIETY, MICROSCOPICAL SOCIETY OF CANADA 2023; 29:1409-1421. [PMID: 37488824 DOI: 10.1093/micmic/ozad068] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/24/2022] [Revised: 03/15/2023] [Accepted: 05/25/2023] [Indexed: 07/26/2023]
Abstract
One approach to three-dimensional structure determination using the wealth of scattering data in four-dimensional (4D) scanning transmission electron microscopy (STEM) is the parallax method proposed by Ophus et al. (2019. Advanced phase reconstruction methods enabled by 4D scanning transmission electron microscopy, Microsc Microanal25, 10-11), which determines the scattering matrix and uses it to synthesize a virtual depth-sectioning reconstruction of the sample structure. Drawing on an equivalence with a hypothetical confocal imaging mode, we derive contrast transfer and point spread functions for this parallax method applied to weakly scattering objects, showing them identical to earlier depth-sectioning STEM modes when only bright field signal is used, but that improved depth resolution is possible if dark field signal can be used. Through a simulation-based study of doped Si, we show that this depth resolution is preserved for thicker samples, explore the impact of shot noise on the parallax reconstructions, discuss challenges to making use of dark field signal, and identify cases where the interpretation of the parallax reconstruction breaks down.
Collapse
Affiliation(s)
| | - T C Petersen
- School of Physics and Astronomy, Monash University, Melbourne, VIC 3800, Australia
- Monash Centre for Electron Microscopy, Monash University, Melbourne, VIC 3800, Australia
| | - H G Brown
- Ian Holmes Imaging Center, Bio21 Molecular Science and Biotechnology Institute, University of Melbourne, Melbourne, VIC 3052, Australia
| | - P M Pelz
- Institute of Micro- and Nanostructure Research and Center for Nanoanalysis and Electron Microscopy, Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen, Bavaria 91058, Germany
| | - C Ophus
- National Center for Electron Microscopy, Molecular Foundry, Lawrence Berkeley National Laboratory, Berkeley, CA 94720, USA
| | - S D Findlay
- School of Physics and Astronomy, Monash University, Melbourne, VIC 3800, Australia
| |
Collapse
|
3
|
Ribet SM, Ophus C, Dos Reis R, Dravid VP. Defect Contrast with 4D-STEM: Understanding Crystalline Order with Virtual Detectors and Beam Modification. MICROSCOPY AND MICROANALYSIS : THE OFFICIAL JOURNAL OF MICROSCOPY SOCIETY OF AMERICA, MICROBEAM ANALYSIS SOCIETY, MICROSCOPICAL SOCIETY OF CANADA 2023; 29:1087-1095. [PMID: 37749690 DOI: 10.1093/micmic/ozad045] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/11/2022] [Revised: 02/15/2023] [Accepted: 03/27/2023] [Indexed: 09/27/2023]
Abstract
Material properties strongly depend on the nature and concentration of defects. Characterizing these features may require nano- to atomic-scale resolution to establish structure-property relationships. 4D-STEM, a technique where diffraction patterns are acquired at a grid of points on the sample, provides a versatile method for highlighting defects. Computational analysis of the diffraction patterns with virtual detectors produces images that can map material properties. Here, using multislice simulations, we explore different virtual detectors that can be applied to the diffraction patterns that go beyond the binary response functions that are possible using ordinary STEM detectors. Using graphene and lead titanate as model systems, we investigate the application of virtual detectors to study local order and in particular defects. We find that using a small convergence angle with a rotationally varying detector most efficiently highlights defect signals. With experimental graphene data, we demonstrate the effectiveness of these detectors in characterizing atomic features, including vacancies, as suggested in simulations. Phase and amplitude modification of the electron beam provides another process handle to change image contrast in a 4D-STEM experiment. We demonstrate how tailored electron beams can enhance signals from short-range order and how a vortex beam can be used to characterize local symmetry.
Collapse
Affiliation(s)
- Stephanie M Ribet
- Department of Materials Science and Engineering, Northwestern University, Evanston, IL, USA
- International Institute of Nanotechnology, Northwestern University, Evanston, IL, USA
- National Center for Electron Microscopy, Molecular Foundry, Lawrence Berkeley National Laboratory, Berkeley, CA, USA
| | - Colin Ophus
- National Center for Electron Microscopy, Molecular Foundry, Lawrence Berkeley National Laboratory, Berkeley, CA, USA
| | - Roberto Dos Reis
- Department of Materials Science and Engineering, Northwestern University, Evanston, IL, USA
- International Institute of Nanotechnology, Northwestern University, Evanston, IL, USA
- The NUANCE Center, Northwestern University, Evanston, IL, USA
| | - Vinayak P Dravid
- Department of Materials Science and Engineering, Northwestern University, Evanston, IL, USA
- International Institute of Nanotechnology, Northwestern University, Evanston, IL, USA
- The NUANCE Center, Northwestern University, Evanston, IL, USA
| |
Collapse
|
4
|
Esser BD, Etheridge J. Complementary ADF-STEM: a Flexible Approach to Quantitative 4D-STEM. Ultramicroscopy 2023; 243:113627. [DOI: 10.1016/j.ultramic.2022.113627] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2022] [Revised: 09/26/2022] [Accepted: 10/02/2022] [Indexed: 11/06/2022]
|
5
|
Kim YH, Yang SH, Jeong M, Jung MH, Yang D, Lee H, Moon T, Heo J, Jeong HY, Lee E, Kim YM. Hybrid Deep Learning Crystallographic Mapping of Polymorphic Phases in Polycrystalline Hf 0.5 Zr 0.5 O 2 Thin Films. SMALL (WEINHEIM AN DER BERGSTRASSE, GERMANY) 2022; 18:e2107620. [PMID: 35373528 DOI: 10.1002/smll.202107620] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/08/2021] [Revised: 03/03/2022] [Indexed: 06/14/2023]
Abstract
By controlling the configuration of polymorphic phases in high-k Hf0.5 Zr0.5 O2 thin films, new functionalities such as persistent ferroelectricity at an extremely small scale can be exploited. To bolster the technological progress and fundamental understanding of phase stabilization (or transition) and switching behavior in the research area, efficient and reliable mapping of the crystal symmetry encompassing the whole scale of thin films is an urgent requisite. Atomic-scale observation with electron microscopy can provide decisive information for discriminating structures with similar symmetries. However, it often demands multiple/multiscale analysis for cross-validation with other techniques, such as X-ray diffraction, due to the limited range of observation. Herein, an efficient and automated methodology for large-scale mapping of the crystal symmetries in polycrystalline Hf0.5 Zr0.5 O2 thin films is developed using scanning probe-based diffraction and a hybrid deep convolutional neural network at a 2 nm2 resolution. The results for the doped hafnia films are fully proven to be compatible with atomic structures revealed by microscopy imaging, not requiring intensive human input for interpretation.
Collapse
Affiliation(s)
- Young-Hoon Kim
- Department of Energy Science, Sungkyunkwan University (SKKU), Suwon, 16419, Republic of Korea
| | - Sang-Hyeok Yang
- Department of Energy Science, Sungkyunkwan University (SKKU), Suwon, 16419, Republic of Korea
| | - Myoungho Jeong
- Analytical Engineering Group, Samsung Advanced Institute of Technology (SAIT), Samsung Electronics, Suwon, 16678, Republic of Korea
| | - Min-Hyoung Jung
- Department of Energy Science, Sungkyunkwan University (SKKU), Suwon, 16419, Republic of Korea
| | - Daehee Yang
- Department of Energy Science, Sungkyunkwan University (SKKU), Suwon, 16419, Republic of Korea
| | - Hyangsook Lee
- Analytical Engineering Group, Samsung Advanced Institute of Technology (SAIT), Samsung Electronics, Suwon, 16678, Republic of Korea
| | - Taehwan Moon
- Beyond Silicon Lab, Samsung Advanced Institute of Technology (SAIT), Samsung Electronics, Suwon, 16678, Republic of Korea
| | - Jinseong Heo
- Beyond Silicon Lab, Samsung Advanced Institute of Technology (SAIT), Samsung Electronics, Suwon, 16678, Republic of Korea
| | - Hu Young Jeong
- Graduate School of Semiconductor Materials and Devices Engineering, Ulsan National Institute of Science and Technology (UNIST), Ulsan, 44919, Republic of Korea
| | - Eunha Lee
- Analytical Engineering Group, Samsung Advanced Institute of Technology (SAIT), Samsung Electronics, Suwon, 16678, Republic of Korea
| | - Young-Min Kim
- Department of Energy Science, Sungkyunkwan University (SKKU), Suwon, 16419, Republic of Korea
| |
Collapse
|
6
|
Moeck P. Objective crystallographic symmetry classifications of a noisy crystal pattern with strong Fedorov-type pseudosymmetries and its optimal image-quality enhancement. Acta Crystallogr A Found Adv 2022; 78:172-199. [PMID: 35502711 PMCID: PMC9062829 DOI: 10.1107/s2053273322000845] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2021] [Accepted: 01/24/2022] [Indexed: 11/29/2022] Open
Abstract
Statistically sound crystallographic symmetry classifications are obtained with information-theory-based methods in the presence of approximately Gaussian distributed noise. A set of three synthetic patterns with strong Fedorov-type pseudosymmetries and varying amounts of noise serve as examples. Contrary to traditional crystallographic symmetry classifications with an image processing program such as CRISP, the classification process does not need to be supervised by a human being and is free of any subjectively set thresholds in the geometric model selection process. This enables crystallographic symmetry classification of digital images that are more or less periodic in two dimensions (2D), also known as crystal patterns, as recorded with sufficient structural resolution from a wide range of crystalline samples with different types of scanning probe and transmission electron microscopes. Correct symmetry classifications enable the optimal crystallographic processing of such images. That processing consists of the averaging over all asymmetric units in all unit cells in the selected image area and significantly enhances both the signal-to-noise ratio and the structural resolution of a microscopic study of a crystal. For sufficiently complex crystal patterns, the information-theoretic symmetry classification methods are more accurate than both visual classifications by human experts and the recommendations of one of the popular crystallographic image processing programs of electron crystallography.
Collapse
Affiliation(s)
- Peter Moeck
- Department of Physics, Portland State University, Portland 97201-0751, USA
| |
Collapse
|
7
|
Correlation symmetry analysis of electron nanodiffraction from amorphous materials. Ultramicroscopy 2021; 232:113405. [PMID: 34673441 DOI: 10.1016/j.ultramic.2021.113405] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2021] [Revised: 09/13/2021] [Accepted: 10/03/2021] [Indexed: 11/22/2022]
Abstract
Angular symmetry in diffraction reflects rotational symmetry in the sample. We introduce the angular symmetry coefficient as a method to extract local symmetry information from electron nanodiffraction patterns of amorphous materials. Symmetry coefficients are the average of the angular autocorrelation function at the characteristic angles of a particular rotational symmetry. The symmetry coefficients avoid non-structural features arising from Fourier transformation and Friedel symmetry breakdown that affect the angular power spectrum approach to determining angular symmetries in amorphous nanodiffraction. Both methods require thin samples to avoid overlapping diffraction from clusters of atoms separated in the thickness of the sample, but symmetry coefficients are more forgiving. Electron nanodiffraction experiments on a Pd-based metallic glass sample demonstrate both potentially misleading information in angular power spectrum and the utility of symmetry coefficients.
Collapse
|