1
|
Olszta M, Hopkins D, Fiedler KR, Oostrom M, Akers S, Spurgeon SR. An Automated Scanning Transmission Electron Microscope Guided by Sparse Data Analytics. MICROSCOPY AND MICROANALYSIS : THE OFFICIAL JOURNAL OF MICROSCOPY SOCIETY OF AMERICA, MICROBEAM ANALYSIS SOCIETY, MICROSCOPICAL SOCIETY OF CANADA 2022; 28:1-11. [PMID: 35686442 DOI: 10.1017/s1431927622012065] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Artificial intelligence (AI) promises to reshape scientific inquiry and enable breakthrough discoveries in areas such as energy storage, quantum computing, and biomedicine. Scanning transmission electron microscopy (STEM), a cornerstone of the study of chemical and materials systems, stands to benefit greatly from AI-driven automation. However, present barriers to low-level instrument control, as well as generalizable and interpretable feature detection, make truly automated microscopy impractical. Here, we discuss the design of a closed-loop instrument control platform guided by emerging sparse data analytics. We hypothesize that a centralized controller, informed by machine learning combining limited a priori knowledge and task-based discrimination, could drive on-the-fly experimental decision-making. This platform may unlock practical, automated analysis of a variety of material features, enabling new high-throughput and statistical studies.
Collapse
Affiliation(s)
- Matthew Olszta
- Energy and Environment Directorate, Pacific Northwest National Laboratory, Richland, WA 99352, USA
| | - Derek Hopkins
- Environmental Molecular Sciences Laboratory, Pacific Northwest National Laboratory, Richland, WA 99352, USA
| | - Kevin R Fiedler
- College of Arts and Sciences, Washington State University - Tri-Cities, Richland, WA 99354, USA
| | - Marjolein Oostrom
- National Security Directorate, Pacific Northwest National Laboratory, Richland, WA 99352, USA
| | - Sarah Akers
- National Security Directorate, Pacific Northwest National Laboratory, Richland, WA 99352, USA
| | - Steven R Spurgeon
- Energy and Environment Directorate, Pacific Northwest National Laboratory, Richland, WA 99352, USA
- Department of Physics, University of Washington, Seattle, WA 98195, USA
| |
Collapse
|
2
|
Deep learning detection of nanoparticles and multiple object tracking of their dynamic evolution during in situ ETEM studies. Sci Rep 2022; 12:2484. [PMID: 35169206 PMCID: PMC8847623 DOI: 10.1038/s41598-022-06308-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2021] [Accepted: 01/24/2022] [Indexed: 11/08/2022] Open
Abstract
In situ transmission electron microscopy (TEM) studies of dynamic events produce large quantities of data especially under the form of images. In the important case of heterogeneous catalysis, environmental TEM (ETEM) under gas and temperature allows to follow a large population of supported nanoparticles (NPs) evolving under reactive conditions. Interpreting properly large image sequences gives precious information on the catalytic properties of the active phase by identifying causes for its deactivation. To perform a quantitative, objective and robust treatment, we propose an automatic procedure to track nanoparticles observed in Scanning ETEM (STEM in ETEM). Our approach involves deep learning and computer vision developments in multiple object tracking. At first, a registration step corrects the image displacements and misalignment inherent to the in situ acquisition. Then, a deep learning approach detects the nanoparticles on all frames of video sequences. Finally, an iterative tracking algorithm reconstructs their trajectories. This treatment allows to deduce quantitative and statistical features about their evolution or motion, such as a Brownian behavior and merging or crossing events. We treat the case of in situ calcination of palladium (oxide) / delta-alumina, where the present approach allows a discussion of operating processes such as Ostwald ripening or NP aggregative coalescence.
Collapse
|
3
|
Wang X, Li J, Ha HD, Dahl JC, Ondry JC, Moreno-Hernandez I, Head-Gordon T, Alivisatos AP. AutoDetect-mNP: An Unsupervised Machine Learning Algorithm for Automated Analysis of Transmission Electron Microscope Images of Metal Nanoparticles. JACS AU 2021; 1:316-327. [PMID: 33778811 PMCID: PMC7988451 DOI: 10.1021/jacsau.0c00030] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/23/2020] [Indexed: 05/27/2023]
Abstract
The synthesis quality of artificial inorganic nanocrystals is most often assessed by transmission electron microscopy (TEM) for which high-throughput advances have dramatically increased both the quantity and information richness of metal nanoparticle (mNP) characterization. Existing automated data analysis algorithms of TEM mNP images generally adopt a supervised approach, requiring a significant effort in human preparation of labeled data that reduces objectivity, efficiency, and generalizability. We have developed an unsupervised algorithm AutoDetect-mNP for automated analysis of TEM images that objectively extracts morphological information on convex mNPs from TEM images based on their shape attributes, requiring little to no human input in the process. The performance of AutoDetect-mNP is tested on two data sets of bright field TEM images of Au nanoparticles with different shapes and further extended to palladium nanocubes and cadmium selenide quantum dots, demonstrating that the algorithm is quantitatively reliable and can thus serve as a generalizable measure of the morphology distributions of any mNP synthesis. The AutoDetect-mNP algorithm will aid in future developments of high-throughput characterization of mNPs and the future advent of time-resolved TEM studies that can investigate reaction mechanisms of mNP synthesis and reactivity.
Collapse
Affiliation(s)
- Xingzhi Wang
- Department
of Chemistry, University of California, Berkeley, California 94720, United States
- Materials
Sciences Division, Lawrence Berkeley National
Laboratory, Berkeley, California 94720, United States
| | - Jie Li
- Department
of Chemistry, University of California, Berkeley, California 94720, United States
- Kenneth
S. Pitzer Theory Center, University of California, Berkeley, California 94720, United States
| | - Hyun Dong Ha
- Department
of Chemistry, University of California, Berkeley, California 94720, United States
- Materials
Sciences Division, Lawrence Berkeley National
Laboratory, Berkeley, California 94720, United States
| | - Jakob C. Dahl
- Department
of Chemistry, University of California, Berkeley, California 94720, United States
- Materials
Sciences Division, Lawrence Berkeley National
Laboratory, Berkeley, California 94720, United States
| | - Justin C. Ondry
- Department
of Chemistry, University of California, Berkeley, California 94720, United States
- Kavli
Energy NanoScience Institute, Berkeley, California 94720, United States
| | - Ivan Moreno-Hernandez
- Department
of Chemistry, University of California, Berkeley, California 94720, United States
| | - Teresa Head-Gordon
- Department
of Chemistry, University of California, Berkeley, California 94720, United States
- Kenneth
S. Pitzer Theory Center, University of California, Berkeley, California 94720, United States
- Chemical
Sciences Division, Lawrence Berkeley National
Laboratory, Berkeley, California 94720, United States
- Departments
of Bioengineering and Chemical and Biomolecular Engineering, University of California, Berkeley, California 94720, United States
| | - A. Paul Alivisatos
- Department
of Chemistry, University of California, Berkeley, California 94720, United States
- Materials
Sciences Division, Lawrence Berkeley National
Laboratory, Berkeley, California 94720, United States
- Kavli
Energy NanoScience Institute, Berkeley, California 94720, United States
- Department
of Materials Science and Engineering, University
of California, Berkeley, California 94720, United States
| |
Collapse
|
4
|
XIA Y, HARRISON P, ORNELAS I, WANG H, LI Z. HAADF‐STEM image analysis for size‐selected platinum nanoclusters. J Microsc 2020; 279:229-233. [DOI: 10.1111/jmi.12877] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2019] [Revised: 02/03/2020] [Accepted: 02/06/2020] [Indexed: 11/29/2022]
Affiliation(s)
- Y. XIA
- School of Physics and AstronomyUniversity of Birmingham Birmingham U.K
- Department of Materials Science and EngineeringSouthern University of Science and Technology Shenzhen Guangdong China
| | - P. HARRISON
- School of Physics and AstronomyUniversity of Birmingham Birmingham U.K
| | - I.M. ORNELAS
- School of Physics and AstronomyUniversity of Birmingham Birmingham U.K
| | - H.L. WANG
- Department of Materials Science and EngineeringSouthern University of Science and Technology Shenzhen Guangdong China
| | - Z.Y. LI
- School of Physics and AstronomyUniversity of Birmingham Birmingham U.K
| |
Collapse
|
5
|
Niihori Y, Yoshida K, Hossain S, Kurashige W, Negishi Y. Deepening the Understanding of Thiolate-Protected Metal Clusters Using High-Performance Liquid Chromatography. BULLETIN OF THE CHEMICAL SOCIETY OF JAPAN 2019. [DOI: 10.1246/bcsj.20180357] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/22/2023]
Affiliation(s)
- Yoshiki Niihori
- Department of Applied Chemistry, Faculty of Science, Tokyo University of Science, 1-3 Kagurazaka, Shinjuku-ku, Tokyo 162-8601, Japan
| | - Kana Yoshida
- Department of Applied Chemistry, Faculty of Science, Tokyo University of Science, 1-3 Kagurazaka, Shinjuku-ku, Tokyo 162-8601, Japan
| | - Sakiat Hossain
- Department of Applied Chemistry, Faculty of Science, Tokyo University of Science, 1-3 Kagurazaka, Shinjuku-ku, Tokyo 162-8601, Japan
| | - Wataru Kurashige
- Department of Applied Chemistry, Faculty of Science, Tokyo University of Science, 1-3 Kagurazaka, Shinjuku-ku, Tokyo 162-8601, Japan
- Photocatalysis International Research Center, Tokyo University of Science, 2641 Yamazaki, Noda, Chiba 278-8510, Japan
| | - Yuichi Negishi
- Department of Applied Chemistry, Faculty of Science, Tokyo University of Science, 1-3 Kagurazaka, Shinjuku-ku, Tokyo 162-8601, Japan
- Photocatalysis International Research Center, Tokyo University of Science, 2641 Yamazaki, Noda, Chiba 278-8510, Japan
| |
Collapse
|
6
|
Zhao S, Austin N, Li M, Song Y, House SD, Bernhard S, Yang JC, Mpourmpakis G, Jin R. Influence of Atomic-Level Morphology on Catalysis: The Case of Sphere and Rod-Like Gold Nanoclusters for CO2 Electroreduction. ACS Catal 2018. [DOI: 10.1021/acscatal.8b00365] [Citation(s) in RCA: 108] [Impact Index Per Article: 18.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Affiliation(s)
- Shuo Zhao
- Department of Chemistry, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, United States
| | - Natalie Austin
- Department of Chemical Engineering, University of Pittsburgh, Pittsburgh, Pennsylvania 15261, United States
| | - Mo Li
- Department of Chemistry, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, United States
| | - Yongbo Song
- Department of Chemistry, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, United States
| | - Stephen D. House
- Chemical and Petroleum Engineering, and Physics, University of Pittsburgh, Pittsburgh, Pennsylvania 15261, United States
| | - Stefan Bernhard
- Department of Chemistry, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, United States
| | - Judith C. Yang
- Chemical and Petroleum Engineering, and Physics, University of Pittsburgh, Pittsburgh, Pennsylvania 15261, United States
| | - Giannis Mpourmpakis
- Department of Chemical Engineering, University of Pittsburgh, Pittsburgh, Pennsylvania 15261, United States
| | - Rongchao Jin
- Department of Chemistry, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, United States
| |
Collapse
|
7
|
Jones L, Varambhia A, Sawada H, Nellist PD. An optical configuration for fastidious STEM detector calibration and the effect of the objective-lens pre-field. J Microsc 2018; 270:176-187. [PMID: 29315554 DOI: 10.1111/jmi.12672] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2017] [Revised: 11/21/2017] [Accepted: 11/28/2017] [Indexed: 12/01/2022]
Abstract
In the scanning transmission electron microscope, an accurate knowledge of detector collection angles is paramount in order to quantify signals on an absolute scale. Here we present an optical configuration designed for the accurate measurement of collection angles for both image-detectors and energy-loss spectrometers. By deflecting a parallel electron beam, carefully calibrated using a diffraction pattern from a known material, we can directly observe the projection-distortion in the post-specimen lenses of probe-corrected instruments, the 3-fold caustic when an image-corrector is fitted, and any misalignment of imaging detectors or spectrometer apertures. We also discuss for the first time, the effect that higher-order aberrations in the objective-lens pre-field has on such an angle-based detector mapping procedure.
Collapse
Affiliation(s)
- L Jones
- School of Physics, Trinity College Dublin, Dublin 2, Ireland.,Advanced Microscopy Laboratory, Centre for Research on Adaptive Nanostructures and Nanodevices (CRANN), Dublin 2, Ireland
| | - A Varambhia
- Department of Materials, University of Oxford, Oxford, U.K
| | - H Sawada
- JEOL Ltd, Musashino, Akishima, Tokyo.,Electron Physical Sciences Imaging Center, Diamond Light Source Ltd. Didcot, U.K
| | - P D Nellist
- Department of Materials, University of Oxford, Oxford, U.K
| |
Collapse
|