Electrostatic Discovery Atomic Force Microscopy

Automated Structure Discovery for Scanning Tunneling Microscopy

Scanning tunneling microscopy (STM) with a functionalized tip apex reveals the geometric and electronic structures of a sample within the same experiment. However, the complex nature of the signal makes images difficult to interpret and has so far limited most research to planar samples with a known chemical composition. Here, we present automated structure discovery for STM (ASD-STM), a machine learning tool for predicting the atomic structure directly from an STM image, by building upon successful methods for structure discovery in noncontact atomic force microscopy (nc-AFM). We apply the method on various organic molecules and achieve good accuracy on structure predictions and chemical identification on a qualitative level while highlighting future development requirements for ASD-STM. This method is directly applicable to experimental STM images of organic molecules, making structure discovery available for a wider scanning probe microscopy audience outside of nc-AFM. This work also allows more advanced machine learning methods to be developed for STM structure discovery.

Automated Generation and Analysis of Molecular Images Using Generative Artificial Intelligence Models

The development of scanning probe microscopy (SPM) has enabled unprecedented scientific discoveries through high-resolution imaging. Simulations and theoretical analysis of SPM images are equally important as obtaining experimental images since their comparisons provide fruitful understandings of the structures and physical properties of the investigated systems. So far, SPM image simulations are conventionally based on quantum mechanical theories, which can take several days in tasks of large-scale systems. Here, we have developed a scanning tunneling microscopy (STM) molecular image simulation and analysis framework based on a generative adversarial model, CycleGAN. It allows efficient translations between STM data and molecular models. Our CycleGAN-based framework introduces an approach for high-fidelity STM image simulation, outperforming traditional quantum mechanical methods in efficiency and accuracy. We envision that the integration of generative networks and high-resolution molecular imaging opens avenues in materials discovery relying on SPM technologies.

Structure Discovery in Atomic Force Microscopy Imaging of Ice

The interaction of water with surfaces is crucially important in a wide range of natural and technological settings. In particular, at low temperatures, unveiling the atomistic structure of adsorbed water clusters would provide valuable data for understanding the ice nucleation process. Using high-resolution atomic force microscopy (AFM) and scanning tunneling microscopy, several studies have demonstrated the presence of water pentamers, hexamers, and heptamers (and of their combinations) on a variety of metallic surfaces, as well as the initial stages of 2D ice growth on an insulating surface. However, in all of these cases, the observed structures were completely flat, providing a relatively straightforward path to interpretation. Here, we present high-resolution AFM measurements of several water clusters on Au(111) and Cu(111), whose understanding presents significant challenges due to both their highly 3D configuration and their large size. For each of them, we use a combination of machine learning, atomistic modeling with neural network potentials, and statistical sampling to propose an underlying atomic structure, finally comparing its AFM simulated images to the experimental ones. These results provide insights into the early phases of ice formation, which is a ubiquitous phenomenon ranging from biology to astrophysics.

Towards smart scanning probe lithography: a framework accelerating nano-fabrication process with in-situ characterization via machine learning

Scanning probe lithography (SPL) is a promising technology to fabricate high-resolution, customized and cost-effective features at the nanoscale. However, the quality of nano-fabrication, particularly the critical dimension, is significantly influenced by various SPL fabrication techniques and their corresponding process parameters. Meanwhile, the identification and measurement of nano-fabrication features are very time-consuming and subjective. To tackle these challenges, we propose a novel framework for process parameter optimization and feature segmentation of SPL via machine learning (ML). Different from traditional SPL techniques that rely on manual labeling-based experimental methods, the proposed framework intelligently extracts reliable and global information for statistical analysis to fine-tune and optimize process parameters. Based on the proposed framework, we realized the processing of smaller critical dimensions through the optimization of process parameters, and performed direct-write nano-lithography on a large scale. Furthermore, data-driven feature extraction and analysis could potentially provide guidance for other characterization methods and fabrication quality optimization.

Machine learning-aided atomic structure identification of interfacial ionic hydrates from AFM images

Relevant to broad applied fields and natural processes, interfacial ionic hydrates have been widely studied by using ultrahigh-resolution atomic force microscopy (AFM). However, the complex relationship between the AFM signal and the investigated system makes it difficult to determine the atomic structure of such a complex system from AFM images alone. Using machine learning, we achieved precise identification of the atomic structures of interfacial water/ionic hydrates based on AFM images, including the position of each atom and the orientations of water molecules. Furthermore, it was found that structure prediction of ionic hydrates can be achieved cost-effectively by transfer learning using neural network trained with easily available interfacial water data. Thus, this work provides an efficient and economical methodology that not only opens up avenues to determine atomic structures of more complex systems from AFM images, but may also help to interpret other scientific studies involving sophisticated experimental results.

Molecular Identification from AFM Images Using the IUPAC Nomenclature and Attribute Multimodal Recurrent Neural Networks

Spectroscopic methods─like nuclear magnetic resonance, mass spectrometry, X-ray diffraction, and UV/visible spectroscopies─applied to molecular ensembles have so far been the workhorse for molecular identification. Here, we propose a radically different chemical characterization approach, based on the ability of noncontact atomic force microscopy with metal tips functionalized with a CO molecule at the tip apex (referred as HR-AFM) to resolve the internal structure of individual molecules. Our work demonstrates that a stack of constant-height HR-AFM images carries enough chemical information for a complete identification (structure and composition) of quasiplanar organic molecules, and that this information can be retrieved using machine learning techniques that are able to disentangle the contribution of chemical composition, bond topology, and internal torsion of the molecule to the HR-AFM contrast. In particular, we exploit multimodal recurrent neural networks (M-RNN) that combine convolutional neural networks for image analysis and recurrent neural networks to deal with language processing, to formulate the molecular identification as an imaging captioning problem. The algorithm is trained using a data set─which contains almost 700,000 molecules and 165 million theoretical AFM images─to produce as final output the IUPAC name of the imaged molecule. Our extensive test with theoretical images and a few experimental ones shows the potential of deep learning algorithms in the automatic identification of molecular compounds by AFM. This achievement supports the development of on-surface synthesis and overcomes some limitations of spectroscopic methods in traditional solution-based synthesis.

QUAM-AFM: A Free Database for Molecular Identification by Atomic Force Microscopy

This paper introduces Quasar Science Resources-Autonomous University of Madrid atomic force microscopy image data set (QUAM-AFM), the largest data set of simulated atomic force microscopy (AFM) images generated from a selection of 685,513 molecules that span the most relevant bonding structures and chemical species in organic chemistry. QUAM-AFM contains, for each molecule, 24 3D image stacks, each consisting of constant-height images simulated for 10 tip-sample distances with a different combination of AFM operational parameters, resulting in a total of 165 million images with a resolution of 256 × 256 pixels. The 3D stacks are especially appropriate to tackle the goal of the chemical identification within AFM experiments by using deep learning techniques. The data provided for each molecule include, besides a set of AFM images, ball-and-stick depictions, IUPAC names, chemical formulas, atomic coordinates, and map of atom heights. In order to simplify the use of the collection as a source of information, we have developed a graphical user interface that allows the search for structures by CID number, IUPAC name, or chemical formula.