A Deep Learning Approach for Molecular Classification Based on AFM Images

Automated Bio-AFM Generation of Large Mechanome Data Set and Their Analysis by Machine Learning to Classify Cancerous Cell Lines

Mechanobiological measurements have the potential to discriminate healthy cells from pathological cells. However, a technology frequently used to measure these properties, i.e., atomic force microscopy (AFM), suffers from its low output and lack of standardization. In this work, we have optimized AFM mechanical measurement on cell populations and developed a technology combining cell patterning and AFM automation that has the potential to record data on hundreds of cells (956 cells measured for publication). On each cell, 16 force curves (FCs) and seven features/FC, constituting the mechanome, were calculated. All of the FCs were then classified using machine learning tools with a statistical approach based on a fuzzy logic algorithm, trained to discriminate between nonmalignant and cancerous cells (training base, up to 120 cells/cell line). The proof of concept was first made on prostate nonmalignant (RWPE-1) and cancerous cell lines (PC3-GFP), then on nonmalignant (Hs 895.Sk) and cancerous (Hs 895.T) skin fibroblast cell lines, and demonstrated the ability of our method to classify correctly 73% of the cells (194 cells in the database/cell line) despite the very high degree of similarity of the whole set of measurements (79-100% similarity).

Possibilities and Limitations of Kinetic Studies in On-Surface Synthesis by Real Time X-ray Photoelectron Spectroscopy

The kinetics of coupling reactions on surfaces can be quantitatively studied in real time by X-ray Photoelectron Spectroscopy (XPS). From fitting experimental data, kinetic reaction parameters such as the rate constant's pre-exponential and activation energy can be deduced and compared to quantum chemical simulations. To elucidate the possibilities and limitations of this approach, we propose studies in which experimental data are first simulated and subsequently fitted. Knowing the exact kinetic parameters used in the simulation allows one to evaluate the accuracy of the fit result. Here, several experimental influences, such as the data point density and the addition of noise, are explored for a model reaction with first-order kinetics. The proposed procedure sheds light on the accuracy with which kinetic parameters can be derived and may also help in the design of future experiments.

Automated Structure Discovery for Scanning Tunneling Microscopy

Scanning tunneling microscopy (STM) with a functionalized tip apex reveals the geometric and electronic structures of a sample within the same experiment. However, the complex nature of the signal makes images difficult to interpret and has so far limited most research to planar samples with a known chemical composition. Here, we present automated structure discovery for STM (ASD-STM), a machine learning tool for predicting the atomic structure directly from an STM image, by building upon successful methods for structure discovery in noncontact atomic force microscopy (nc-AFM). We apply the method on various organic molecules and achieve good accuracy on structure predictions and chemical identification on a qualitative level while highlighting future development requirements for ASD-STM. This method is directly applicable to experimental STM images of organic molecules, making structure discovery available for a wider scanning probe microscopy audience outside of nc-AFM. This work also allows more advanced machine learning methods to be developed for STM structure discovery.

On machine learning analysis of atomic force microscopy images for image classification, sample surface recognition

Atomic force microscopy (AFM or SPM) imaging is one of the best matches with machine learning (ML) analysis among microscopy techniques. The digital format of AFM images allows for direct utilization in ML algorithms without the need for additional processing. Additionally, AFM enables the simultaneous imaging of distributions of over a dozen different physicochemical properties of sample surfaces, a process known as multidimensional imaging. While this wealth of information can be challenging to analyze using traditional methods, ML provides a seamless approach to this task. However, the relatively slow speed of AFM imaging poses a challenge in applying deep learning methods broadly used in image recognition. This prospective is focused on ML recognition/classification when using a relatively small number of AFM images, aka small database. We discuss ML methods other than popular deep-learning neural networks. The described approach has already been successfully used to analyze and classify the surfaces of biological cells. It can be applied to recognize medical images, specific material processing, in forensic studies, even to identify the authenticity of arts. A general template for ML analysis specific to AFM is suggested, with a specific example of the identification of cell phenotype. Special attention is given to the analysis of the statistical significance of the obtained results, an important feature that is often overlooked in papers dealing with machine learning. A simple method for finding statistical significance is also described.

Precise Surface Profiling at the Nanoscale Enabled by Deep Learning

Surface topography, or height profile, is a critical property for various micro- and nanostructured materials and devices, as well as biological systems. At the nanoscale, atomic force microscopy (AFM) is the tool of choice for surface profiling due to its capability to noninvasively map the topography of almost all types of samples. However, this method suffers from one drawback: the convolution of the nanoprobe's shape in the height profile of the samples, which is especially severe for sharp protrusion features. Here, we report a deep learning (DL) approach to overcome this limit. Adopting an image-to-image translation methodology, we use data sets of tip-convoluted and deconvoluted image pairs to train an encoder-decoder based deep convolutional neural network. The trained network successfully removes the tip convolution from AFM topographic images of various nanocorrugated surfaces and recovers the true, precise 3D height profiles of these samples.

Machine learning-aided atomic structure identification of interfacial ionic hydrates from AFM images

Relevant to broad applied fields and natural processes, interfacial ionic hydrates have been widely studied by using ultrahigh-resolution atomic force microscopy (AFM). However, the complex relationship between the AFM signal and the investigated system makes it difficult to determine the atomic structure of such a complex system from AFM images alone. Using machine learning, we achieved precise identification of the atomic structures of interfacial water/ionic hydrates based on AFM images, including the position of each atom and the orientations of water molecules. Furthermore, it was found that structure prediction of ionic hydrates can be achieved cost-effectively by transfer learning using neural network trained with easily available interfacial water data. Thus, this work provides an efficient and economical methodology that not only opens up avenues to determine atomic structures of more complex systems from AFM images, but may also help to interpret other scientific studies involving sophisticated experimental results.

Molecular Identification from AFM Images Using the IUPAC Nomenclature and Attribute Multimodal Recurrent Neural Networks

Spectroscopic methods─like nuclear magnetic resonance, mass spectrometry, X-ray diffraction, and UV/visible spectroscopies─applied to molecular ensembles have so far been the workhorse for molecular identification. Here, we propose a radically different chemical characterization approach, based on the ability of noncontact atomic force microscopy with metal tips functionalized with a CO molecule at the tip apex (referred as HR-AFM) to resolve the internal structure of individual molecules. Our work demonstrates that a stack of constant-height HR-AFM images carries enough chemical information for a complete identification (structure and composition) of quasiplanar organic molecules, and that this information can be retrieved using machine learning techniques that are able to disentangle the contribution of chemical composition, bond topology, and internal torsion of the molecule to the HR-AFM contrast. In particular, we exploit multimodal recurrent neural networks (M-RNN) that combine convolutional neural networks for image analysis and recurrent neural networks to deal with language processing, to formulate the molecular identification as an imaging captioning problem. The algorithm is trained using a data set─which contains almost 700,000 molecules and 165 million theoretical AFM images─to produce as final output the IUPAC name of the imaged molecule. Our extensive test with theoretical images and a few experimental ones shows the potential of deep learning algorithms in the automatic identification of molecular compounds by AFM. This achievement supports the development of on-surface synthesis and overcomes some limitations of spectroscopic methods in traditional solution-based synthesis.

How lignin sticks to cellulose-insights from atomic force microscopy enhanced by machine-learning analysis and molecular dynamics simulations

Elucidating cellulose-lignin interactions at the molecular and nanometric scales is an important research topic with impacts on several pathways of biomass valorization. Here, the interaction forces between a cellulosic substrate and lignin are investigated. Atomic force microscopy with lignin-coated tips is employed to probe the site-specific adhesion to a cellulose film in liquid water. Over seven thousand force-curves are analyzed by a machine-learning approach to cluster the experimental data into types of cellulose-tip interactions. The molecular mechanisms for distinct types of cellulose-lignin interactions are revealed by molecular dynamics simulations of lignin globules interacting with different cellulose Iβ crystal facets. This unique combination of experimental force-curves, data-driven analysis, and molecular simulations opens a new approach of investigation and updates the understanding of cellulose-lignin interactions at the nanoscale.

QUAM-AFM: A Free Database for Molecular Identification by Atomic Force Microscopy

This paper introduces Quasar Science Resources-Autonomous University of Madrid atomic force microscopy image data set (QUAM-AFM), the largest data set of simulated atomic force microscopy (AFM) images generated from a selection of 685,513 molecules that span the most relevant bonding structures and chemical species in organic chemistry. QUAM-AFM contains, for each molecule, 24 3D image stacks, each consisting of constant-height images simulated for 10 tip-sample distances with a different combination of AFM operational parameters, resulting in a total of 165 million images with a resolution of 256 × 256 pixels. The 3D stacks are especially appropriate to tackle the goal of the chemical identification within AFM experiments by using deep learning techniques. The data provided for each molecule include, besides a set of AFM images, ball-and-stick depictions, IUPAC names, chemical formulas, atomic coordinates, and map of atom heights. In order to simplify the use of the collection as a source of information, we have developed a graphical user interface that allows the search for structures by CID number, IUPAC name, or chemical formula.