Revealing ferroelectric switching character using deep recurrent neural networks

Fcg-Former: Identification of Functional Groups in FTIR Spectra Using Enhanced Transformer-Based Model

Deep learning (DL) is becoming more popular as a useful tool in various scientific domains, especially in chemistry applications. In the infrared spectroscopy field, where identifying functional groups in unknown compounds poses a significant challenge, there is a growing need for innovative approaches to streamline and enhance analysis processes. This study introduces a transformative approach leveraging a DL methodology based on transformer attention models. With a data set containing approximately 8677 spectra, our model utilizes self-attention mechanisms to capture complex spectral features and precisely predict 17 functional groups, outperforming conventional architectures in both functional group prediction accuracy and compound-level precision. The success of our approach underscores the potential of transformer-based methodologies in enhancing spectral analysis techniques.

Automated Experiments of Local Non-Linear Behavior in Ferroelectric Materials

An automated experiment in multimodal imaging to probe structural, chemical, and functional behaviors in complex materials and elucidate the dominant physical mechanisms that control device function is developed and implemented. Here, the emergence of non-linear electromechanical responses in piezoresponse force microscopy (PFM) is explored. Non-linear responses in PFM can originate from multiple mechanisms, including intrinsic material responses often controlled by domain structure, surface topography that affects the mechanical phenomena at the tip-surface junction, and the presence of surface contaminants. Using an automated experiment to probe the origins of non-linear behavior in ferroelectric lead titanate (PTO) and ferroelectric Al_0.93 B_0.07 N films, it is found that PTO shows asymmetric nonlinear behavior across a/c domain walls and a broadened high nonlinear response region around c/c domain walls. In contrast, for Al_0.93 B_0.07 N, well-poled regions show high linear piezoelectric responses, when paired with low non-linear responses regions that are multidomain show low linear responses and high nonlinear responses. It is shown that formulating dissimilar exploration strategies in deep kernel learning as alternative hypotheses allows for establishing the preponderant physical mechanisms behind the non-linear behaviors, suggesting that automated experiments can potentially discern between competing physical mechanisms. This technique can also be extended to electron, probe, and chemical imaging.

Exploring Physics of Ferroelectric Domain Walls in Real Time: Deep Learning Enabled Scanning Probe Microscopy

The functionality of ferroelastic domain walls in ferroelectric materials is explored in real-time via the in situ implementation of computer vision algorithms in scanning probe microscopy (SPM) experiment. The robust deep convolutional neural network (DCNN) is implemented based on a deep residual learning framework (Res) and holistically nested edge detection (Hed), and ensembled to minimize the out-of-distribution drift effects. The DCNN is implemented for real-time operations on SPM, converting the data stream into the semantically segmented image of domain walls and the corresponding uncertainty. Further the pre-defined experimental workflows perform piezoresponse spectroscopy measurement on thus discovered domain walls, and alternating high- and low-polarization dynamic (out-of-plane) ferroelastic domain walls in a PbTiO3 (PTO) thin film and high polarization dynamic (out-of-plane) at short ferroelastic walls (compared with long ferroelastic walls) in a lead zirconate titanate (PZT) thin film is reported. This work establishes the framework for real-time DCNN analysis of data streams in scanning probe and other microscopies and highlights the role of out-of-distribution effects and strategies to ameliorate them in real time analytics.

Why it is Unfortunate that Linear Machine Learning "Works" so well in Electromechanical Switching of Ferroelectric Thin Films

Machine learning (ML) is relied on for materials spectroscopy. It is challenging to make ML models fail because statistical correlations can mimic the physics without causality. Here, using a benchmark band-excitation piezoresponse force microscopy polarization spectroscopy (BEPS) dataset the pitfalls of the so-called "better", "faster", and "less-biased" ML of electromechanical switching are demonstrated and overcome. Using a toy and real experimental dataset, it is demonstrated how linear nontemporal ML methods result in physically reasonable embedding (eigenvalues) while producing nonsensical eigenvectors and generated spectra, promoting misleading interpretations. A new method of unsupervised multimodal hyperspectral analysis of BEPS is demonstrated using long-short-term memory (LSTM) β-variational autoencoders (β-VAEs) . By including LSTM neurons, the ordinal nature of ferroelectric switching is considered. To improve the interpretability of the latent space, a variational Kullback-Leibler-divergency regularization is imposed . Finally, regularization scheduling of β as a disentanglement metric is leveraged to reduce user bias. Combining these experiment-inspired modifications enables the automated detection of ferroelectric switching mechanisms, including a complex two-step, three-state one. Ultimately, this work provides a robust ML method for the rapid discovery of electromechanical switching mechanisms in ferroelectrics and is applicable to other multimodal hyperspectral materials spectroscopies.

High Velocity, Low-Voltage Collective In-Plane Switching in (100) BaTiO3 Thin Films

Ferroelectrics are being increasingly called upon for electronic devices in extreme environments. Device performance and energy efficiency is highly correlated to clock frequency, operational voltage, and resistive loss. To increase performance it is common to engineer ferroelectric domain structure with highly-correlated electrical and elastic coupling that elicit fast and efficient collective switching. Designing domain structures with advantageous properties is difficult because the mechanisms involved in collective switching are poorly understood and difficult to investigate. Collective switching is a hierarchical process where the nano- and mesoscale responses control the macroscopic properties. Using chemical solution synthesis, epitaxially nearly-relaxed (100) BaTiO₃ films are synthesized. Thermal strain induces a strongly-correlated domain structure with alternating domains of polarization along the [010] and [001] in-plane axes and 90° domain walls along the [011] or [01 1 ¯ $\bar{1}$ ] directions. Simultaneous capacitance-voltage measurements and band-excitation piezoresponse force microscopy revealed strong collective switching behavior. Using a deep convolutional autoencoder, hierarchical switching is automatically tracked and the switching pathway is identified. The collective switching velocities are calculated to be ≈500 cm s^-1 at 5 V (7 kV cm^-1 ), orders-of-magnitude faster than expected. These combinations of properties are promising for high-speed tunable dielectrics and low-voltage ferroelectric memories and logic.

A Robust Neural Network for Extracting Dynamics from Electrostatic Force Microscopy Data

Advances in scanning probe microscopy (SPM) methods such as time-resolved electrostatic force microscopy (trEFM) now permit the mapping of fast local dynamic processes with high resolution in both space and time, but such methods can be time-consuming to analyze and calibrate. Here, we design and train a regression neural network (NN) that accelerates and simplifies the extraction of local dynamics from SPM data directly in a cantilever-independent manner, allowing the network to process data taken with different cantilevers. We validate the NN's ability to recover local dynamics with a fidelity equal to or surpassing conventional, more time-consuming, calibrations using both simulated and real microscopy data. We apply this method to extract accurate photoinduced carrier dynamics on n = 1 butylammonium lead iodide, a halide perovskite semiconductor film that is of interest for applications in both solar photovoltaics and quantum light sources. Finally, we use SHapley Additive exPlanations to evaluate the robustness of the trained model, confirm its cantilever-independence, and explore which parts of the trEFM signal are important to the network.

Applications and Techniques for Fast Machine Learning in Science

In this community review report, we discuss applications and techniques for fast machine learning (ML) in science-the concept of integrating powerful ML methods into the real-time experimental data processing loop to accelerate scientific discovery. The material for the report builds on two workshops held by the Fast ML for Science community and covers three main areas: applications for fast ML across a number of scientific domains; techniques for training and implementing performant and resource-efficient ML algorithms; and computing architectures, platforms, and technologies for deploying these algorithms. We also present overlapping challenges across the multiple scientific domains where common solutions can be found. This community report is intended to give plenty of examples and inspiration for scientific discovery through integrated and accelerated ML solutions. This is followed by a high-level overview and organization of technical advances, including an abundance of pointers to source material, which can enable these breakthroughs.

Decomposing and analyzing contact resonance frequency in contact mode voltage modulated scanning probe microscopies

Contact mode voltage modulated scanning probe microscopy (SPM) techniques, such as switching spectroscopy piezoresponse force microscopy (SS-PFM), are powerful tools for detecting local electromechanical behaviors. However, interpreting their signals, especially the contact resonance frequency (f₀), is difficult due to the complexity of the tip-sample contact and the signal tracking system. In a previous study, an interesting phenomenon on the variation of f₀ was observed, showing distinctions between ferroelectric and non-ferroelectric materials. Therefore, it is believed that f₀ conveys important information about the tip-sample contact which is real-timely affected by charge accumulation or polarization switching. In this study, principal component analysis (PCA) and correlation analysis are applied to decompose the signal of f₀. The first and second principal components successfully restore most information of the original data, and they describe different features which are related to the surface morphology and electromechanical behavior, respectively. In addition, a customized method based on the PCA and correlation analysis of f₀ can well distinguish the responses from different materials of which the amplitude and phase signals show "ferroelectric like" phenomena during the SS-PFM measurements.

Integrating mechanistic and deep learning models for accurately predicting the enrichment of polyhydroxyalkanoates accumulating bacteria in mixed microbial cultures

The enrichment of polyhydroxyalkanoates (PHA) accumulating bacteria (PAB) in mixed microbial cultures (MMC) is extremely difficult to be predicted and optimized. Here we demonstrate that mechanistic and deep learning models can be integrated innovatively to accurately predict the dynamic enrichment of PAB. Well-calibrated activated sludge models (ASM) of the PAB enrichment process provide time-dependent data under different operating conditions. Recurrent neural network (RNN) models are trained and tested based on the time-dependent dataset generated by ASM. The accurate prediction performance is achieved (R² > 0.991) for three different PAB enrichment datasets by the optimized RNN model. The optimized RNN model can also predict the equilibrium concentration of PAB (R² = 0.944) and corresponding time, which represents the end of the PAB enrichment process. This study demonstrates the strength of integrating mechanistic and deep learning models to predict long-term variations of specific microbes, helping to optimize their selection process for PHA production.

Artificial intelligence: machine learning for chemical sciences

Research in molecular sciences witnessed the rise and fall of Artificial Intelligence (AI)/ Machine Learning (ML) methods, especially artificial neural networks, few decades ago. However, we see a major resurgence in the use of modern ML methods in scientific research during the last few years. These methods have had phenomenal success in the areas of computer vision, speech recognition, natural language processing (NLP), etc. This has inspired chemists and biologists to apply these algorithms to problems in natural sciences. Availability of high performance Graphics Processing Unit (GPU) accelerators, large datasets, new algorithms, and libraries has enabled this surge. ML algorithms have successfully been applied to various domains in molecular sciences by providing much faster and sometimes more accurate solutions compared to traditional methods like Quantum Mechanical (QM) calculations, Density Functional Theory (DFT) or Molecular Mechanics (MM) based methods, etc. Some of the areas where the potential of ML methods are shown to be effective are in drug design, prediction of high-level quantum mechanical energies, molecular design, molecular dynamics materials, and retrosynthesis of organic compounds, etc. This article intends to conceptually introduce various modern ML methods and their relevance and applications in computational natural sciences. Synopsis Recent surge in the application of machine learning (ML) methods in fundamental sciences has led to a perspective that these methods may become important tools in chemical science. This perspective provides an overview of the modern ML methods and their successful applications in chemistry during the last few years.

Maximizing Information: A Machine Learning Approach for Analysis of Complex Nanoscale Electromechanical Behavior in Defect-Rich PZT Films

Scanning Probe Microscopy (SPM) based techniques probe material properties over microscale regions with nanoscale resolution, ultimately resulting in investigation of mesoscale functionalities. Among SPM techniques, piezoresponse force microscopy (PFM) is a highly effective tool in exploring polarization switching in ferroelectric materials. However, its signal is also sensitive to sample-dependent electrostatic and chemo-electromechanical changes. Literature reports have often concentrated on the evaluation of the Off-field piezoresponse, compared to On-field piezoresponse, based on the latter's increased sensitivity to non-ferroelectric contributions. Using machine learning approaches incorporating both Off- and On-field piezoresponse response as well as Off-field resonance frequency to maximize information, switching piezoresponse in a defect-rich Pb(Zr,Ti)O₃ thin film is investigated. As expected, one major contributor to the piezoresponse is mostly ferroelectric, coupled with electrostatic phenomena during On-field measurements. A second component is electrostatic in nature, while a third component is likely due to a superposition of multiple non-ferroelectric processes. The proposed approach will enable deeper understanding of switching phenomena in weakly ferroelectric samples and materials with large chemo-electromechanical response.

Bayesian Learning of Adatom Interactions from Atomically Resolved Imaging Data

Atomic structures and adatom geometries of surfaces encode information about the thermodynamics and kinetics of the processes that lead to their formation, and which can be captured by a generative physical model. Here we develop a workflow based on a machine-learning-based analysis of scanning tunneling microscopy images to reconstruct the atomic and adatom positions, and a Bayesian optimization procedure to minimize statistical distance between the chosen physical models and experimental observations. We optimize the parameters of a 2- and 3-parameter Ising model describing surface ordering and use the derived generative model to make predictions across the parameter space. For concentration dependence, we compare the predicted morphologies at different adatom concentrations with the dissimilar regions on the sample surfaces that serendipitously had different adatom concentrations. The proposed workflow can be used to reconstruct the thermodynamic models and associated uncertainties from the experimental observations of materials microstructures. The code used in the manuscript is available at https://github.com/saimani5/Adatom_interactions.

Reducing Time to Discovery: Materials and Molecular Modeling, Imaging, Informatics, and Integration

Multiscale and multimodal imaging of material structures and properties provides solid ground on which materials theory and design can flourish. Recently, KAIST announced 10 flagship research fields, which include KAIST Materials Revolution: Materials and Molecular Modeling, Imaging, Informatics and Integration (M3I3). The M3I3 initiative aims to reduce the time for the discovery, design and development of materials based on elucidating multiscale processing-structure-property relationship and materials hierarchy, which are to be quantified and understood through a combination of machine learning and scientific insights. In this review, we begin by introducing recent progress on related initiatives around the globe, such as the Materials Genome Initiative (U.S.), Materials Informatics (U.S.), the Materials Project (U.S.), the Open Quantum Materials Database (U.S.), Materials Research by Information Integration Initiative (Japan), Novel Materials Discovery (E.U.), the NOMAD repository (E.U.), Materials Scientific Data Sharing Network (China), Vom Materials Zur Innovation (Germany), and Creative Materials Discovery (Korea), and discuss the role of multiscale materials and molecular imaging combined with machine learning in realizing the vision of M3I3. Specifically, microscopies using photons, electrons, and physical probes will be revisited with a focus on the multiscale structural hierarchy, as well as structure-property relationships. Additionally, data mining from the literature combined with machine learning will be shown to be more efficient in finding the future direction of materials structures with improved properties than the classical approach. Examples of materials for applications in energy and information will be reviewed and discussed. A case study on the development of a Ni-Co-Mn cathode materials illustrates M3I3's approach to creating libraries of multiscale structure-property-processing relationships. We end with a future outlook toward recent developments in the field of M3I3.

Toward Decoding the Relationship between Domain Structure and Functionality in Ferroelectrics via Hidden Latent Variables

Polarization switching mechanisms in ferroelectric materials are fundamentally linked to local domain structure and the presence of the structural defects, which both can act as nucleation and pinning centers and create local electrostatic and mechanical depolarization fields affecting wall dynamics. However, the general correlative mechanisms between domain structure and polarization dynamics are only weakly explored, precluding insight into the associated physical mechanisms. Here, the correlation between local domain structures and switching behavior in ferroelectric materials is explored using convolutional encoder-decoder networks, enabling image to spectral (im2spec) and spectral to image (spec2im) translations via encoding of latent variables. The latter reflect the assumption that the relationship between domain structure and polarization switching is parsimonious, i.e., is based upon a small number of local mechanisms. The analysis of latent variables distributions and their real-space representations provides insight into the predictability of the local switching behavior and hence associated physical mechanisms. We further pose that the regions where these correlative relationships are violated, i.e., predictability of the polarization dynamics from domain structure is reduced, represent the obvious target for detailed studies, e.g., in the context of automated experiments. This approach provides a workflow to establish the presence of correlation between local spectral responses and local structure and can be universally applied to spectral imaging techniques such as piezoresponse force microscopy (PFM), scanning tunneling microscopy (STM) and spectroscopy, and electron energy loss spectroscopy (EELS) in scanning transmission electron microscopy (STEM).

Multiferroic heterostructures for spintronics

For next-generation technology, magnetic systems are of interest due to the natural ability to store information and, through spin transport, propagate this information for logic functions. Controlling the magnetization state through currents has proven energy inefficient. Multiferroic thin-film heterostructures, combining ferroelectric and ferromagnetic orders, hold promise for energy efficient electronics. The electric field control of magnetic order is expected to reduce energy dissipation by 2–3 orders of magnitude relative to the current state-of-the-art. The coupling between electrical and magnetic orders in multiferroic and magnetoelectric thin-film heterostructures relies on interfacial coupling though magnetic exchange or mechanical strain and the correlation between domains in adjacent functional ferroic layers. We review the recent developments in electrical control of magnetism through artificial magnetoelectric heterostructures, domain imprint, emergent physics and device paradigms for magnetoelectric logic, neuromorphic devices, and hybrid magnetoelectric/spin-current-based applications. Finally, we conclude with a discussion of experiments that probe the crucial dynamics of the magnetoelectric switching and optical tuning of ferroelectric states towards all-optical control of magnetoelectric switching events.

Learning molecular dynamics with simple language model built upon long short-term memory neural network

Recurrent neural networks have led to breakthroughs in natural language processing and speech recognition. Here we show that recurrent networks, specifically long short-term memory networks can also capture the temporal evolution of chemical/biophysical trajectories. Our character-level language model learns a probabilistic model of 1-dimensional stochastic trajectories generated from higher-dimensional dynamics. The model captures Boltzmann statistics and also reproduces kinetics across a spectrum of timescales. We demonstrate how training the long short-term memory network is equivalent to learning a path entropy, and that its embedding layer, instead of representing contextual meaning of characters, here exhibits a nontrivial connectivity between different metastable states in the underlying physical system. We demonstrate our model's reliability through different benchmark systems and a force spectroscopy trajectory for multi-state riboswitch. We anticipate that our work represents a stepping stone in the understanding and use of recurrent neural networks for understanding the dynamics of complex stochastic molecular systems.

Better, Faster, and Less Biased Machine Learning: Electromechanical Switching in Ferroelectric Thin Films

Machine-learning techniques are more and more often applied to the analysis of complex behaviors in materials research. Frequently used to identify fundamental behaviors within large and multidimensional datasets, these techniques are strictly based on mathematical models. Thus, without inherent physical or chemical meaning or constraints, they are prone to biased interpretation. The interpretability of machine-learning results in materials science, specifically materials' functionalities, can be vastly improved through physical insights and careful data handling. The use of techniques such as dimensional stacking can provide the much needed physical and chemical constraints, while proper understanding of the assumptions imposed by model parameters can help avoid overinterpretation. These concepts are illustrated by application to recently reported ferroelectric switching experiments in PbZr_0.2 Ti_0.8 O₃ thin films. Through systematic analysis and introduction of physical constraints, it is argued that the behaviors present are not necessarily due to exotic mechanisms previously suggested, but rather well described by classical ferroelectric switching superimposed by non-ferroelectric phenomena, such as electrochemical deformation, electrostatic interactions, and/or charge injection.