Self-supervised learning and prediction of microstructure evolution with convolutional recurrent neural networks

Ultrafast and accurate prediction of polycrystalline hafnium oxide phase-field ferroelectric hysteresis using graph neural networks

Polycrystalline hafnium oxide emerges as a promising material for the future of nanoelectronic devices. While phase-field modeling stands as a primary choice tool for forecasting domain structure evolution and electromechanical properties of ferroelectric materials, it suffers from a high computational cost, which impedes its applicability to real-size systems. Here, we propose a Graph Neural Network (GNN) machine-learning framework to predict the ferroelectric hysteresis of polycrystalline hafnium oxide, with the goal of significantly accelerating computations in contrast to high-fidelity phase-field methods. By leveraging the inherent graph structure of the polycrystalline system and incorporating edge-level feature properties through graph attentional layers, our approach accurately predicts hysteresis behaviors across a broad range of polycrystalline structures, grain numbers, and Landau coefficients. The GNN framework exhibits high accuracy, with an average relative error of ∼4%, and demonstrates remarkable computational efficiency with respect to ground truth phase-field simulations, offering speed-ups exceeding a million-fold. Furthermore, we showcase the transferability of our model to efficiently scale predictions in polycrystals comprising up to a thousand grains, paving the way for effective simulations of real-sized systems. Our approach, by overcoming computational limitations in polycrystalline hafnium oxide, opens doors for accelerating discovery and design in ferroelectric materials.

Score Dynamics: Scaling Molecular Dynamics with Picoseconds Time Steps via Conditional Diffusion Model

We propose score dynamics (SD), a general framework for learning accelerated evolution operators with large timesteps from molecular dynamics (MD) simulations. SD is centered around scores or derivatives of the transition log-probability with respect to the dynamical degrees of freedom. The latter play the same role as force fields in MD but are used in denoising diffusion probability models to generate discrete transitions of the dynamical variables in an SD time step, which can be orders of magnitude larger than a typical MD time step. In this work, we construct graph neural network-based SD models of realistic molecular systems that are evolved with 10 ps timesteps. We demonstrate the efficacy of SD with case studies of the alanine dipeptide and short alkanes in aqueous solution. Both equilibrium predictions derived from the stationary distributions of the conditional probability and kinetic predictions for the transition rates and transition paths are in good agreement with MD. Our current SD implementation is about 2 orders of magnitude faster than the MD counterpart for the systems studied in this work. Open challenges and possible future remedies to improve SD are also discussed.

Accurate generation of stochastic dynamics based on multi-model generative adversarial networks

Generative Adversarial Networks (GANs) have shown immense potential in fields such as text and image generation. Only very recently attempts to exploit GANs to statistical-mechanics models have been reported. Here we quantitatively test this approach by applying it to a prototypical stochastic process on a lattice. By suitably adding noise to the original data we succeed in bringing both the Generator and the Discriminator loss functions close to their ideal value. Importantly, the discreteness of the model is retained despite the noise. As typical for adversarial approaches, oscillations around the convergence limit persist also at large epochs. This undermines model selection and the quality of the generated trajectories. We demonstrate that a simple multi-model procedure where stochastic trajectories are advanced at each step upon randomly selecting a Generator leads to a remarkable increase in accuracy. This is illustrated by quantitative analysis of both the predicted equilibrium probability distribution and of the escape-time distribution. Based on the reported findings, we believe that GANs are a promising tool to tackle complex statistical dynamics by machine learning techniques.

Deep Learning Model to Predict Ice Crystal Growth

The demand for highly specific and complex materials has made the development of controllable manufacturing processes crucial. Among the numerous manufacturing methods, casting is important because it is economical and highly flexible regarding the geometry of manufactured parts. Since solidification is an important stage in the casting process that influences the properties of the final product, the development of a controllable solidification process using modeling methods is necessary to create superior structural properties. However, traditional modeling methods are computationally expensive and require sophisticated mathematical schemes. Therefore, a deep learning model is proposed to predict the morphology of the dendritic crystal growth solidification process, along with a reinforcement learning model to control the solidification process. By training the deep learning model with data generated using the phase field method, the solidification process can be successfully predicted. The crystal growth structures are designed to be altered by adjusting the degree of supercooling in the deep learning model while implementing reinforcement learning to control the dendritic arteries. This research opens new avenues for applying artificial intelligence to the optimization of casting processes, with the potential to utilize it in the processing of advanced materials and to improve the target properties of material design.

Machine learning unifies flexibility and efficiency of spinodal structure generation for stochastic biomaterial design

Porous biomaterials design for bone repair is still largely limited to regular structures (e.g. rod-based lattices), due to their easy parameterization and high controllability. The capability of designing stochastic structure can redefine the boundary of our explorable structure-property space for synthesizing next-generation biomaterials. We hereby propose a convolutional neural network (CNN) approach for efficient generation and design of spinodal structure-an intriguing structure with stochastic yet interconnected, smooth, and constant pore channel conducive to bio-transport. Our CNN-based approach simultaneously possesses the tremendous flexibility of physics-based model in generating various spinodal structures (e.g. periodic, anisotropic, gradient, and arbitrarily large ones) and comparable computational efficiency to mathematical approximation model. We thus successfully design spinodal bone structures with target anisotropic elasticity via high-throughput screening, and directly generate large spinodal orthopedic implants with desired gradient porosity. This work significantly advances stochastic biomaterials development by offering an optimal solution to spinodal structure generation and design.

New Platforms Based on Frontal Cellular Automata and Lattice Boltzmann Method for Modeling the Forming and Additive Manufacturing

Materials science gives theoretical and practical tools, while new modeling methods and platforms provide rapid and efficient development, improvement, and optimization of old and new technologies. Recently, impressive progress has been made in the development of computer software and systems. The frontal cellular automata (FCA), lattice Boltzmann method (LBM), and modeling platforms based on them are considered in the paper. The paper presents basic information on these methods and their application for modeling phenomena and processes in materials science. Recrystallization, crystallization, phase transformation, processes such as flat and shape rolling, additive manufacturing technologies (Selective Laser Sintering (SLS)/ Selective Laser Melting (SLM)), and others are examples of comprehensive and effective modeling by the developed systems. Selected modeling results are also presented.

Machine-Learning-Based Predictions of Polymer and Postconsumer Recycled Polymer Properties: A Comprehensive Review

There has been a tremendous increase in demand for virgin and postconsumer recycled (PCR) polymers due to their wide range of chemical and physical characteristics. Despite the numerous potential benefits of using a data-driven approach to polymer design, major hurdles exist in the development of polymer informatics due to the complicated hierarchical polymer structures. In this review, a brief introduction on virgin polymer structure, PCR polymers, compatibilization of polymers to be recycled, and their characterization using sensor array technologies as well as factors affecting the polymer properties are provided. Machine-learning (ML) algorithms are gaining attention as cost-effective scalable solutions to exploit the physical and chemical structures of polymers. The basic steps for applying ML in polymer science such as fingerprinting, algorithms, open-source databases, representations, and polymer design are detailed in this review. Further, a state-of-the-art review of the prediction of various polymer material properties using ML is reviewed. Finally, we discuss open-ended research questions on ML application to PCR polymers as well as potential challenges in the prediction of their properties using artificial intelligence for more efficient and targeted PCR polymer discovery and development.

Forecasting molecular dynamics energetics of polymers in solution from supervised machine learning

Machine learning techniques including neural networks are popular tools for chemical, physical and materials applications searching for viable alternative methods in the analysis of structure and energetics of systems ranging from crystals to biomolecules. Efforts are less abundant for prediction of kinetics and dynamics. Here we explore the ability of three well established recurrent neural network architectures for reproducing and forecasting the energetics of a liquid solution of ethyl acetate containing a macromolecular polymer-lipid aggregate at ambient conditions. Data models from three recurrent neural networks, ERNN, LSTM and GRU, are trained and tested on half million points time series of the macromolecular aggregate potential energy and its interaction energy with the solvent obtained from molecular dynamics simulations. Our exhaustive analyses convey that the recurrent neural network architectures investigated generate data models that reproduce excellently the time series although their capability of yielding short or long term energetics forecasts with expected statistical distributions of the time points is limited. We propose an in silico protocol by extracting time patterns of the original series and utilizing these patterns to create an ensemble of artificial network models trained on an ensemble of time series seeded by the additional time patters. The energetics forecast improve, predicting a band of forecasted time series with a spread of values consistent with the molecular dynamics energy fluctuations span. Although the distribution of points from the band of energy forecasts is not optimal, the proposed in silico protocol provides useful estimates of the solvated macromolecular aggregate fate. Given the growing application of artificial networks in materials design, the data-based protocol presented here expands the realm of science areas where supervised machine learning serves as a decision making tool aiding the simulation practitioner to assess when long simulations are worth to be continued.

Multi-input convolutional network for ultrafast simulation of field evolvement

There is a compelling need for the regression capability of mapping the initial field and applied conditions to the evolved field, e.g., given current flow field and fluid properties predicting next-step flow field. Such a capability can provide a maximum to full substitute of a physics-based model, enabling fast simulation of various field evolvements. We propose a conceptually simple, lightweight, but powerful multi-input convolutional network (ConvNet), yNet, that merges multi-input signals by manipulating high-level encodings of field/image input. yNet can significantly reduce the model size compared with its ConvNet counterpart (e.g., to only one-tenth for main architecture of 38-layer depth) and is as much as six orders of magnitude faster than a physics-based model. yNet is applied for data-driven modeling of fluid dynamics, porosity evolution in sintering, stress field development, and grain growth. It consistently shows great extrapolative prediction beyond training datasets in terms of temporal ranges, spatial domains, and geometrical shapes.

•

A multi-input convolutional network is proposed to model evolution of physical fields

•

The lightweight model shows general effectiveness in four diverse applications

•

It also displays good extrapolative prediction beyond training datasets

•

Full-component selective laser sintering and large grain growth modeling are given

In physical sciences and engineering, the convolutional network (ConvNet) has been used increasingly to simulate the evolvement of physical fields, e.g., flow field evolvement. Physical field data are fed as images, and ConvNet treats the field evolvement as a field-to-field/image-to-image regression problem, i.e., building the mapping from the input flow field to the evolved flow field. The ConvNet, when trained, can be a cheap substitute for physics-based models, enabling fast simulation of field evolvement. However, a big challenge still lies in incorporating conditions that dictate field evolvement, e.g., fluid properties associated with fluid dynamics. We propose a light multi-input ConvNet as a general-purpose, multi-input, image-to-image regression tool. Its simplicity and usefulness are demonstrated by modeling various condition-dependent field evolvements and developments. Large- and extreme-scale simulations are also performed based on its computational superiority.

Comparing transfer learning to feature optimization in microstructure classification

Human analysis of research data is slow and inefficient. In recent years, machine learning tools have advanced our capability to perform tasks normally carried out by humans, such as image segmentation and classification. In this work, we seek to further improve binary classification models for high-throughput identification of different microstructural morphologies. We utilize a dataset with limited observations (133 dendritic structures, 444 non-dendritic) and employ data augmentation via rotation and translation to enhance the dataset six-fold. Then, transfer learning is carried out using pre-trained networks VGG16, InceptionV3, and Xception achieving only moderate F1 scores (0.801–0.822). We hypothesize that feature engineering could yield better results than transfer learning alone. To test this, we employ a new nature-inspired feature optimization algorithm, the Binary Red Deer Algorithm (BRDA), to carry out binary classification and observe F1 scores in the range of 0.96.

•

A dataset comprising two categories of micrographs has been prepared

•

Transfer learning has been implemented for micrograph classification

•

To improve upon the classification accuracy, we perform feature engineering

•

Feature engineering has been performed using Binary Red Deer Algorithm

Enhancing human-machine teaming for medical prognosis through neural ordinary differential equations (NODEs)

Machine learning (ML) has recently been demonstrated to rival expert-level human accuracy in prediction and detection tasks in a variety of domains, including medicine. Despite these impressive findings, however, a key barrier to the full realization of ML’s potential in medical prognoses is technology acceptance. Recent efforts to produce explainable AI (XAI) have made progress in improving the interpretability of some ML models, but these efforts suffer from limitations intrinsic to their design: they work best at identifying why a system fails, but do poorly at explaining when and why a model’s prediction is correct. We posit that the acceptability of ML predictions in expert domains is limited by two key factors: the machine’s horizon of prediction that extends beyond human capability, and the inability for machine predictions to incorporate human intuition into their models. We propose the use of a novel ML architecture, Neural Ordinary Differential Equations (NODEs) to enhance human understanding and encourage acceptability. Our approach prioritizes human cognitive intuition at the center of the algorithm design, and offers a distribution of predictions rather than single outputs. We explain how this approach may significantly improve human-machine collaboration in prediction tasks in expert domains such as medical prognoses. We propose a model and demonstrate, by expanding a concrete example from the literature, how our model advances the vision of future hybrid human-AI systems.

Predicting material microstructure evolution via data-driven machine learning

Predicting microstructure evolution can be a formidable challenge, yet it is essential to building microstructure-processing-property relationships. Yang et al. offer a new solution to traditional partial differential equation-based simulations: a data-driven machine learning approach motivated by the practical needs to accelerate the materials design process and deal with incomplete information in the real world of microstructure simulation.