Formula Graph Self-Attention Network for Representation-Domain Independent Materials Discovery

Rational Design of Deep Learning Networks Based on a Fusion Strategy for Improved Material Property Predictions

The success of machine learning in predicting material properties is largely dependent on the design of the model. However, the current designs of deep learning models in materials science have the following prominent problems. First, the model design lacks a rational guidance strategy and heavily relies on a large amount of trial and error. Second, numerous deep learning models are utilized across various fields, each with its own advantages and disadvantages. Therefore, it is important to incorporate a fusion strategy to fully leverage them and further expand the design strategies of the models. To address these problems, we analyze that the main reason is the lack of a new feedback method rich in physical insights. In this study, we developed a feedback method called the Chemical Environment Clustering Vector (CECV) of compounds at different thresholds, which is rich in physical insights. Based on CECV, we rationally designed the Long Short-Term Memory and Gated Recurrent Unit fused with Deep Convolutional Neural Network (L-G-DCNN) to explore the field of structure-agnostic material property predictions. L-G-DCNN accurately captures the interactions between elements in compounds, enabling more accurate and efficient predictions of the material properties. Our results demonstrate that the performance of the L-G-DCNN surpasses the current state-of-the-art structure-agnostic models across 28 benchmark data sets, exhibiting superior sample efficiency and faster convergence speed. By employing different visualization methods, we demonstrate that the fusion strategy based on CECV significantly enhances the comprehension of the L-G-DCNN model design and provides a fresh perspective for researchers in the field of materials informatics.

Pretraining Strategies for Structure Agnostic Material Property Prediction

In recent years, machine learning (ML), especially graph neural network (GNN) models, has been successfully used for fast and accurate prediction of material properties. However, most ML models rely on relaxed crystal structures to develop descriptors for accurate predictions. Generating these relaxed crystal structures can be expensive and time-consuming, thus requiring an additional processing step for models that rely on them. To address this challenge, structure-agnostic methods have been developed, which use fixed-length descriptors engineered based on human knowledge about the material. However, the fixed-length descriptors are often hand-engineered and require extensive domain knowledge and generally are not used in the context of learnable models which are known to have a superior performance. Recent advancements have proposed learnable frameworks that can construct representations based on stoichiometry alone, allowing the flexibility of using deep learning frameworks as well as leveraging structure-agnostic learning. In this work, we propose three different pretraining strategies that can be used to pretrain these structure-agnostic, learnable frameworks to further improve the downstream material property prediction performance. We incorporate strategies such as self-supervised learning (SSL), fingerprint learning (FL), and multimodal learning (ML) and demonstrate their efficacy on downstream tasks for the Roost architecture, a popular structure-agnostic framework. Our results show significant improvement in small data sets and data efficiency in the larger data sets, underscoring the potential of our pretrain strategies that effectively leverage unlabeled data for accurate material property prediction.

Gradient boosted and statistical feature selection workflow for materials property predictions

With the emergence of big data initiatives and the wealth of available chemical data, data-driven approaches are becoming a vital component of materials discovery pipelines or workflows. The screening of materials using machine-learning models, in particular, is increasingly gaining momentum to accelerate the discovery of new materials. However, the black-box treatment of machine-learning methods suffers from a lack of model interpretability, as feature relevance and interactions can be overlooked or disregarded. In addition, naive approaches to model training often lead to irrelevant features being used which necessitates the need for various regularization techniques to achieve model generalization; this incurs a high computational cost. We present a feature-selection workflow that overcomes this problem by leveraging a gradient boosting framework and statistical feature analyses to identify a subset of features, in a recursive manner, which maximizes their relevance to the target variable or classes. We subsequently obtain minimal feature redundancy through multicollinearity reduction by performing feature correlation and hierarchical cluster analyses. The features are further refined using a wrapper method, which follows a greedy search approach by evaluating all possible feature combinations against the evaluation criterion. A case study on elastic material-property prediction and a case study on the classification of materials by their metallicity are used to illustrate the use of our proposed workflow; although it is highly general, as demonstrated through our wider subsequent prediction of various material properties. Our Bayesian-optimized machine-learning models generated results, without the use of regularization techniques, which are comparable to the state-of-the-art that are reported in the scientific literature.