Overview of Surrogate Modeling in Chemical Process Engineering

Modeling Framework for Cost Optimization of Process-Scale Desalination Systems with Mineral Scaling and Precipitation

Cost-optimization models are powerful tools for evaluating emerging water treatment processes. However, to date, optimization models do not incorporate detailed chemical reaction phenomena, limiting the assessment of pretreatment and mineral scaling. Moreover, novel approaches for high-salinity and high-recovery desalination are typically proposed without direct quantification of pretreatment needs or mineral scaling. This work addresses a critical gap in the literature by presenting a modeling framework that includes complex water chemistry predictions with process-scale optimization. We use this approach to conduct a technoeconomic assessment on a conceptual high-recovery treatment train that includes chemical pretreatment (i.e., soda ash softening and recarbonation) and membrane-based desalination (i.e., standard and high-pressure reverse osmosis). We demonstrate how to develop and integrate accurate multidimensional surrogate models for predicting precipitation, pH, and mineral scaling tendencies. Our findings show that cost-optimal results balance the costs of pretreatment with reverse osmosis system design. Optimizing across a range of water recoveries (i.e., 50-90%) reveals multiple cost-optimal schemas that vary the chemical dosing in pretreatment and the design and operation of reverse osmosis. Our results reveal that pretreatment costs can be more than double the cost of the primary desalination process at high recoveries due to the extensive pretreatment required to control scaling. This work emphasizes the importance of and provides a framework for including chemistry and mineral scaling predictions in the evaluation of emerging technologies in high-recovery desalination.

Noisecut: a python package for noise-tolerant classification of binary data using prior knowledge integration and max-cut solutions

BACKGROUND

Classification of binary data arises naturally in many clinical applications, such as patient risk stratification through ICD codes. One of the key practical challenges in data classification using machine learning is to avoid overfitting. Overfitting in supervised learning primarily occurs when a model learns random variations from noisy labels in training data rather than the underlying patterns. While traditional methods such as regularization and early stopping have demonstrated effectiveness in interpolation tasks, addressing overfitting in the classification of binary data, in which predictions always amount to extrapolation, demands extrapolation-enhanced strategies. One such approach is hybrid mechanistic/data-driven modeling, which integrates prior knowledge on input features into the learning process, enhancing the model's ability to extrapolate.

RESULTS

We present NoiseCut, a Python package for noise-tolerant classification of binary data by employing a hybrid modeling approach that leverages solutions of defined max-cut problems. In a comparative analysis conducted on synthetically generated binary datasets, NoiseCut exhibits better overfitting prevention compared to the early stopping technique employed by different supervised machine learning algorithms. The noise tolerance of NoiseCut stems from a dropout strategy that leverages prior knowledge of input features and is further enhanced by the integration of max-cut problems into the learning process.

CONCLUSIONS

NoiseCut is a Python package for the implementation of hybrid modeling for the classification of binary data. It facilitates the integration of mechanistic knowledge on the input features into learning from data in a structured manner and proves to be a valuable classification tool when the available training data is noisy and/or limited in size. This advantage is especially prominent in medical and biomedical applications where data scarcity and noise are common challenges. The codebase, illustrations, and documentation for NoiseCut are accessible for download at https://pypi.org/project/noisecut/ . The implementation detailed in this paper corresponds to the version 0.2.1 release of the software.

A numerical compass for experiment design in chemical kinetics and molecular property estimation

Kinetic process models are widely applied in science and engineering, including atmospheric, physiological and technical chemistry, reactor design, or process optimization. These models rely on numerous kinetic parameters such as reaction rate, diffusion or partitioning coefficients. Determining these properties by experiments can be challenging, especially for multiphase systems, and researchers often face the task of intuitively selecting experimental conditions to obtain insightful results. We developed a numerical compass (NC) method that integrates computational models, global optimization, ensemble methods, and machine learning to identify experimental conditions with the greatest potential to constrain model parameters. The approach is based on the quantification of model output variance in an ensemble of solutions that agree with experimental data. The utility of the NC method is demonstrated for the parameters of a multi-layer model describing the heterogeneous ozonolysis of oleic acid aerosols. We show how neural network surrogate models of the multiphase chemical reaction system can be used to accelerate the application of the NC for a comprehensive mapping and analysis of experimental conditions. The NC can also be applied for uncertainty quantification of quantitative structure-activity relationship (QSAR) models. We show that the uncertainty calculated for molecules that are used to extend training data correlates with the reduction of QSAR model error. The code is openly available as the Julia package KineticCompass.

Accelerating Elastic Property Prediction in Fe-C Alloys through Coupling of Molecular Dynamics and Machine Learning

The scarcity of high-quality data presents a major challenge to the prediction of material properties using machine learning (ML) models. Obtaining material property data from experiments is economically cost-prohibitive, if not impossible. In this work, we address this challenge by generating an extensive material property dataset comprising thousands of data points pertaining to the elastic properties of Fe-C alloys. The data were generated using molecular dynamic (MD) calculations utilizing reference-free Modified embedded atom method (RF-MEAM) interatomic potential. This potential was developed by fitting atomic structure-dependent energies, forces, and stress tensors evaluated at ground state and finite temperatures using ab-initio. Various ML algorithms were subsequently trained and deployed to predict elastic properties. In addition to individual algorithms, super learner (SL), an ensemble ML technique, was incorporated to refine predictions further. The input parameters comprised the alloy's composition, crystal structure, interstitial sites, lattice parameters, and temperature. The target properties were the bulk modulus and shear modulus. Two distinct prediction approaches were undertaken: employing individual models for each property prediction and simultaneously predicting both properties using a single integrated model, enabling a comparative analysis. The efficiency of these models was assessed through rigorous evaluation using a range of accuracy metrics. This work showcases the synergistic power of MD simulations and ML techniques for accelerating the prediction of elastic properties in alloys.

From plastic waste to potential wealth: Upcycling technologies, process synthesis, assessment and optimization

Global plastics production has doubled since the beginning of 21st century. Efficient technology is called for plastics waste valorization. The current review provides an overview of the main waste plastic chemical upcycling technologies to produce value-added products. Various technologies including gasification and pyrolysis are under reviewed. However, several review literatures have paid attention to the details and experimental progress in these chemical upcycling techniques. In this review, we attempt to conclude the progress in a multi-scale systems-by-systems perspective. After a brief overview of the current state-of-the-art chemical upcycling techniques, larger-scale process synthesis, assessment, and optimization methodologies to address the sustainability and environmental issues are summarized. Techno-economic analysis and life cycle assessment are selected as two powerful tools for process assessment. Three particular application scenarios of optimization methodologies including experimental design, process synthesis and supply chain management are consequently introduced. Very little work on review articles have summarized the plastic waste-to-wealth process in the systems engineering perspective. Review results show that (1) gasification and pyrolysis offer promising avenues for the conversion of plastic waste into valuable products. These technologies can be integrated with other subsystems to enhance the economic and environmental performance of the overall system. (2) Response surface methodology is commonly used in experimental design and parameter optimization. It allows researchers to systematically investigate the effects of various parameters and optimize process conditions to maximize desired outputs. (3) Superstructure optimization frameworks are valuable tools for process synthesis and pathway selection in plastic waste conversion. However, the potential superstructure is pre-defined. (4) Green supply chain and multi-objective supply chain frameworks can be applied to the design of plastic waste recycling networks, taking into account both economic and environmental considerations.

Using physical property surrogate models to perform accelerated multi-fidelity optimization of force field parameters†

Accurate representations of van der Waals dispersion–repulsion interactions play an important role in high-quality molecular dynamics simulations. Training the force field parameters used in the Lennard Jones (LJ) potential typically used to represent these interactions is challenging, generally requiring adjustment based on simulations of macroscopic physical properties. The large computational expense of these simulations, especially when many parameters must be trained simultaneously, limits the size of training data set and number of optimization steps that can be taken, often requiring modelers to perform optimizations within a local parameter region. To allow for more global LJ parameter optimization against large training sets, we introduce a multi-fidelity optimization technique which uses Gaussian process surrogate modeling to build inexpensive models of physical properties as a function of LJ parameters. This approach allows for fast evaluation of approximate objective functions, greatly accelerating searches over parameter space and enabling the use of optimization algorithms capable of searching more globally. In this study, we use an iterative framework which performs global optimization with differential evolution at the surrogate level, followed by validation at the simulation level and surrogate refinement. Using this technique on two previously studied training sets, containing up to 195 physical property targets, we refit a subset of the LJ parameters for the OpenFF 1.0.0 (Parsley) force field. We demonstrate that this multi-fidelity technique can find improved parameter sets compared to a purely simulation-based optimization by searching more broadly and escaping local minima. Additionally, this technique often finds significantly different parameter minima that have comparably accurate performance. In most cases, these parameter sets are transferable to other similar molecules in a test set. Our multi-fidelity technique provides a platform for rapid, more global optimization of molecular models against physical properties, as well as a number of opportunities for further refinement of the technique.

We present a multi-fidelity method for optimizing nonbonded force field parameters against physical property data. Leveraging fast surrogate models, we accelerate the parameter search and find novel solutions that improve force field performance.

Simulation-optimization framework for the digital design of pharmaceutical processes using Pyomo and PharmaPy

The problem of performing model-based process design and optimization in the pharmaceutical industry is an important and challenging one both computationally and in choice of solution implementation. In this work, a framework is presented to directly utilize a process simulator via callbacks during derivative-based optimization. The framework allows users with little experience in translating mechanistic ODEs and PDEs to robust, fully discretized algebraic formulations, required for executing simultaneous equation-oriented optimization, to obtain mathematically guaranteed optima at a competitive solution time when compared with existing derivative-free and derivative-based frameworks. The effectiveness of the framework in accuracy of optimal solution as well as computational efficiency is analyzed on on two case studies: (i) an integrated 2-unit reaction synthesis train used for the synthesis of an anti-cancer active pharmaceutical ingredient, and (ii) a more complex flowsheet representing a common synthesis-purification-isolation train of a pharmaceutical manufacturing processes.

Wrapper Functions for Integrating Mathematical Models into Digital Twin Event Processing

Analog sensors often require complex mathematical models for data analysis. Digital twins (DTs) provide platforms to display sensor data in real time but still lack generic solutions regarding how mathematical models and algorithms can be integrated. Based on previous tests for monitoring and predicting banana fruit quality along the cool chain, we demonstrate how a system of multiple models can be converted into a DT. Our new approach provides a set of generic "wrapper functions", which largely simplify model integration. The wrappers connect the in- and outputs of models to the streaming platform and, thus, require only minor changes to the model software. Different scenarios for model linking structures are considered, including simultaneous processing of multiple models, sequential processing of life-cycle-specific models, and predictive models, based on data from the current and previous life cycles. The wrapper functions can be easily adapted to host models or microservices from various applications fields, to predict the future system behavior and to test what-if scenarios.

A Review of Proxy Modeling Highlighting Applications for Reservoir Engineering

Numerical models can be used for many purposes in oil and gas engineering, such as production optimization and forecasting, uncertainty analysis, history matching, and risk assessment. However, subsurface problems are complex and non-linear, and making reliable decisions in reservoir management requires substantial computational effort. Proxy models have gained much attention in recent years. They are advanced non-linear interpolation tables that can approximate complex models and alleviate computational effort. Proxy models are constructed by running high-fidelity models to gather the necessary data to create the proxy model. Once constructed, they can be a great choice for different tasks such as uncertainty analysis, optimization, forecasting, etc. The application of proxy modeling in oil and gas has had an increasing trend in recent years, and there is no consensus rule on the correct choice of proxy model. As a result, it is crucial to better understand the advantages and disadvantages of various proxy models. The existing work in the literature does not comprehensively cover all proxy model types, and there is a considerable requirement for fulfilling the existing gaps in summarizing the classification techniques with their applications. We propose a novel categorization method covering all proxy model types. This review paper provides a more comprehensive guideline on comparing and developing a proxy model compared to the existing literature. Furthermore, we point out the advantages of smart proxy models (SPM) compared to traditional proxy models (TPM) and suggest how we may further improve SPM accuracy where the literature is limited. This review paper first introduces proxy models and shows how they are classified in the literature. Then, it explains that the current classifications cannot cover all types of proxy models and proposes a novel categorization based on various development strategies. This new categorization includes four groups multi-fidelity models (MFM), reduced-order models (ROM), TPM, and SPM. MFMs are constructed based on simplifying physics assumptions (e.g., coarser discretization), and ROMs are based on dimensional reduction (i.e., neglecting irrelevant parameters). Developing these two models requires an in-depth knowledge of the problem. In contrast, TPMs and novel SPMs require less effort. In other words, they do not solve the complex underlying mathematical equations of the problem; instead, they decouple the mathematical equations into a numeric dataset and train statistical/AI-driven models on the dataset. Nevertheless, SPMs implement feature engineering techniques (i.e., generating new parameters) for its development and can capture the complexities within the reservoir, such as the constraints and characteristics of the grids. The newly introduced parameters can help find the hidden patterns within the parameters, which eventually increase the accuracy of SPMs compared to the TPMs. This review highlights the superiority of SPM over traditional statistical/AI-based proxy models. Finally, the application of various proxy models in the oil and gas industry, especially in subsurface modeling with a set of real examples, is presented. The introduced guideline in this review aids the researchers in obtaining valuable information on the current state of PM problems in the oil and gas industry.

Kinetics of the Direct DME Synthesis: State of the Art and Comprehensive Comparison of Semi-Mechanistic, Data-Based and Hybrid Modeling Approaches

Hybrid kinetic models represent a promising alternative to describe and evaluate the effect of multiple variables in the performance of complex chemical processes, since they combine system knowledge and extrapolability of the (semi-)mechanistic models in a wide range of reaction conditions with the adaptability and fast convergence of data-based approaches (e.g., artificial neural networks—ANNs). For the first time, a hybrid kinetic model for the direct DME synthesis was developed consisting of a reactor model, i.e., balance equations, and an ANN for the reaction kinetics. The accuracy, computational time, interpolation and extrapolation ability of the new hybrid model were compared to those of aumped and a data-based model with the same validity range, using both simulations and experiments. The convergence of parameter estimation and simulations with the hybrid model is much faster than with theumped model, and the predictions show a greater degree of accuracy within the models’ validity range. A satisfactory dimension and range extrapolation was reached when the extrapolated variable was included in the knowledge module of the model. This feature is particularly dependent on the network architecture and phenomena covered by the underlying model, andess on the experimental conditions evaluated during model development.

Increasing Superstructure Optimization Capacity Through Self-Learning Surrogate Models

Simulation-based optimization models are widely applied to find optimal operating conditions of processes. Often, computational challenges arise from model complexity, making the generation of reliable design solutions difficult. We propose an algorithm for replacing non-linear process simulation models integrated in multi-level optimization of a process and energy system superstructure with surrogate models, applying an active learning strategy to continuously enrich the database on which the surrogate models are trained and evaluated. Surrogate models are generated and trained on an initial data set, each featuring the ability to quantify the uncertainty with which a prediction is made. Until a defined prediction quality is met, new data points are continuously labeled and added to the training set. They are selected from a pool of unlabeled data points based on the predicted uncertainty, ensuring a rapid improvement of surrogate quality. When applied in the optimization superstructure, the surrogates can only be used when the prediction quality for the given data point reaches a specified threshold, otherwise the original simulation model is called for evaluating the process performance and the newly obtained data points are used to improve the surrogates. The method is tested on three simulation models, ranging in size and complexity. The proposed approach yields mean squared errors of the test prediction below 2% for all cases. Applying the active learning approach leads to better predictions compared to random sampling for the same size of database. When integrated in the optimization framework, simpler surrogates are favored in over 60% of cases, while the more complex ones are enabled by using simulation results generated during optimization for improving the surrogates after the initial generation. Significant time savings are recorded when using complex process simulations, though the advantage gained for simpler processes is marginal. Overall, we show that the proposed method saves time and adds flexibility to complex superstructure optimization problems that involve optimizing process operating conditions. Computational time can be greatly reduced without penalizing result quality, while the continuous improvement of surrogates when simulation is used in the optimization leads to a natural refinement of the model.