Optimal Design of Energy Systems Using Constrained Grey-Box Multi-Objective Optimization

Effects of Surrogate Hybridization and Adaptive Sampling for Simulation-Based Optimization

Process simulators are essential for modeling of complex processes; however, optimization of expensive models remains challenging due to lack of equations, simulation cost, and lack of convergence guarantees. To tackle these challenges, surrogate modeling and surrogate-based optimization methods have been proposed. Most commonly, surrogates are treated as black-box models, while recently hybrid surrogates have gained popularity. In this work, we assess two main methodologies: (a) optimization of surrogates trained using a set of fixed a priori samples using deterministic solvers, and (b) adaptive sampling-based optimization, which leverages surrogate predictions to guide the search process. Across both methods, we systematically compare the effect of black-box versus hybrid surrogates, that utilize a "model-correction" architecture combining different fidelity data. Through mathematical benchmarks with up to ten dimensions, and two engineering case studies for process design of an extractive distillation simulation model and an adsorption simulation model, we present the effects of sampling quantity, dimensionality, formulation, and hybridization on solution convergence, reliability, and CPU efficiency. Our results show that hybrid modeling improves surrogate robustness and reduces solution variability with fewer samples, though it increases optimization costs. Additionally, adaptive sampling methods are more efficient and consistent than fixed-sampling surrogate strategies, even across different sampling and dimensionality scenarios.

Surrogate-Assisted Optimization of Highly Constrained Oil Recovery Processes Using Classification-Based Constraint Modeling

Real-world problems often involve constraints that must be carefully managed for feasible and efficient operations. In optimization, this becomes especially challenging with complex, high-dimensional problems that are computationally expensive and subject to hundreds or even thousands of constraints. We address these challenges by optimizing the highly constrained waterflooding process using a surrogate model of the reservoir and a classification-based constraint handling technique. Our study uses benchmark reservoir simulations, beginning with the low-dimensional Egg model and extending to the high-dimensional UNISIM model. We employ a Feedforward Neural Network (FFNN) surrogate for objective quantification and use classification-based modeling to transform the numerous constraints into a binary problem, distinguishing between feasible and infeasible reservoir settings. Our methodology involves an offline phase to develop and train models using reservoir simulation data, achieving high predictive accuracy (R 2 > 0.98) with 20,000 bottom-hole pressure (BHP) settings and net present value (NPV) outputs. The classifier algorithms are then trained to model the constraints, ensuring that the solutions identified during optimization are feasible. In the online phase, we employ different model-based and search-based optimizers to find the optimal BHP settings that maximize the NPV throughout the production horizon. By integrating a highly accurate surrogate model and classification-based constraint handling, our approach significantly reduces the computational burden while ensuring that the solutions remain feasible, optimized for maximum economic gain, and yield better results compared to the deterministic approach.

Bi-level Data-driven Enterprise-wide Optimization with Mixed-integer Nonlinear Scheduling Problems

Planning and scheduling are crucial components of enterprise-wide optimization (EWO). For the successful execution of EWO, it is vital to view the enterprise operations as a holistic decision-making problem, composed of different interconnected elements or layers, to make the most efficient use of resources in process industries. Among different layers of the operating decisions, planning and scheduling are often treated sequentially, leading to impractical solutions. To tackle this problem, integrated approaches, such as bi-level programming are utilized to address these two layers simultaneously. Yet, the bi-level optimization of such interdependent and holistic formulations is still difficult, particularly when dealing with mixed-integer nonlinear programming (MINLP) problems, due to a lack of effective algorithms. In this study, we utilize the Data-driven Optimization of bi-level Mixed-Integer NOnlinear problems (DOMINO) framework, a data-driven algorithm developed to handle single-leader single-follower bi-level mixed-integer problems, to solve single-leader multi-follower planning and scheduling problems subject to MINLP scheduling formulations. We apply DOMINO to the continuous production of multi-product methyl methacrylate polymerization process formulated as a Traveling Salesman Problem and demonstrate its capability in achieving near-optimal guaranteed feasible solutions. Building on this foundation, we extend this strategy to solve a high-dimensional and highly constrained nonlinear crude oil refinery operation problem that has not been previously tackled in this context. Our study further evaluates the efficacy of using local, NOMAD (Nonlinear Optimization by Mesh Adaptive Direct Search), and a global data-driven optimizer, ARGONAUT (AlgoRithms for Global Optimization of coNstrAined grey-box compUTational), within the DOMINO framework and characterize their performance both in terms of solution quality and computational expense. The results indicate that DOMINO-NOMAD consistently achieves superior performance compared to DOMINO-ARGONAUT by identifying lower planning costs and generating more feasible solutions across multiple runs. Overall, this study demonstrates DOMINO's ability to optimize production targets, meet market demands, and address large-scale EWO problems.

From Then to Now and Beyond: Exploring How Machine Learning Shapes Process Design Problems

Following the discovery of the least squares method in 1805 by Legendre and later in 1809 by Gauss, surrogate modeling and machine learning have come a long way. From identifying patterns and trends in process data to predictive modeling, optimization, fault detection, reaction network discovery, and process operations, machine learning became an integral part of all aspects of process design and process systems engineering. This is enabled, at the same time necessitated, by the vast amounts of data that are readily available from processes, increased digitalization, automation, increasing computation power, and simulation software that can model complex phenomena that span over several temporal and spatial scales. Although this paper is not a comprehensive review, it gives an overview of the recent history of machine learning models that we use every day and how they shaped process design problems from the recent advances to the exploration of their prospects.

Machine Learning Methods for Endocrine Disrupting Potential Identification Based on Single-Cell Data

Humans are continuously exposed to a variety of toxicants and chemicals which is exacerbated during and after environmental catastrophes such as floods, earthquakes, and hurricanes. The hazardous chemical mixtures generated during these events threaten the health and safety of humans and other living organisms. This necessitates the development of rapid decision-making tools to facilitate mitigating the adverse effects of exposure on the key modulators of the endocrine system, such as the estrogen receptor alpha (ERα), for example. The mechanistic stages of the estrogenic transcriptional activity can be measured with high content/high throughput microscopy-based biosensor assays at the single-cell level, which generates millions of object-based minable data points. By combining computational modeling and experimental analysis, we built a highly accurate data-driven classification framework to assess the endocrine disrupting potential of environmental compounds. The effects of these compounds on the ERα pathway are predicted as being receptor agonists or antagonists using the principal component analysis (PCA) projections of high throughput, high content image analysis descriptors. The framework also combines rigorous preprocessing steps and nonlinear machine learning algorithms, such as the Support Vector Machines and Random Forest classifiers, to develop highly accurate mathematical representations of the separation between ERα agonists and antagonists. The results show that Support Vector Machines classify the unseen chemicals correctly with more than 96% accuracy using the proposed framework, where the preprocessing and the PCA steps play a key role in suppressing experimental noise and unraveling hidden patterns in the dataset.

Modular Framework for Simulation-Based Multi-objective Optimization of a Cryogenic Air Separation Unit

A framework to obtain optimal operating conditions is proposed for a cryogenic air separation unit case study. The optimization problem is formulated considering three objective functions, 11 decision variables, and two constraint setups. Different optimization algorithms simultaneously evaluate the conflicting objective functions: the annualized cash flow, the efficiency at the compression stage, and capital expenditures. The framework follows a modular approach, in which the process simulator PRO/II and a Python environment are combined. The results permit us to assess the applicability of the tested algorithms and to determine optimal operational windows based on the resultant 3-D Pareto fronts.

Data-Driven Optimization of Mixed-integer Bi-level Multi-follower Integrated Planning and Scheduling Problems Under Demand Uncertainty

The coordination of interconnected elements across the different layers of the supply chain is essential for all industrial processes and the key to optimal decision-making. Yet, the modeling and optimization of such interdependent systems are still burdensome. In this paper, we address the simultaneous modeling and optimization of medium-term planning and short-term scheduling problems under demand uncertainty using mixed-integer bi-level multi-follower programming and data-driven optimization. Bi-level multi-follower programs model the natural hierarchy between different layers of supply chain management holistically, while scenario analysis and data-driven optimization allow us to retrieve the guaranteed feasible solutions of the integrated formulation under various demand considerations. We address the data-driven optimization of this challenging class of problems using the DOMINO framework, which was initially developed to solve single-leader single-follower bi-level optimization problems to guaranteed feasibility. This framework is extended to solve single-leader multi-follower stochastic formulations and its performance is characterized by well-known single and multi-product process scheduling case studies. Through our data-driven algorithmic approach, we present guaranteed feasible solutions to linear and nonlinear mixed-integer bi-level formulations of simultaneous planning and scheduling problems and further characterize the effects of the scheduling level complexity on the solution performance, which spans over several hundred continuous and binary variables, and thousands of constraints.

Agricultural land resource allocation to develop food crop commodities: lesson from Indonesia

This study estimates agricultural land resource allocation to develop food-crop commodities in order to safeguard food security in Indonesia in the middle of the coronavirus pandemic. The recommended commodities to be developed in Indonesia are corn, soybean, mungbean, peanut, and rice that are produced with advanced technology and input-output coefficient. There are five introduced scenarios namely, basic scenario, I, II, III, and IV. There are problems related to resource allocation such as limited resources, the ways of using it, and time constraints. In order to maintain and improve the comparative advantage of agricultural production as well as to broaden the agricultural activities, agricultural development is directed to increase business efficiency, improvements in agricultural science, technology, and human resource quality. The utilization of agricultural land resources should be well-planned for better development.

Combining Experimental Isotherms, Minimalistic Simulations, and a Model to Understand and Predict Chemical Adsorption onto Montmorillonite Clays

An attractive approach to minimize human and animal exposures to toxic environmental contaminants is the use of safe and effective sorbent materials to sequester them. Montmorillonite clays have been shown to tightly bind diverse toxic chemicals. Due to their promise as sorbents to mitigate chemical exposures, it is important to understand their function and rapidly screen and predict optimal clay-chemical combinations for further testing. We derived adsorption free-energy values for a structurally and physicochemically diverse set of toxic chemicals using experimental adsorption isotherms performed in the current and previous studies. We studied the diverse set of chemicals using minimalistic MD simulations and showed that their interaction energies with calcium montmorillonite clays calculated using simulation snapshots in combination with their net charge and their corresponding solvent's dielectric constant can be used as inputs to a minimalistic model to predict adsorption free energies in agreement with experiments. Additionally, experiments and computations were used to reveal structural and physicochemical properties associated with chemicals that can be adsorbed to calcium montmorillonite clay. These properties include positively charged groups, phosphine groups, halide-rich moieties, hydrogen bond donor/acceptors, and large, rigid structures. The combined experimental and computational approaches used in this study highlight the importance and potential applicability of analogous methods to study and design novel advanced sorbent systems in the future, broadening their applicability for environmental contaminants.

Multiobjective Optimization of Mixed-Integer Linear Programming Problems: A Multiparametric Optimization Approach

Industrial process systems need to be optimized, simultaneously satisfying financial, quality and safety criteria. To meet all those potentially conflicting optimization objectives, multiobjective optimization formulations can be used to derive optimal trade-off solutions. In this work, we present a framework that provides the exact Pareto front of multiobjective mixed-integer linear optimization problems through multiparametric programming. The original multiobjective optimization program is reformulated through the well-established ϵ-constraint scalarization method, in which the vector of scalarization parameters is treated as a right-hand side uncertainty for the multiparametric program. The algorithmic procedure then derives the optimal solution of the resulting multiparametric mixed-integer linear programming problem as an affine function of the ϵ parameters, which explicitly generates the Pareto front of the multiobjective problem. The solution of a numerical example is analytically presented to exhibit the steps of the approach, while its practicality is shown through a simultaneous process and product design problem case study. Finally, the computational performance is benchmarked with case studies of varying dimensionality with respect to the number of objective functions and decision variables.

Modelling for Digital Twins—Potential Role of Surrogate Models

The application of white box models in digital twins is often hindered by missing knowledge, uncertain information and computational difficulties. Our aim was to overview the difficulties and challenges regarding the modelling aspects of digital twin applications and to explore the fields where surrogate models can be utilised advantageously. In this sense, the paper discusses what types of surrogate models are suitable for different practical problems as well as introduces the appropriate techniques for building and using these models. A number of examples of digital twin applications from both continuous processes and discrete manufacturing are presented to underline the potentials of utilising surrogate models. The surrogate models and model-building methods are categorised according to the area of applications. The importance of keeping these models up to date through their whole model life cycle is also highlighted. An industrial case study is also presented to demonstrate the applicability of the concept.

A framework to predict the price of energy for the end-users with applications to monetary and energy policies

Energy affects every single individual and entity in the world. Therefore, it is crucial to precisely quantify the “price of energy” and study how it evolves through time, through major political and social events, and through changes in energy and monetary policies. Here, we develop a predictive framework, an index to calculate the average price of energy in the United States. The complex energy landscape is thoroughly analysed to accurately determine the two key factors of this framework: the total demand of the energy products directed to the end-use sectors, and the corresponding price of each product. A rolling horizon predictive methodology is introduced to estimate future energy demands, with excellent predictive capability, shown over a period of 174 months. The effectiveness of the framework is demonstrated by addressing two policy questions of significant public interest.

Global energy transformation requires quantifying the "price of energy" and studying its evolution. Here the authors present a predictive framework that calculates the average US price of energy, estimating future energy demands for up to four years with excellent accuracy, designing and optimizing energy and monetary policies.

A data-driven optimization algorithm for differential algebraic equations with numerical infeasibilities

Support Vector Machines (SVMs) based optimization framework is presented for the data-driven optimization of numerically infeasible Differential Algebraic Equations (DAEs) without the full discretization of the underlying first-principles model. By formulating the stability constraint of the numerical integration of a DAE system as a supervised classification problem, we are able to demonstrate that SVMs can accurately map the boundary of numerical infeasibility. The necessity of this data-driven approach is demonstrated on a 2-dimensional motivating example, where highly accurate SVM models are trained, validated, and tested using the data collected from the numerical integration of DAEs. Furthermore, this methodology is extended and tested for a multi-dimensional case study from reaction engineering (i.e., thermal cracking of natural gas liquids). The data-driven optimization of this complex case study is explored through integrating the SVM models with a constrained global grey-box optimization algorithm, namely the ARGONAUT framework.

Classification of estrogenic compounds by coupling high content analysis and machine learning algorithms

Environmental toxicants affect human health in various ways. Of the thousands of chemicals present in the environment, those with adverse effects on the endocrine system are referred to as endocrine-disrupting chemicals (EDCs). Here, we focused on a subclass of EDCs that impacts the estrogen receptor (ER), a pivotal transcriptional regulator in health and disease. Estrogenic activity of compounds can be measured by many in vitro or cell-based high throughput assays that record various endpoints from large pools of cells, and increasingly at the single-cell level. To simultaneously capture multiple mechanistic ER endpoints in individual cells that are affected by EDCs, we previously developed a sensitive high throughput/high content imaging assay that is based upon a stable cell line harboring a visible multicopy ER responsive transcription unit and expressing a green fluorescent protein (GFP) fusion of ER. High content analysis generates voluminous multiplex data comprised of minable features that describe numerous mechanistic endpoints. In this study, we present a machine learning pipeline for rapid, accurate, and sensitive assessment of the endocrine-disrupting potential of benchmark chemicals based on data generated from high content analysis. The multidimensional imaging data was used to train a classification model to ultimately predict the impact of unknown compounds on the ER, either as agonists or antagonists. To this end, both linear logistic regression and nonlinear Random Forest classifiers were benchmarked and evaluated for predicting the estrogenic activity of unknown compounds. Furthermore, through feature selection, data visualization, and model discrimination, the most informative features were identified for the classification of ER agonists/antagonists. The results of this data-driven study showed that highly accurate and generalized classification models with a minimum number of features can be constructed without loss of generality, where these machine learning models serve as a means for rapid mechanistic/phenotypic evaluation of the estrogenic potential of many chemicals.

Chemical contaminants or toxicants pose environmental and health-related risks for exposure. The ability to rapidly understand their biological impact, specifically on a key modulator of important physiological and pathological states in the human body is essential for diagnosing and avoiding undesirable health outcomes during environmental emergencies. In this study, we use advanced data analytics for creating statistical models that can accurately predict the endocrinological activity of toxic chemicals based on high throughput/high content image analysis data. We focus on a subclass of chemicals that affect the estrogen receptor (ER), which is a pivotal transcriptional regulator in health and disease. The multidimensional imaging data of these benchmark chemicals are used to train a classification model to ultimately predict the impact of unknown compounds on the ER, either as agonists or antagonists. To this end, we evaluate linear and nonlinear classifiers for predicting the estrogenic activity of unknown compounds and use feature selection, data visualization, and model discrimination methodologies to identify the most informative features for the classification of ER agonists/antagonists.

DOMINO: Data-driven Optimization of bi-level Mixed-Integer NOnlinear Problems

The Data-driven Optimization of bi-level Mixed-Integer NOnlinear problems (DOMINO) framework is presented for addressing the optimization of bi-level mixed-integer nonlinear programming problems. In this framework, bi-level optimization problems are approximated as single-level optimization problems by collecting samples of the upper-level objective and solving the lower-level problem to global optimality at those sampling points. This process is done through the integration of the DOMINO framework with a grey-box optimization solver to perform design of experiments on the upper-level objective, and to consecutively approximate and optimize bi-level mixed-integer nonlinear programming problems that are challenging to solve using exact methods. The performance of DOMINO is assessed through solving numerous bi-level benchmark problems, a land allocation problem in Food-Energy-Water Nexus, and through employing different data-driven optimization methodologies, including both local and global methods. Although this data-driven approach cannot provide a theoretical guarantee to global optimality, we present an algorithmic advancement that can guarantee feasibility to large-scale bi-level optimization problems when the lower-level problem is solved to global optimality at convergence.

Multi-Objective Optimization Applications in Chemical Process Engineering: Tutorial and Review

This tutorial and review of multi-objective optimization (MOO) gives a detailed explanation of the 5 steps to create, solve, and then select the optimum result. Unlike single-objective optimization, the fifth step of selection or ranking of solutions is often overlooked by the authors of papers dealing with MOO applications. It is necessary to undertake a multi-criteria analysis to choose the best solution. A review of the recent publications using MOO for chemical process engineering problems shows a doubling of publications between 2016 and 2019. MOO applications in the energy area have seen a steady increase of over 20% annually over the last 10 years. The three key methods for solving MOO problems are presented in detail, and an emerging area of surrogate-assisted MOO is also described. The objectives used in MOO trade off conflicting requirements of a chemical engineering problem; these include fundamental criteria such as reaction yield or selectivity; economics; energy requirements; environmental performance; and process control. Typical objective functions in these categories are described, selection/ranking techniques are outlined, and available software for MOO are listed. It is concluded that MOO is gaining popularity as an important tool and is having an increasing use and impact in chemical process engineering.

Grouping of complex substances using analytical chemistry data: A framework for quantitative evaluation and visualization

A detailed characterization of the chemical composition of complex substances, such as products of petroleum refining and environmental mixtures, is greatly needed in exposure assessment and manufacturing. The inherent complexity and variability in the composition of complex substances obfuscate the choices for their detailed analytical characterization. Yet, in lieu of exact chemical composition of complex substances, evaluation of the degree of similarity is a sensible path toward decision-making in environmental health regulations. Grouping of similar complex substances is a challenge that can be addressed via advanced analytical methods and streamlined data analysis and visualization techniques. Here, we propose a framework with unsupervised and supervised analyses to optimally group complex substances based on their analytical features. We test two data sets of complex oil-derived substances. The first data set is from gas chromatography-mass spectrometry (GC-MS) analysis of 20 Standard Reference Materials representing crude oils and oil refining products. The second data set consists of 15 samples of various gas oils analyzed using three analytical techniques: GC-MS, GC×GC-flame ionization detection (FID), and ion mobility spectrometry-mass spectrometry (IM-MS). We use hierarchical clustering using Pearson correlation as a similarity metric for the unsupervised analysis and build classification models using the Random Forest algorithm for the supervised analysis. We present a quantitative comparative assessment of clustering results via Fowlkes-Mallows index, and classification results via model accuracies in predicting the group of an unknown complex substance. We demonstrate the effect of (i) different grouping methodologies, (ii) data set size, and (iii) dimensionality reduction on the grouping quality, and (iv) different analytical techniques on the characterization of the complex substances. While the complexity and variability in chemical composition are an inherent feature of complex substances, we demonstrate how the choices of the data analysis and visualization methods can impact the communication of their characteristics to delineate sufficient similarity.