A Survey of Recent Trends in Multiobjective Optimal Control—Surrogate Models, Feedback Control and Objective Reduction

Data-Driven Dynamic Multiobjective Optimal Control: An Aspiration-Satisfying Reinforcement Learning Approach

This article presents an iterative data-driven algorithm for solving dynamic multiobjective (MO) optimal control problems arising in control of nonlinear continuous-time systems. It is first shown that the Hamiltonian functional corresponding to each objective can be leveraged to compare the performance of admissible policies. Hamiltonian inequalities are then used for which their satisfaction guarantees satisfying the objectives' aspirations. Relaxed Hamilton-Jacobi-Bellman (HJB) equations in terms of HJB inequalities are then solved in a dynamic constrained MO framework to find Pareto optimal solutions. Relation to satisficing (good enough) decision-making framework is shown. A sum-of-square (SOS)-based iterative algorithm is developed to solve the formulated aspiration-satisfying MO optimization. To obviate the requirement of complete knowledge of the system dynamics, a data-driven satisficing reinforcement learning approach is proposed to solve the SOS optimization problem in real time using only the information of the system trajectories measured during a time interval without having full knowledge of the system dynamics. Finally, two simulation examples are utilized to verify the analytical results of the proposed algorithm.

A Pareto–Pontryagin Maximum Principle for Optimal Control

In this paper, an attempt to unify two important lines of thought in applied optimization is proposed. We wish to integrate the well-known (dynamic) theory of Pontryagin optimal control with the Pareto optimization (of the static type), involving the maximization/minimization of a non-trivial number of functions or functionals, Pontryagin optimal control offers the definitive theoretical device for the dynamic realization of the objectives to be optimized. The Pareto theory is undoubtedly less known in mathematical literature, even if it was studied in topological and variational details (Morse theory) by Stephen Smale. This reunification, obviously partial, presents new conceptual problems; therefore, a basic review is necessary and desirable. After this review, we define and unify the two theories. Finally, we propose a Pontryagin extension of a recent multiobjective optimization application to the evolution of trees and the related anatomy of the xylems . This work is intended as the first contribution to a series to be developed by the authors on this subject.

Multiobjective Optimal Control of Wind Turbines: A Survey on Methods and Recommendations for the Implementation

Advanced control system design for large wind turbines is becoming increasingly complex, and high-level optimization techniques are receiving particular attention as an instrument to fulfil this significant degree of design requirements. Multiobjective optimal (MOO) control, in particular, is today a popular methodology for achieving a control system that conciliates multiple design objectives that may typically be incompatible. Multiobjective optimization was a matter of theoretical study for a long time, particularly in the areas of game theory and operations research. Nevertheless, the discipline experienced remarkable progress and multiple advances over the last two decades. Thus, many high-complexity optimization algorithms are currently accessible to address current control problems in systems engineering. On the other hand, utilizing such methods is not straightforward and requires a long period of trying and searching for, among other aspects, start parameters, adequate objective functions, and the best optimization algorithm for the problem. Hence, the primary intention of this work is to investigate old and new MOO methods from the application perspective for the purpose of control system design, offering practical experience, some open topics, and design hints. A very challenging problem in the system engineering application of power systems is to dominate the dynamic behavior of very large wind turbines. For this reason, it is used as a numeric case study to complete the presentation of the paper.

Combinatorial Optimization Problems and Metaheuristics: Review, Challenges, Design, and Development

In the past few decades, metaheuristics have demonstrated their suitability in addressing complex problems over different domains. This success drives the scientific community towards the definition of new and better-performing heuristics and results in an increased interest in this research field. Nevertheless, new studies have been focused on developing new algorithms without providing consolidation of the existing knowledge. Furthermore, the absence of rigor and formalism to classify, design, and develop combinatorial optimization problems and metaheuristics represents a challenge to the field’s progress. This study discusses the main concepts and challenges in this area and proposes a formalism to classify, design, and code combinatorial optimization problems and metaheuristics. We believe these contributions may support the progress of the field and increase the maturity of metaheuristics as problem solvers analogous to other machine learning algorithms.

Prediction of Covid-19 spreading and optimal coordination of counter-measures: From microscopic to macroscopic models to Pareto fronts

The Covid-19 disease has caused a world-wide pandemic with more than 60 million positive cases and more than 1.4 million deaths by the end of November 2020. As long as effective medical treatment and vaccination are not available, non-pharmaceutical interventions such as social distancing, self-isolation and quarantine as well as far-reaching shutdowns of economic activity and public life are the only available strategies to prevent the virus from spreading. These interventions must meet conflicting requirements where some objectives, like the minimization of disease-related deaths or the impact on health systems, demand for stronger counter-measures, while others, such as social and economic costs, call for weaker counter-measures. Therefore, finding the optimal compromise of counter-measures requires the solution of a multi-objective optimization problem that is based on accurate prediction of future infection spreading for all combinations of counter-measures under consideration. We present a strategy for construction and solution of such a multi-objective optimization problem with real-world applicability. The strategy is based on a micro-model allowing for accurate prediction via a realistic combination of person-centric data-driven human mobility and behavior, stochastic infection models and disease progression models including micro-level inclusion of governmental intervention strategies. For this micro-model, a surrogate macro-model is constructed and validated that is much less computationally expensive and can therefore be used in the core of a numerical solver for the multi-objective optimization problem. The resulting set of optimal compromises between counter-measures (Pareto front) is discussed and its meaning for policy decisions is outlined.

Derivative-Free Multiobjective Trust Region Descent Method Using Radial Basis Function Surrogate Models

We present a local trust region descent algorithm for unconstrained and convexly constrained multiobjective optimization problems. It is targeted at heterogeneous and expensive problems, i.e., problems that have at least one objective function that is computationally expensive. Convergence to a Pareto critical point is proven. The method is derivative-free in the sense that derivative information need not be available for the expensive objectives. Instead, a multiobjective trust region approach is used that works similarly to its well-known scalar counterparts and complements multiobjective line-search algorithms. Local surrogate models constructed from evaluation data of the true objective functions are employed to compute possible descent directions. In contrast to existing multiobjective trust region algorithms, these surrogates are not polynomial but carefully constructed radial basis function networks. This has the important advantage that the number of data points needed per iteration scales linearly with the decision space dimension. The local models qualify as fully linear and the corresponding general scalar framework is adapted for problems with multiple objectives.

Modelling for Digital Twins—Potential Role of Surrogate Models

The application of white box models in digital twins is often hindered by missing knowledge, uncertain information and computational difficulties. Our aim was to overview the difficulties and challenges regarding the modelling aspects of digital twin applications and to explore the fields where surrogate models can be utilised advantageously. In this sense, the paper discusses what types of surrogate models are suitable for different practical problems as well as introduces the appropriate techniques for building and using these models. A number of examples of digital twin applications from both continuous processes and discrete manufacturing are presented to underline the potentials of utilising surrogate models. The surrogate models and model-building methods are categorised according to the area of applications. The importance of keeping these models up to date through their whole model life cycle is also highlighted. An industrial case study is also presented to demonstrate the applicability of the concept.

Surrogate Modeling Approaches for Multiobjective Optimization: Methods, Taxonomy, and Results

Most practical optimization problems are comprised of multiple conflicting objectives and constraints which involve time-consuming simulations. Construction of metamodels of objectives and constraints from a few high-fidelity solutions and a subsequent optimization of metamodels to find in-fill solutions in an iterative manner remain a common metamodeling based optimization strategy. The authors have previously proposed a taxonomy of 10 different metamodeling frameworks for multiobjective optimization problems, each of which constructs metamodels of objectives and constraints independently or in an aggregated manner. Of the 10 frameworks, five follow a generative approach in which a single Pareto-optimal solution is found at a time and other five frameworks were proposed to find multiple Pareto-optimal solutions simultaneously. Of the 10 frameworks, two frameworks (M3-2 and M4-2) are detailed here for the first time involving multimodal optimization methods. In this paper, we also propose an adaptive switching based metamodeling (ASM) approach by switching among all 10 frameworks in successive epochs using a statistical comparison of metamodeling accuracy of all 10 frameworks. On 18 problems from three to five objectives, the ASM approach performs better than the individual frameworks alone. Finally, the ASM approach is compared with three other recently proposed multiobjective metamodeling methods and superior performance of the ASM approach is observed. With growing interest in metamodeling approaches for multiobjective optimization, this paper evaluates existing strategies and proposes a viable adaptive strategy by portraying importance of using an ensemble of metamodeling frameworks for a more reliable multiobjective optimization for a limited budget of solution evaluations.

The Pareto Tracer for General Inequality Constrained Multi-Objective Optimization Problems

Problems where several incommensurable objectives have to be optimized concurrently arise in many engineering and financial applications. Continuation methods for the treatment of such multi-objective optimization methods (MOPs) are very efficient if all objectives are continuous since in that case one can expect that the solution set forms at least locally a manifold. Recently, the Pareto Tracer (PT) has been proposed, which is such a multi-objective continuation method. While the method works reliably for MOPs with box and equality constraints, no strategy has been proposed yet to adequately treat general inequalities, which we address in this work. We formulate the extension of the PT and present numerical results on some selected benchmark problems. The results indicate that the new method can indeed handle general MOPs, which greatly enhances its applicability.

Multi-Objective Optimization of Production Objectives Based on Surrogate Model

The article addresses an approximate solution to the multi-objective optimization problem for a black-box function of a manufacturing system. We employ the surrogate of the discrete-event simulation model of a batch production system in an analytical form. Integration of simulation, Design of Experiments methods, and Weighted Sum and Weighted Product multi-objective methods are used in an arrangement of a priori defined preferences to find a solution near the Pareto optimal solution in a criterion space. We compare the results obtained through the analytical approach to the outcomes of simulation-based optimization. The observed results indicate a possibility to apply the suitable analytical model for quickly finding the acceptable approximate solution close to the Pareto optimal front.

Numerical Computation of Lightly Multi-Objective Robust Optimal Solutions by Means of Generalized Cell Mapping

In this paper, we present a novel algorithm for the computation of lightly robust optimal solutions for multi-objective optimization problems. To this end, we adapt the generalized cell mapping, originally designed for the global analysis of dynamical systems, to the current context. This is the first time that a set-based method is developed for such kinds of problems. We demonstrate the strength of the novel algorithms on several benchmark problems as well as on one feed-back control design problem where the objectives are given by the peak time, the overshoot, and the absolute tracking error for the linear control system, which has a control time delay. The numerical results indicate that the new algorithm is well-suited for the reliable treatment of low dimensional problems.

Pareto Explorer for Finding the Knee for Many Objective Optimization Problems

Optimization problems where several objectives have to be considered concurrently arise in many applications. Since decision-making processes are getting more and more complex, there is a recent trend to consider more and more objectives in such problems, known as many objective optimization problems (MaOPs). For such problems, it is not possible any more to compute finite size approximations that suitably represent the entire solution set. If no users preferences are at hand, so-called knee points are promising candidates since they represent at least locally the best trade-off solutions among the considered objective values. In this paper, we extend the global/local exploration tool Pareto Explorer (PE) for the detection of such solutions. More precisely, starting from an initial solution, the goal of the modified PE is to compute a path of evenly spread solutions from this point along the Pareto front leading to a knee of the MaOP. The knee solution, as well as all other points from this path, are of potential interest for the underlying decision-making process. The benefit of the approach is demonstrated in several examples.

The Averaged Hausdorff Distances in Multi-Objective Optimization: A Review

A brief but comprehensive review of the averaged Hausdorff distances that have recentlybeen introduced as quality indicators in multi-objective optimization problems (MOPs) is presented.First, we introduce all the necessary preliminaries, definitions, and known properties of thesedistances in order to provide a stat-of-the-art overview of their behavior from a theoretical pointof view. The presentation treats separately the definitions of the (p, q)-distances GDp,q, IGDp,q, and Δp,q for finite sets and their generalization for arbitrary measurable sets that covers as an importantexample the case of continuous sets. Among the presented results, we highlight the rigorousconsideration of metric properties of these definitions, including a proof of the triangle inequalityfor distances between disjoint subsets when p, q ≥ 1, and the study of the behavior of associatedindicators with respect to the notion of compliance to Pareto optimality. Illustration of these resultsin particular situations are also provided. Finally, we discuss a collection of examples and numericalresults obtained for the discrete and continuous incarnations of these distances that allow for anevaluation of their usefulness in concrete situations and for some interesting conclusions at the end,justifying their use and further study.

Variation Rate to Maintain Diversity in Decision Space within Multi-Objective Evolutionary Algorithms

The performance of a multi-objective evolutionary algorithm (MOEA) is in most cases measured in terms of the populations’ approximation quality in objective space. As a consequence, most MOEAs focus on such approximations while neglecting the distribution of the individuals of their populations in decision space. This, however, represents a potential shortcoming in certain applications as in many cases one can obtain the same or very similar qualities (measured in objective space) in several ways (measured in decision space). Hence, a high diversity in decision space may represent valuable information for the decision maker for the realization of a given project. In this paper, we propose the Variation Rate, a heuristic selection strategy that aims to maintain diversity both in decision and objective space. The core of this strategy is the proper combination of the averaged distance applied in variable space together with the diversity mechanism in objective space that is used within a chosen MOEA. To show the applicability of the method, we propose the resulting selection strategies for some of the most representative state-of-the-art MOEAs and show numerical results on several benchmark problems. The results demonstrate that the consideration of the Variation Rate can greatly enhance the diversity in decision space for all considered algorithms and problems without a significant loss in the approximation qualities in objective space.