1
|
Yang C, Ding J, Jin Y, Chai T. A Data Stream Ensemble Assisted Multifactorial Evolutionary Algorithm for Offline Data-Driven Dynamic Optimization. EVOLUTIONARY COMPUTATION 2023; 31:433-458. [PMID: 37155647 DOI: 10.1162/evco_a_00332] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/06/2022] [Accepted: 04/01/2023] [Indexed: 05/10/2023]
Abstract
Existing work on offline data-driven optimization mainly focuses on problems in static environments, and little attention has been paid to problems in dynamic environments. Offline data-driven optimization in dynamic environments is a challenging problem because the distribution of collected data varies over time, requiring surrogate models and optimal solutions tracking with time. This paper proposes a knowledge-transfer-based data-driven optimization algorithm to address these issues. First, an ensemble learning method is adopted to train surrogate models to leverage the knowledge of data in historical environments as well as adapt to new environments. Specifically, given data in a new environment, a model is constructed with the new data, and the preserved models of historical environments are further trained with the new data. Then, these models are considered to be base learners and combined as an ensemble surrogate model. After that, all base learners and the ensemble surrogate model are simultaneously optimized in a multitask environment for finding optimal solutions for real fitness functions. In this way, the optimization tasks in the previous environments can be used to accelerate the tracking of the optimum in the current environment. Since the ensemble model is the most accurate surrogate, we assign more individuals to the ensemble surrogate than its base learners. Empirical results on six dynamic optimization benchmark problems demonstrate the effectiveness of the proposed algorithm compared with four state-of-the-art offline data-driven optimization algorithms. Code is available at https://github.com/Peacefulyang/DSE_MFS.git.
Collapse
Affiliation(s)
- Cuie Yang
- State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University, Shenyang 110819, China
| | - Jinliang Ding
- State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University, Shenyang 110819, China
| | - Yaochu Jin
- Bielefeld University, 33619 Bielefeld, Germany State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University, Shenyang 110819, China
| | - Tianyou Chai
- State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University, Shenyang 110819, China
| |
Collapse
|
2
|
Bäck THW, Kononova AV, van Stein B, Wang H, Antonov KA, Kalkreuth RT, de Nobel J, Vermetten D, de Winter R, Ye F. Evolutionary Algorithms for Parameter Optimization-Thirty Years Later. EVOLUTIONARY COMPUTATION 2023; 31:81-122. [PMID: 37339005 DOI: 10.1162/evco_a_00325] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Accepted: 02/22/2023] [Indexed: 06/22/2023]
Abstract
Thirty years, 1993-2023, is a huge time frame in science. We address some major developments in the field of evolutionary algorithms, with applications in parameter optimization, over these 30 years. These include the covariance matrix adaptation evolution strategy and some fast-growing fields such as multimodal optimization, surrogate-assisted optimization, multiobjective optimization, and automated algorithm design. Moreover, we also discuss particle swarm optimization and differential evolution, which did not exist 30 years ago, either. One of the key arguments made in the paper is that we need fewer algorithms, not more, which, however, is the current trend through continuously claiming paradigms from nature that are suggested to be useful as new optimization algorithms. Moreover, we argue that we need proper benchmarking procedures to sort out whether a newly proposed algorithm is useful or not. We also briefly discuss automated algorithm design approaches, including configurable algorithm design frameworks, as the proposed next step toward designing optimization algorithms automatically, rather than by hand.
Collapse
Affiliation(s)
- Thomas H W Bäck
- Leiden Institute of Advanced Computer Science, Leiden University, Netherlands
| | - Anna V Kononova
- Leiden Institute of Advanced Computer Science, Leiden University, Netherlands
| | - Bas van Stein
- Leiden Institute of Advanced Computer Science, Leiden University, Netherlands
| | - Hao Wang
- Leiden Institute of Advanced Computer Science, Leiden University, Netherlands
| | - Kirill A Antonov
- Leiden Institute of Advanced Computer Science, Leiden University, Netherlands
| | - Roman T Kalkreuth
- Leiden Institute of Advanced Computer Science, Leiden University, Netherlands
| | - Jacob de Nobel
- Leiden Institute of Advanced Computer Science, Leiden University, Netherlands
| | - Diederick Vermetten
- Leiden Institute of Advanced Computer Science, Leiden University, Netherlands
| | - Roy de Winter
- Leiden Institute of Advanced Computer Science, Leiden University, Netherlands
| | - Furong Ye
- Leiden Institute of Advanced Computer Science, Leiden University, Netherlands
| |
Collapse
|
3
|
Hu C, Zeng S, Li C, Zhao F. On Nonstationary Gaussian Process Model for Solving Data-Driven Optimization Problems. IEEE TRANSACTIONS ON CYBERNETICS 2023; 53:2440-2453. [PMID: 34699381 DOI: 10.1109/tcyb.2021.3120188] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
In data-driven evolutionary optimization, most existing Gaussian processes (GPs)-assisted evolutionary algorithms (EAs) adopt stationary GPs (SGPs) as surrogate models, which might be insufficient for solving most optimization problems. This article finds that GPs in the optimization problems are nonstationary with great probability. We propose to employ a nonstationary GP (NSGP) surrogate model for data-driven evolutionary optimization, where the mean of the NSGP is allowed to vary with the decision variables, while its residue variance follows an SGP. In this article, the nonstationarity of GPs in the tested functions is theoretically analyzed. In addition, this article constructs an NSGP where the SGP is a degenerate case. Performance comparisons of the NSGP with the SGP and the NSGP-assisted EA (NSGP-MAEA) with the SGP-assisted EA (SGP-MAEA) are carried out on a set of benchmark problems and an antenna design problem. These comparison results demonstrate the competitiveness of the NSGP model.
Collapse
|
4
|
Zhen H, Gong W, Wang L, Ming F, Liao Z. Two-Stage Data-Driven Evolutionary Optimization for High-Dimensional Expensive Problems. IEEE TRANSACTIONS ON CYBERNETICS 2023; 53:2368-2379. [PMID: 34665754 DOI: 10.1109/tcyb.2021.3118783] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Surrogate-assisted evolutionary algorithms (SAEAs) have been widely used for solving complex and computationally expensive optimization problems. However, most of the existing algorithms converge slowly in the later stage. This article proposes a novel two-stage data-driven evolutionary optimization (TS-DDEO) that meets the requirements of early exploration and later exploitation. In the first stage, a surrogate-assisted hierarchical particle swarm optimization method is used to find a promising area from the entire search space. In the second stage, we propose a best-data-driven optimization (BDDO) method with a strong exploitation ability to accelerate the optimization process. BDDO has a real-time update mechanism for the surrogate model and population and uses a predefined number of ranking-top solutions to update population and surrogates. BDDO combines three surrogate-assisted evolutionary sampling strategies: 1) surrogate-assisted differential evolution sampling; 2) surrogate-assisted local search; and 3) a surrogate-assisted full-crossover (FC) strategy which is proposed to integrate existing best genotypes in the population. Experiments and analysis have validated the effectiveness of the two-stage framework, the BDDO method, and the FC strategy. Moreover, the proposed algorithm is compared with five state-of-the-art SAEAs on high-dimensional benchmark functions. The result shows that TS-DDEO performs better both in effectiveness and robustness.
Collapse
|
5
|
Bi-indicator driven surrogate-assisted multi-objective evolutionary algorithms for computationally expensive problems. COMPLEX INTELL SYST 2023. [DOI: 10.1007/s40747-023-00969-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/11/2023]
Abstract
AbstractThis paper presents a bi-indicator-based surrogate-assisted evolutionary algorithm (BISAEA) for multi-objective optimization problems (MOPs) with computationally expensive objectives. In BISAEA, a Pareto-based bi-indictor strategy is proposed based on convergence and diversity indicators, where a nondominated sorting approach is adopted to carry out two-objective optimization (convergence and diversity indicators) problems. The radius-based function (RBF) models are used to approximate the objective values. In addition, the proposed algorithm adopts a one-by-one selection strategy to obtain promising samples from new samples for evaluating the true objectives by their angles and Pareto dominance relationship with real non-dominated solutions to improve the diversity. After the comparison with four state-of-the-art surrogate-assisted evolutionary algorithms and three evolutionary algorithms on 76 widely used benchmark problems, BISAEA shows high efficiency and a good balance between convergence and diversity. Finally, BISAEA is applied to the multidisciplinary optimization of blend-wing-body underwater gliders with 30 decision variables and three objectives, and the results demonstrate that BISAEA has superior performance on computationally expensive engineering problems.
Collapse
|
6
|
Precision agriculture management based on a surrogate model assisted multiobjective algorithmic framework. Sci Rep 2023; 13:1142. [PMID: 36670167 PMCID: PMC9860027 DOI: 10.1038/s41598-023-27990-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2022] [Accepted: 01/11/2023] [Indexed: 01/22/2023] Open
Abstract
Sustainable intensification needs to optimize irrigation and fertilization strategies while increasing crop yield. To enable more precision and effective agricultural management, a bi-level screening and bi-level optimization framework is proposed. Irrigation and fertilization dates are obtained by upper-level screening and upper-level optimization. Subsequently, due to the complexity of the problem, the lower-level optimization uses a data-driven evolutionary algorithm, which combines the fast non-dominated sorting genetic algorithm (NSGA-II), surrogate-assisted model of radial basis function and Decision Support System for Agrotechnology Transfer to handle the expensive objective problem and produce a set of optimal solutions representing a trade-off between conflicting objectives. Then, the lower-level screening quickly finds better irrigation and fertilization strategies among thousands of solutions. Finally, the experiment produces a better irrigation and fertilization strategy, with water consumption reduced by 44%, nitrogen application reduced by 37%, and economic benefits increased by 7 to 8%.
Collapse
|
7
|
Wang H, Sun C, Xie G, Gao XZ, Akhtar F. A Performance Approximation Assisted Expensive Many-objective Evolutionary Algorithm. Inf Sci (N Y) 2023. [DOI: 10.1016/j.ins.2023.01.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/07/2023]
|
8
|
Tian J, Hou M, Bian H, Li J. Variable surrogate model-based particle swarm optimization for high-dimensional expensive problems. COMPLEX INTELL SYST 2022. [DOI: 10.1007/s40747-022-00910-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
AbstractMany industrial applications require time-consuming and resource-intensive evaluations of suitable solutions within very limited time frames. Therefore, many surrogate-assisted evaluation algorithms (SAEAs) have been widely used to optimize expensive problems. However, due to the curse of dimensionality and its implications, scaling SAEAs to high-dimensional expensive problems is still challenging. This paper proposes a variable surrogate model-based particle swarm optimization (called VSMPSO) to meet this challenge and extends it to solve 200-dimensional problems. Specifically, a single surrogate model constructed by simple random sampling is taken to explore different promising areas in different iterations. Moreover, a variable model management strategy is used to better utilize the current global model and accelerate the convergence rate of the optimizer. In addition, the strategy can be applied to any SAEA irrespective of the surrogate model used. To control the trade-off between optimization results and optimization time consumption of SAEAs, we consider fitness value and running time as a bi-objective problem. Applying the proposed approach to a benchmark test suite of dimensions ranging from 30 to 200 and comparisons with four state-of-the-art algorithms show that the proposed VSMPSO achieves high-quality solutions and computational efficiency for high-dimensional problems.
Collapse
|
9
|
Wang Y, Zhang T, Chang Y, Wang X, Liang B, Yuan B. A Surrogate-Assisted Controller for Expensive Evolutionary Reinforcement Learning. Inf Sci (N Y) 2022. [DOI: 10.1016/j.ins.2022.10.134] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
10
|
Combining Lipschitz and RBF Surrogate Models for High-dimensional Computationally Expensive Problems. Inf Sci (N Y) 2022. [DOI: 10.1016/j.ins.2022.11.045] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
11
|
An improved bagging ensemble surrogate-assisted evolutionary algorithm for expensive many-objective optimization. APPL INTELL 2022. [DOI: 10.1007/s10489-021-02709-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
12
|
Liu Y, Liu J, Tan S, Yang Y, Li F. A bagging-based surrogate-assisted evolutionary algorithm for expensive multi-objective optimization. Neural Comput Appl 2022. [DOI: 10.1007/s00521-022-07097-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
13
|
Li JY, Zhan ZH, Zhang J. Evolutionary Computation for Expensive Optimization: A Survey. MACHINE INTELLIGENCE RESEARCH 2022. [PMCID: PMC8777172 DOI: 10.1007/s11633-022-1317-4] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 01/14/2023]
Abstract
Expensive optimization problem (EOP) widely exists in various significant real-world applications. However, EOP requires expensive or even unaffordable costs for evaluating candidate solutions, which is expensive for the algorithm to find a satisfactory solution. Moreover, due to the fast-growing application demands in the economy and society, such as the emergence of the smart cities, the internet of things, and the big data era, solving EOP more efficiently has become increasingly essential in various fields, which poses great challenges on the problem-solving ability of optimization approach for EOP. Among various optimization approaches, evolutionary computation (EC) is a promising global optimization tool widely used for solving EOP efficiently in the past decades. Given the fruitful advancements of EC for EOP, it is essential to review these advancements in order to synthesize and give previous research experiences and references to aid the development of relevant research fields and real-world applications. Motivated by this, this paper aims to provide a comprehensive survey to show why and how EC can solve EOP efficiently. For this aim, this paper firstly analyzes the total optimization cost of EC in solving EOP. Then, based on the analysis, three promising research directions are pointed out for solving EOP, which are problem approximation and substitution, algorithm design and enhancement, and parallel and distributed computation. Note that, to the best of our knowledge, this paper is the first that outlines the possible directions for efficiently solving EOP by analyzing the total expensive cost. Based on this, existing works are reviewed comprehensively via a taxonomy with four parts, including the above three research directions and the real-world application part. Moreover, some future research directions are also discussed in this paper. It is believed that such a survey can attract attention, encourage discussions, and stimulate new EC research ideas for solving EOP and related real-world applications more efficiently.
Collapse
|
14
|
Wang X, Jin Y, Schmitt S, Olhofer M, Allmendinger R. Transfer learning based surrogate assisted evolutionary bi-objective optimization for objectives with different evaluation times. Knowl Based Syst 2021. [DOI: 10.1016/j.knosys.2021.107190] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
15
|
Abstract
AbstractComplex continuous optimization problems widely exist nowadays due to the fast development of the economy and society. Moreover, the technologies like Internet of things, cloud computing, and big data also make optimization problems with more challenges including Many-dimensions, Many-changes, Many-optima, Many-constraints, and Many-costs. We term these as 5-M challenges that exist in large-scale optimization problems, dynamic optimization problems, multi-modal optimization problems, multi-objective optimization problems, many-objective optimization problems, constrained optimization problems, and expensive optimization problems in practical applications. The evolutionary computation (EC) algorithms are a kind of promising global optimization tools that have not only been widely applied for solving traditional optimization problems, but also have emerged booming research for solving the above-mentioned complex continuous optimization problems in recent years. In order to show how EC algorithms are promising and efficient in dealing with the 5-M complex challenges, this paper presents a comprehensive survey by proposing a novel taxonomy according to the function of the approaches, including reducing problem difficulty, increasing algorithm diversity, accelerating convergence speed, reducing running time, and extending application field. Moreover, some future research directions on using EC algorithms to solve complex continuous optimization problems are proposed and discussed. We believe that such a survey can draw attention, raise discussions, and inspire new ideas of EC research into complex continuous optimization problems and real-world applications.
Collapse
|
16
|
Zhao Y, Zeng J, Tan Y. Neighborhood samples and surrogate assisted multi-objective evolutionary algorithm for expensive many-objective optimization problems. Appl Soft Comput 2021. [DOI: 10.1016/j.asoc.2021.107268] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
17
|
He C, Huang S, Cheng R, Tan KC, Jin Y. Evolutionary Multiobjective Optimization Driven by Generative Adversarial Networks (GANs). IEEE TRANSACTIONS ON CYBERNETICS 2021; 51:3129-3142. [PMID: 32365041 DOI: 10.1109/tcyb.2020.2985081] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Recently, increasing works have been proposed to drive evolutionary algorithms using machine-learning models. Usually, the performance of such model-based evolutionary algorithms is highly dependent on the training qualities of the adopted models. Since it usually requires a certain amount of data (i.e., the candidate solutions generated by the algorithms) for model training, the performance deteriorates rapidly with the increase of the problem scales due to the curse of dimensionality. To address this issue, we propose a multiobjective evolutionary algorithm driven by the generative adversarial networks (GANs). At each generation of the proposed algorithm, the parent solutions are first classified into real and fake samples to train the GANs; then the offspring solutions are sampled by the trained GANs. Thanks to the powerful generative ability of the GANs, our proposed algorithm is capable of generating promising offspring solutions in high-dimensional decision space with limited training data. The proposed algorithm is tested on ten benchmark problems with up to 200 decision variables. The experimental results on these test problems demonstrate the effectiveness of the proposed algorithm.
Collapse
|
18
|
Evolutionary Optimisation for Reduction of the Low-Frequency Discrete-Spectrum Force of Marine Propeller Based on a Data-Driven Surrogate Model. JOURNAL OF MARINE SCIENCE AND ENGINEERING 2020. [DOI: 10.3390/jmse9010018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
For practical problems with non-convex, large-scale and highly constrained characteristics, evolutionary optimisation algorithms are widely used. However, advanced data-driven methods have yet to be comprehensively applied in related fields. In this study, a surrogate model combined with the Non-dominated Sorting Genetic Algorithm II-Differential Evolution (NSGA-II-DE) is applied to reduce the low-frequency Discrete-Spectrum (DS) force of propeller noise. Reduction of this force has drawn a lot of attention as it is the primary signal used in the sonar-based detection and identification of ships. In the present study, a surrogate model is proposed based on a trained Back-Propagation (BP) fully connected neural network, which improves the optimisation efficiency. The neural network is designed by analysing the depth and width of the hidden layers. The results indicate that a four-layer neural network with 64, 128, 256 and 64 nodes in each layer, respectively, exhibits the highest prediction accuracy. The prediction errors for the first order of DST, second order of DST and the thrust coefficient are only 0.21%, 5.71% and 0.01%, respectively. Data-Driven Evolutionary Optimisation (DDEO) is applied to a standard high-skew propeller to reduce DST. DDEO and a Traditional Evolutionary Optimisation Method (TEOM) obtain the same optimisation results, while the time cost of DDEO is only 0.68% that of the TEOM. Thus, the proposed DDEO is applicable to complex engineering problems in various fields.
Collapse
|
19
|
Cheng S, Ma L, Lu H, Lei X, Shi Y. Evolutionary computation for solving search-based data analytics problems. Artif Intell Rev 2020. [DOI: 10.1007/s10462-020-09882-x] [Citation(s) in RCA: 26] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
|
20
|
Weise T, Chen Y, Li X, Wu Z. Selecting a diverse set of benchmark instances from a tunable model problem for black-box discrete optimization algorithms. Appl Soft Comput 2020. [DOI: 10.1016/j.asoc.2020.106269] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
|
21
|
Wang H, Jin Y, Yang C, Jiao L. Transfer stacking from low-to high-fidelity: A surrogate-assisted bi-fidelity evolutionary algorithm. Appl Soft Comput 2020. [DOI: 10.1016/j.asoc.2020.106276] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
22
|
Wang X, Jin Y, Schmitt S, Olhofer M. An adaptive Bayesian approach to surrogate-assisted evolutionary multi-objective optimization. Inf Sci (N Y) 2020. [DOI: 10.1016/j.ins.2020.01.048] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
23
|
Wang H, Jin Y. A Random Forest-Assisted Evolutionary Algorithm for Data-Driven Constrained Multiobjective Combinatorial Optimization of Trauma Systems. IEEE TRANSACTIONS ON CYBERNETICS 2020; 50:536-549. [PMID: 30273180 DOI: 10.1109/tcyb.2018.2869674] [Citation(s) in RCA: 35] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Many real-world optimization problems can be solved by using the data-driven approach only, simply because no analytic objective functions are available for evaluating candidate solutions. In this paper, we address a class of expensive data-driven constrained multiobjective combinatorial optimization problems, where the objectives and constraints can be calculated only on the basis of a large amount of data. To solve this class of problems, we propose using random forests (RFs) and radial basis function networks as surrogates to approximate both objective and constraint functions. In addition, logistic regression models are introduced to rectify the surrogate-assisted fitness evaluations and a stochastic ranking selection is adopted to further reduce the influences of the approximated constraint functions. Three variants of the proposed algorithm are empirically evaluated on multiobjective knapsack benchmark problems and two real-world trauma system design problems. Experimental results demonstrate that the variant using RF models as the surrogates is effective and efficient in solving data-driven constrained multiobjective combinatorial optimization problems.
Collapse
|
24
|
Xu W, Xu JX, He D, Tan KC. An Evolutionary Constraint-Handling Technique for Parametric Optimization of a Cancer Immunotherapy Model. IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE 2019. [DOI: 10.1109/tetci.2018.2880516] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
25
|
Guo D, Jin Y, Ding J, Chai T. Heterogeneous Ensemble-Based Infill Criterion for Evolutionary Multiobjective Optimization of Expensive Problems. IEEE TRANSACTIONS ON CYBERNETICS 2019; 49:1012-1025. [PMID: 29994577 DOI: 10.1109/tcyb.2018.2794503] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Gaussian processes (GPs) are the most popular model used in surrogate-assisted evolutionary optimization of computationally expensive problems, mainly because GPs are able to measure the uncertainty of the estimated fitness values, based on which certain infill sampling criteria can be used to guide the search and update the surrogate model. However, the computation time for constructing GPs may become excessively long when the number of training samples increases, which makes it inappropriate to use them as surrogates in evolutionary optimization. To address this issue, this paper proposes to use ensembles as surrogates and infill criteria for model management in evolutionary optimization. A heterogeneous ensemble consisting of a least square support vector machine and two radial basis function networks is constructed to enhance the reliability of ensembles for uncertainty estimation. In addition to the original decision variables, a selected subset of the decision variables and a set of transformed variables are used as inputs of the heterogeneous ensemble to further promote the diversity of the ensemble. The proposed heterogeneous ensemble is compared with a GP and a homogeneous ensemble for infill sampling criteria in evolutionary multiobjective optimization. Experimental results demonstrate that the heterogeneous ensemble is competitive in performance compared with GPs and much more scalable in computational complexity to the increase in search dimension.
Collapse
|
26
|
A complete expected improvement criterion for Gaussian process assisted highly constrained expensive optimization. Inf Sci (N Y) 2019. [DOI: 10.1016/j.ins.2018.09.003] [Citation(s) in RCA: 21] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
27
|
|
28
|
Yang Z, Sendhoff B, Tang K, Yao X. Target shape design optimization by evolving B-splines with cooperative coevolution. Appl Soft Comput 2016. [DOI: 10.1016/j.asoc.2016.07.027] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
29
|
Evolutionary Computation and Big Data: Key Challenges and Future Directions. DATA MINING AND BIG DATA 2016. [DOI: 10.1007/978-3-319-40973-3_1] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/03/2022]
|
30
|
Computational Intelligence and Optimization for Transportation Big Data: Challenges and Opportunities. COMPUTATIONAL METHODS IN APPLIED SCIENCES 2015. [DOI: 10.1007/978-3-319-18320-6_7] [Citation(s) in RCA: 23] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/28/2023]
|
31
|
Cheng S, Ting TO, Yang XS. Large-Scale Global Optimization via Swarm Intelligence. SOLVING COMPUTATIONALLY EXPENSIVE ENGINEERING PROBLEMS 2014. [DOI: 10.1007/978-3-319-08985-0_10] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/20/2023]
|
32
|
Shim VA, Tan KC, Cheong CY, Chia JY. Enhancing the scalability of multi-objective optimization via restricted Boltzmann machine-based estimation of distribution algorithm. Inf Sci (N Y) 2013. [DOI: 10.1016/j.ins.2013.06.037] [Citation(s) in RCA: 20] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
33
|
Ding Y, Jin Y, Ren L, Hao K. An Intelligent Self-Organization Scheme for the Internet of Things. IEEE COMPUT INTELL M 2013. [DOI: 10.1109/mci.2013.2264251] [Citation(s) in RCA: 69] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
34
|
Evolutionary Complex Engineering Optimization: Opportunities and Challenges [Guest Editorial]. IEEE COMPUT INTELL M 2013. [DOI: 10.1109/mci.2013.2264563] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
35
|
Cheng S, Shi Y, Qin Q, Bai R. Swarm Intelligence in Big Data Analytics. INTELLIGENT DATA ENGINEERING AND AUTOMATED LEARNING – IDEAL 2013 2013. [DOI: 10.1007/978-3-642-41278-3_51] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
36
|
Cheng S, Shi Y, Qin Q. Population Diversity of Particle Swarm Optimizer Solving Single and Multi-Objective Problems. INTERNATIONAL JOURNAL OF SWARM INTELLIGENCE RESEARCH 2012. [DOI: 10.4018/jsir.2012100102] [Citation(s) in RCA: 22] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Premature convergence occurs in swarm intelligence algorithms searching for optima. A swarm intelligence algorithm has two kinds of abilities: exploration of new possibilities and exploitation of old certainties. The exploration ability means that an algorithm can explore more search place to increase the possibility that the algorithm can find good enough solutions. In contrast, the exploitation ability means that an algorithm focuses on the refinement of found promising areas. An algorithm should have a balance between exploration and exploitation, that is, the allocation of computational resources should be optimized to ensure that an algorithm can find good enough solutions effectively. The diversity measures the distribution of individuals’ information. From the observation of the distribution and diversity change, the degree of exploration and exploitation can be obtained. Another issue in multiobjective is the solution metric. Pareto domination is utilized to compare between two solutions, however, solutions are almost Pareto non-dominated for multi-objective problems with more than ten objectives. In this paper, the authors analyze the population diversity of particle swarm optimizer for solving both single objective and multiobjective problems. The population diversity of solutions is used to measure the goodness of a set of solutions. This metric may guide the search in problems with numerous objectives. Adaptive optimization algorithms can be designed through controlling the balance between exploration and exploitation.
Collapse
Affiliation(s)
- Shi Cheng
- University of Liverpool, Liverpool, UK
| | - Yuhui Shi
- Xi’an Jiaotong-Liverpool University, Dushu Lake, Higher Education Town, Suzhou, China
| | - Quande Qin
- Shenzhen University, Nanshan, Shenzhen, Guangdong, China
| |
Collapse
|
37
|
Chen CH, Liu TK, Huang IM, Chou JH. Multiobjective Synthesis of Six-Bar Mechanisms Under Manufacturing and Collision-Free Constraints. IEEE COMPUT INTELL M 2012. [DOI: 10.1109/mci.2011.2176996] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
38
|
Jin Y, Meng Y. Emergence of robust regulatory motifs from in silico evolution of sustained oscillation. Biosystems 2011; 103:38-44. [DOI: 10.1016/j.biosystems.2010.09.009] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2009] [Revised: 09/21/2010] [Accepted: 09/21/2010] [Indexed: 10/19/2022]
|