Multimodal Estimation of Distribution Algorithms

Solving Expensive Multimodal Optimization Problem by a Decomposition Differential Evolution Algorithm

An expensive multimodal optimization problem (EMMOP) is that the computation of the objective function is time consuming and it has multiple global optima. This article proposes a decomposition differential evolution (DE) based on the radial basis function (RBF) for EMMOPs, called D/REM. It mainly consists of two phases: the promising subregions detection (PSD) and the local search phase (LSP). In PSD, a population update strategy is designed and the mean-shift clustering is employed to predict the promising subregions of EMMOP. In LSP, a local RBF surrogate model is constructed for each promising subregion and each local RBF surrogate model tracks a global optimum of EMMOP. In this way, an EMMOP is decomposed into many expensive global optimization subproblems. To handle these subproblems, a popular DE variant, JADE, acts as the search engine to deal with these subproblems. A large number of numerical experiments unambiguously validate that D/REM can solve EMMOPs effectively and efficiently.

Generative Text Convolutional Neural Network for Hierarchical Document Representation Learning

For document analysis, existing methods often resort to the document representation that either discards the word order information or projects each word into a low-dimensional dense embedding vector. However, confined by the data's sparsity and high-dimensionality, limited effort has been made to explore the semantic structures underlying the document representation that formulates each document as a sequence of one-hot vectors, especially in the probabilistic modeling literature. To construct a probabilistic generative model for this type of document representation, we first develop convolutional Poisson factor analysis (CPFA) that not only utilizes the sparse property of data but also enables model parallelism. Through interleaving probabilistic Dirichlet-gamma pooling layers with learnable parameters, we extend the shallow CPFA into a generative text convolutional neural network (GTCNN), which captures richer semantic information with multiple probabilistic convolutional layers and can be coupled with existing deep topic models to alleviate their loss of word order. For efficient and scalable model inference, we not only develop both a parallel upward-downward Gibbs sampler and SG-MCMC based algorithm for training GTCNN, but also construct a hierarchical Weibull convolutional inference network for fast out-of-sample prediction. Experimental results on document representation learning tasks demonstrate the effectiveness of the proposed methods.

Multisurrogate-Assisted Multitasking Particle Swarm Optimization for Expensive Multimodal Problems

Many real-world applications can be formulated as expensive multimodal optimization problems (EMMOPs). When surrogate-assisted evolutionary algorithms (SAEAs) are employed to tackle these problems, they not only face the problem of selecting surrogate models but also need to tackle the problem of discovering and updating multiple modalities. Different optimization problems and different stages of evolutionary algorithms (EAs) generally require different types of surrogate models. To address this issue, in this article, we present a multisurrogate-assisted multitasking particle swarm optimization algorithm to seek multiple optimal solutions of EMMOPs at a low computational cost. The proposed algorithm first transforms an EMMOP into a multitasking optimization problem by integrating various surrogate models, and designs a multitasking niche particle swarm algorithm to solve it. Following that, a surrogate model management strategy based on the skill factor and clustering is developed to effectively balance the number of real function evaluations and the prediction accuracy of candidate optimal solutions. In addition, an adaptive local search strategy based on the trust region is proposed to enhance the capability of swarm in exploiting potential optimal modalities. We compare the proposed algorithm with five state-of-the-art SAEAs and seven multimodal EAs on 19 benchmark functions and the building energy conservation problem and experimental results show that the proposed algorithm can obtain multiple highly competitive optimal solutions.

Optimizing Niche Center for Multimodal Optimization Problems

Many real-world optimization problems require searching for multiple optimal solutions simultaneously, which are called multimodal optimization problems (MMOPs). For MMOPs, the algorithm is required both to enlarge population diversity for locating more global optima and to enhance refine ability for increasing the accuracy of the obtained solutions. Thus, numerous niching techniques have been proposed to divide the population into different niches, and each niche is responsible for searching on one or more peaks. However, it is often a challenge to distinguish proper individuals as niche centers in existing niching approaches, which has become a key issue for efficiently solving MMOPs. In this article, the niche center distinguish (NCD) problem is treated as an optimization problem and an NCD-based differential evolution (NCD-DE) algorithm is proposed. In NCD-DE, the niches are formed by using an internal genetic algorithm (GA) to online solve the NCD optimization problem. In the internal GA, a fitness-entropy measurement objective function is designed to evaluate whether a group of niche centers (i.e., encoded by a chromosome in the internal GA) is promising. Moreover, to enhance the exploration and exploitation abilities of NCD-DE in solving the MMOPs, a niching and global cooperative mutation strategy that uses both niche and population information is proposed to generate new individuals. The proposed NCD-DE is compared with some state-of-the-art and recent well-performing algorithms. The experimental results show that NCD-DE achieves better or competitive performance on both the accuracy and completeness of the solutions than the compared algorithms.

A novel ensemble estimation of distribution algorithm with distribution modification strategies

The canonical estimation of distribution algorithm (EDA) easily falls into a local optimum with an ill-shaped population distribution, which leads to weak convergence performance and less stability when solving global optimization problems. To overcome this defect, we explore a novel EDA variant with an ensemble of three distribution modification strategies, i.e., archive-based population updating (APU), multileader-based search diversification (MSD), and the triggered distribution shrinkage (TDS) strategy, named E3-EDA. The APU strategy utilizes historical population information to rebuild the search scope and avoid ill-shaped distributions. Moreover, it continuously updates the archive to avoid overfitting the distribution model. The MSD makes full use of the location differences among populations to evolve the sampling toward promising regions. TDS is triggered when the search stagnates, shrinking the distribution scope to achieve local exploitation. Additionally, the E3-EDA performance is evaluated using the CEC 2014 and CEC 2018 test suites on 10-dimensional, 30-dimensional, 50-dimensional and 100-dimensional problems. Moreover, several prominent EDA variants and other top methods from CEC competitions are comprehensively compared with the proposed method. The competitive performance of E3-EDA in solving complex problems is supported by the nonparametric test results.

An Integrated Cluster Detection, Optimization, and Interpretation Approach for Financial Data

In many financial applications, such as fraud detection, reject inference, and credit evaluation, detecting clusters automatically is critical because it helps to understand the subpatterns of the data that can be used to infer user's behaviors and identify potential risks. Due to the complexity of human behaviors and changing social environments, the distributions of financial data are usually complex and it is challenging to find clusters and give reasonable interpretations. The goal of this study is to develop an integrated approach to detect clusters in financial data, and optimize the scope of the clusters such that the clusters can be easily interpreted. Specifically, we first proposed a new cluster quality evaluation criterion, which is free from large-scale computation and can guide base clustering algorithms such as k -Means to detect hyperellipsoidal clusters adaptively. Then, we designed a new solver for a revised support vector data description model, which efficiently refines the centroids and scopes of the detected clusters to make the clusters tighter such that the data in the clusters share greater similarities, and thus, the clusters can be easily interpreted with eigenvectors. Using ten financial datasets, the experiments showed that the proposed algorithm can efficiently find reasonable number of clusters. The proposed approach is suitable for large-scale financial datasets whose features are meaningful, and also applicable to financial mining tasks, such as data distribution interpretation and anomaly detection.

Multimodal Weibull Variational Autoencoder for Jointly Modeling Image-Text Data

For multimodal representation learning, traditional black-box approaches often fall short of extracting interpretable multilayer hidden structures, which contribute to visualize the connections between different modalities at multiple semantic levels. To extract interpretable multimodal latent representations and visualize the hierarchial semantic relationships between different modalities, based on deep topic models, we develop a novel multimodal Poisson gamma belief network (mPGBN) that tightly couples the observations of different modalities via imposing sparse connections between their modality-specific hidden layers. To alleviate the time-consuming Gibbs sampler adopted by traditional topic models in the testing stage, we construct a Weibull-based variational inference network (encoder) to directly map the observations to their latent representations, and further combine it with the mPGBN (decoder), resulting in a novel multimodal Weibull variational autoencoder (MWVAE), which is fast in out-of-sample prediction and can handle large-scale multimodal datasets. Qualitative evaluations on bimodal data consisting of image-text pairs show that the developed MWVAE can successfully extract expressive multimodal latent representations for downstream tasks like missing modality imputation and multimodal retrieval. Further extensive quantitative results demonstrate that both MWVAE and its supervised extension sMWVAE achieve state-of-the-art performance on various multimodal benchmarks.

A Penalty-Based Differential Evolution for Multimodal Optimization

It is very difficult to locate multiple global optimal solutions (GOSs) of multimodal optimization problems (MMOPs). To deal with this issue, a penalty-based multimodal optimization differential evolution (DE), called PMODE, is developed in this article. In PMODE, a penalty strategy with a dynamic penalty radius is constructed to solve MMOPs. An elite selection mechanism is designed to identify and select elite solutions. The neighboring areas of these elite solutions are penalized. PMODE uses a popular DE variant-JADE as its search engine. The proposed PMODE is compared with several other state-of-the-art multimodal optimization algorithms on 20 MMOPs used in the IEEE CEC2013 special session. The experimental results show that PMODE performs better than other state-of-the-art methods.

Adaptive Estimation Distribution Distributed Differential Evolution for Multimodal Optimization Problems

Multimodal optimization problems (MMOPs) require algorithms to locate multiple optima simultaneously. When using evolutionary algorithms (EAs) to deal with MMOPs, an intuitive idea is to divide the population into several small "niches," where different niches focus on locating different optima. These population partition strategies are called "niching" techniques, which have been frequently used for MMOPs. The algorithms for simultaneously locating multiple optima of MMOPs are called multimodal algorithms. However, many multimodal algorithms still face the difficulty of population partition since most of the niching techniques involve the sensitive niching parameters. Considering this issue, in this article, we propose a parameter-free niching method based on adaptive estimation distribution (AED) and develop a distributed differential evolution (DDE) algorithm, which is called AED-DDE, for solving MMOPs. In AED-DDE, each individual finds its own appropriate niche size to form a niche and acts as an independent unit to find a global optimum. Therefore, we can avoid the difficulty of population partition and the sensitivity of niching parameters. Different niches are co-evolved by using the master-slave multiniche distributed model. The multiniche co-evolution mechanism can improve the population diversity for fully exploring the search space and finding more global optima. Moreover, the AED-DDE algorithm is further enhanced by a probabilistic local search (PLS) to refine the solution accuracy. Compared with other multimodal algorithms, even the winner of CEC2015 multimodal competition, the comparison results fully demonstrate the superiority of AED-DDE.

A Differential Evolution Algorithm With Adaptive Niching and K-Means Operation for Data Clustering

Clustering, as an important part of data mining, is inherently a challenging problem. This article proposes a differential evolution algorithm with adaptive niching and k -means operation (denoted as DE_ANS_AKO) for partitional data clustering. Within the proposed algorithm, an adaptive niching scheme, which can dynamically adjust the size of each niche in the population, is devised and integrated to prevent premature convergence of evolutionary search, thus appropriately searching the space to identify the optimal or near-optimal solution. Furthermore, to improve the search efficiency, an adaptive k -means operation has been designed and employed at the niche level of population. The performance of the proposed algorithm has been evaluated on synthetic as well as real datasets and compared with related methods. The experimental results reveal that the proposed algorithm is able to reliably and efficiently deliver high quality clustering solutions and generally outperforms related methods implemented for comparisons.

DMIB: Dual-Correlated Multivariate Information Bottleneck for Multiview Clustering

Multiview clustering (MVC) has recently been the focus of much attention due to its ability to partition data from multiple views via view correlations. However, most MVC methods only learn either interfeature correlations or intercluster correlations, which may lead to unsatisfactory clustering performance. To address this issue, we propose a novel dual-correlated multivariate information bottleneck (DMIB) method for MVC. DMIB is able to explore both interfeature correlations (the relationship among multiple distinct feature representations from different views) and intercluster correlations (the close agreement among clustering results obtained from individual views). For the former, we integrate both view-shared feature correlations discovered by learning a shared discriminative feature subspace and view-specific feature information to fully explore the interfeature correlation. This allows us to attain multiple reliable local clustering results of different views. Following this, we explore the intercluster correlations by learning the shared mutual information over different local clusterings for an improved global partition. By integrating both correlations, we formulate the problem as a unified information maximization function and further design a two-step method for optimization. Moreover, we theoretically prove the convergence of the proposed algorithm, and discuss the relationships between our method and several existing clustering paradigms. The experimental results on multiple datasets demonstrate the superiority of DMIB compared to several state-of-the-art clustering methods.

Predominant Cognitive Learning Particle Swarm Optimization for Global Numerical Optimization

Particle swarm optimization (PSO) has witnessed giant success in problem optimization. Nevertheless, its optimization performance seriously degrades when coping with optimization problems with a lot of local optima. To alleviate this issue, this paper designs a predominant cognitive learning particle swarm optimization (PCLPSO) method to effectively tackle complicated optimization problems. Specifically, for each particle, a new promising exemplar is constructed by letting its personal best position cognitively learn from a better personal experience randomly selected from those of others based on a novel predominant cognitive learning strategy. As a result, different particles preserve different guiding exemplars. In this way, the learning effectiveness and the learning diversity of particles are expectedly improved. To eliminate the dilemma that PCLPSO is sensitive to the involved parameters, we propose dynamic adjustment strategies, so that different particles preserve different parameter settings, which is further beneficial to promote the learning diversity of particles. With the above techniques, the proposed PCLPSO could expectedly compromise the search intensification and diversification in a good way to search the complex solution space properly to achieve satisfactory performance. Comprehensive experiments are conducted on the commonly adopted CEC 2017 benchmark function set to testify the effectiveness of the devised PCLPSO. Experimental results show that PCLPSO obtains considerably competitive or even much more promising performance than several representative and state-of-the-art peer methods.

Novel Multitask Conditional Neural-Network Surrogate Models for Expensive Optimization

Multiple-related tasks can be learned simultaneously by sharing information among tasks to avoid tabula rasa learning and to improve performance in the no transfer case (i.e., when each task learns in isolation). This study investigates multitask learning with conditional neural process (CNP) networks and proposes two multitask learning network models on the basis of CNPs, namely, the one-to-many multitask CNP (OMc-MTCNP) and the many-to-many MTCNP (MMc-MTCNP). Compared with existing multitask models, the proposed models add an extensible correlation learning layer to learn the correlation among tasks. Moreover, the proposed multitask CNP (MTCNP) networks are regarded as surrogate models and applied to a Bayesian optimization framework to replace the Gaussian process (GP) to avoid the complex covariance calculation. The proposed Bayesian optimization framework simultaneously infers multiple tasks by utilizing the possible dependencies among them to share knowledge across tasks. The proposed surrogate models augment the observed dataset with a number of related tasks to estimate model parameters confidently. The experimental studies under several scenarios indicate that the proposed algorithms are competitive in performance compared with GP-, single-task-, and other multitask model-based Bayesian optimization methods.

A Memetic Algorithm Based on Probability Learning for Solving the Multidimensional Knapsack Problem

The multidimensional knapsack problem (MKP) is a well-known combinatorial optimization problem with many real-life applications. In this article, a memetic algorithm based on probability learning (MA/PL) is proposed to solve MKP. The main highlights of this article are two-fold: 1) problem-dependent heuristics for MKP and 2) a novel framework of MA/PL. For the problem-dependent heuristics, we first propose two kinds of logarithmic utility functions (LUFs) based on the special structure of MKP, in which the profit value and weight vector of each item are considered simultaneously. Then, LUFs are applied to effectively guide the repair operator for infeasible solutions and the local search operator. For the framework of MA/PL, we propose two problem-dependent probability distributions to extract the special knowledge of MKP, that is, the marginal probability distribution (MPD) of each item and the joint probability distribution (JPD) of two conjoint items. Next, learning rules for MPD and JPD, which borrow ideas from competitive learning and binary Markov chain, are proposed. Thereafter, we generate MA/PL's offspring by integrating MPD and JPD, such that the univariate probability information of each item as well as the dependency of conjoint items can be sufficiently used. Results of experiments on 179 benchmark instances and a real-life case study demonstrate the effectiveness and practical values of the proposed MKP.

A Dimension Group-Based Comprehensive Elite Learning Swarm Optimizer for Large-Scale Optimization

High-dimensional optimization problems are more and more common in the era of big data and the Internet of things (IoT), which seriously challenge the optimization performance of existing optimizers. To solve these kinds of problems effectively, this paper devises a dimension group-based comprehensive elite learning swarm optimizer (DGCELSO) by integrating valuable evolutionary information in different elite particles in the swarm to guide the updating of inferior ones. Specifically, the swarm is first separated into two exclusive sets, namely the elite set (ES) containing the top best individuals, and the non-elite set (NES), consisting of the remaining individuals. Then, the dimensions of each particle in NES are randomly divided into several groups with equal sizes. Subsequently, each dimension group of each non-elite particle is guided by two different elites randomly selected from ES. In this way, each non-elite particle in NES is comprehensively guided by multiple elite particles in ES. Therefore, not only could high diversity be maintained, but fast convergence is also likely guaranteed. To alleviate the sensitivity of DGCELSO to the associated parameters, we further devise dynamic adjustment strategies to change the parameter settings during the evolution. With the above mechanisms, DGCELSO is expected to explore and exploit the solution space properly to find the optimum solutions for optimization problems. Extensive experiments conducted on two commonly used large-scale benchmark problem sets demonstrate that DGCELSO achieves highly competitive or even much better performance than several state-of-the-art large-scale optimizers.

Stochastic Triad Topology Based Particle Swarm Optimization for Global Numerical Optimization

Particle swarm optimization (PSO) has exhibited well-known feasibility in problem optimization. However, its optimization performance still encounters challenges when confronted with complicated optimization problems with many local areas. In PSO, the interaction among particles and utilization of the communication information play crucial roles in improving the learning effectiveness and learning diversity of particles. To promote the communication effectiveness among particles, this paper proposes a stochastic triad topology to allow each particle to communicate with two random ones in the swarm via their personal best positions. Then, unlike existing studies that employ the personal best positions of the updated particle and the neighboring best position of the topology to direct its update, this paper adopts the best one and the mean position of the three personal best positions in the associated triad topology as the two guiding exemplars to direct the update of each particle. To further promote the interaction diversity among particles, an archive is maintained to store the obsolete personal best positions of particles and is then used to interact with particles in the triad topology. To enhance the chance of escaping from local regions, a random restart strategy is probabilistically triggered to introduce initialized solutions to the archive. To alleviate sensitivity to parameters, dynamic adjustment strategies are designed to dynamically adjust the associated parameter settings during the evolution. Integrating the above mechanism, a stochastic triad topology-based PSO (STTPSO) is developed to effectively search complex solution space. With the above techniques, the learning diversity and learning effectiveness of particles are largely promoted and thus the developed STTPSO is expected to explore and exploit the solution space appropriately to find high-quality solutions. Extensive experiments conducted on the commonly used CEC 2017 benchmark problem set with different dimension sizes substantiate that the proposed STTPSO achieves highly competitive or even much better performance than state-of-the-art and representative PSO variants.

An estimation of distribution algorithm with clustering for scenario-based robust financial optimization

One important problem in financial optimization is to search for robust investment plans that can maximize return while minimizing risk. The market environment, namely the scenario of the problem in optimization, always affects the return and risk of an investment plan. Those financial optimization problems that the performance of the investment plans largely depends on the scenarios are defined as scenario-based optimization problems. This kind of uncertainty is called scenario-based uncertainty. The consideration of scenario-based uncertainty in multi-objective optimization problem is a largely under explored domain. In this paper, a nondominated sorting estimation of distribution algorithm with clustering (NSEDA-C) is proposed to deal with scenario-based robust financial problems. A robust group insurance portfolio problem is taken as an instance to study the features of scenario-based robust financial problems. A simplified simulation method is applied to measure the return while an estimation model is devised to measure the risk. Applications of the NSEDA-C on the group insurance portfolio problem for real-world insurance products have validated the effectiveness of the proposed algorithm.

An Adaptive Stochastic Dominant Learning Swarm Optimizer for High-Dimensional Optimization

High-dimensional problems are ubiquitous in many fields, yet still remain challenging to be solved. To tackle such problems with high effectiveness and efficiency, this article proposes a simple yet efficient stochastic dominant learning swarm optimizer. Particularly, this optimizer not only compromises swarm diversity and convergence speed properly, but also consumes as little computing time and space as possible to locate the optima. In this optimizer, a particle is updated only when its two exemplars randomly selected from the current swarm are its dominators. In this way, each particle has an implicit probability to directly enter the next generation, making it possible to maintain high swarm diversity. Since each updated particle only learns from its dominators, good convergence is likely to be achieved. To alleviate the sensitivity of this optimizer to newly introduced parameters, an adaptive parameter adjustment strategy is further designed based on the evolutionary information of particles at the individual level. Finally, extensive experiments on two high dimensional benchmark sets substantiate that the devised optimizer achieves competitive or even better performance in terms of solution quality, convergence speed, scalability, and computational cost, compared to several state-of-the-art methods. In particular, experimental results show that the proposed optimizer performs excellently on partially separable problems, especially partially separable multimodal problems, which are very common in real-world applications. In addition, the application to feature selection problems further demonstrates the effectiveness of this optimizer in tackling real-world problems.

Stochastic Cognitive Dominance Leading Particle Swarm Optimization for Multimodal Problems

Optimization problems become increasingly complicated in the era of big data and Internet of Things, which significantly challenges the effectiveness and efficiency of existing optimization methods. To effectively solve this kind of problems, this paper puts forward a stochastic cognitive dominance leading particle swarm optimization algorithm (SCDLPSO). Specifically, for each particle, two personal cognitive best positions are first randomly selected from those of all particles. Then, only when the cognitive best position of the particle is dominated by at least one of the two selected ones, this particle is updated by cognitively learning from the better personal positions; otherwise, this particle is not updated and directly enters the next generation. With this stochastic cognitive dominance leading mechanism, it is expected that the learning diversity and the learning efficiency of particles in the proposed optimizer could be promoted, and thus the optimizer is expected to explore and exploit the solution space properly. At last, extensive experiments are conducted on a widely acknowledged benchmark problem set with different dimension sizes to evaluate the effectiveness of the proposed SCDLPSO. Experimental results demonstrate that the devised optimizer achieves highly competitive or even much better performance than several state-of-the-art PSO variants.

A Probabilistic Niching Evolutionary Computation Framework Based on Binary Space Partitioning

Multimodal optimization problems have multiple satisfactory solutions to identify. Most of the existing works conduct the search based on the information of the current population, which can be inefficient. This article proposes a probabilistic niching evolutionary computation framework that guides the future search based on more sufficient historical information, in order to locate diverse and high-quality solutions. A binary space partition tree is built to structurally organize the space visiting information. Based on the tree, a probabilistic niching strategy is defined to reinforce exploration and exploitation by making full use of the structural historical information. The proposed framework is universal for incorporating various baseline niching algorithms. In this article, we integrate the proposed framework with two niching algorithms: 1) a distance-based differential evolution algorithm and 2) a topology-based particle swarm optimization algorithm. The two new algorithms are evaluated on 20 multimodal optimization test functions. The experimental results show that the proposed framework helps the algorithms obtain competitive performance. They outperform a number of state-of-the-art niching algorithms on most of the test functions.

An Adaptive Covariance Scaling Estimation of Distribution Algorithm

Optimization problems are ubiquitous in every field, and they are becoming more and more complex, which greatly challenges the effectiveness of existing optimization methods. To solve the increasingly complicated optimization problems with high effectiveness, this paper proposes an adaptive covariance scaling estimation of distribution algorithm (ACSEDA) based on the Gaussian distribution model. Unlike traditional EDAs, which estimate the covariance and the mean vector, based on the same selected promising individuals, ACSEDA calculates the covariance according to an enlarged number of promising individuals (compared with those for the mean vector). To alleviate the sensitivity of the parameters in promising individual selections, this paper further devises an adaptive promising individual selection strategy for the estimation of the mean vector and an adaptive covariance scaling strategy for the covariance estimation. These two adaptive strategies dynamically adjust the associated numbers of promising individuals as the evolution continues. In addition, we further devise a cross-generation individual selection strategy for the parent population, used to estimate the probability distribution by combing the sampled offspring in the last generation and the one in the current generation. With the above mechanisms, ACSEDA is expected to compromise intensification and diversification of the search process to explore and exploit the solution space and thus could achieve promising performance. To verify the effectiveness of ACSEDA, extensive experiments are conducted on 30 widely used benchmark optimization problems with different dimension sizes. Experimental results demonstrate that the proposed ACSEDA presents significant superiority to several state-of-the-art EDA variants, and it preserves good scalability in solving optimization problems.

A survey on evolutionary computation for complex continuous optimization

Complex continuous optimization problems widely exist nowadays due to the fast development of the economy and society. Moreover, the technologies like Internet of things, cloud computing, and big data also make optimization problems with more challenges including Many-dimensions, Many-changes, Many-optima, Many-constraints, and Many-costs. We term these as 5-M challenges that exist in large-scale optimization problems, dynamic optimization problems, multi-modal optimization problems, multi-objective optimization problems, many-objective optimization problems, constrained optimization problems, and expensive optimization problems in practical applications. The evolutionary computation (EC) algorithms are a kind of promising global optimization tools that have not only been widely applied for solving traditional optimization problems, but also have emerged booming research for solving the above-mentioned complex continuous optimization problems in recent years. In order to show how EC algorithms are promising and efficient in dealing with the 5-M complex challenges, this paper presents a comprehensive survey by proposing a novel taxonomy according to the function of the approaches, including reducing problem difficulty, increasing algorithm diversity, accelerating convergence speed, reducing running time, and extending application field. Moreover, some future research directions on using EC algorithms to solve complex continuous optimization problems are proposed and discussed. We believe that such a survey can draw attention, raise discussions, and inspire new ideas of EC research into complex continuous optimization problems and real-world applications.

Solving nonlinear equation systems via clustering-based adaptive speciation differential evolution

In numerical computation, locating multiple roots of nonlinear equations (NESs) in a single run is a challenging work. In order to solve the problem of population grouping and parameters settings during the evolutionary, a clustering-based adaptive speciation differential evolution, referred to as CASDE, is presented to deal with NESs. CASDE offers three advantages: 1) the clustering with dynamic clustering sizes is used to set clustering sizes for different problems; 2) adaptive parameter control at the niche level is proposed to enhance the search ability and efficiency; 3) re-initialization mechanism motivates the algorithm to search new roots and saves computing resources. To evaluate the performance of CASDE, we select 30 problems with different features as test suite. Experimental results indicate that the speciation clustering with dynamic clustering sizes, niche adaptive parameter control, and re-initialization mechanism when combined together in a synergistic manner can improve the ability to find multiple roots in a single run. Additionally, our method is also compared with other state-of-the-art methods, which is capable of obtaining better results in terms of peak ratio and success rate. Finally, two practical mechanical problems are used to verify the performance of CASDE, and it also demonstrates superior results.

A niching cross-entropy method for multimodal satellite layout optimization design

Satellite layout optimization design (SLOD) relies on solving a high-dimensional and multimodal optimization problem, in which there exist multiple global optimal solutions. Existing algorithms for SLOD focus on seeking only one approximate global optimum. However, finding multiple solutions simultaneously could provide more design diversity for the designers. To alleviate this problem, multimodal optimization method is studied for SLOD in this paper, and an improved niching-based cross-entropy method (INCE) is proposed. INCE consists of an improved niching strategy, cross-entropy method-based offspring generation and a cross operator. CEC2013 benchmarks and satellite layout optimization design problem are investigated to verify the validity and feasibility of the proposed INCE. Compared with several state-of-the-art algorithms, the proposed algorithm performs better.

Differential Evolution for Multimodal Optimization With Species by Nearest-Better Clustering

Multimodal optimization problems (MMOPs) are common in real-world applications and involve identifying multiple optimal solutions for decision makers to choose from. The core requirement for dealing with such problems is to balance the ability of exploration in the global space and exploitation in the multiple optimal areas. In this paper, based on the differential evolution (DE), we propose a novel algorithm focusing on the formulation, balance, and keypoint of species for MMOPs, called FBK-DE. First, nearest-better clustering (NBC) is used to divide the population into multiple species with minimum size limitations. Second, to avoid placing too many individuals into one species, a species balance strategy is proposed to adjust the size of each species. Third, two keypoint-based mutation operators named DE/keypoint/1 and DE/keypoint/2 are proposed to evolve each species together with traditional mutation operators. The experimental results of FBK-DE on 20 benchmark functions are compared with 15 state-of-the-art multimodal optimization algorithms. The comparisons show that the proposed FBK-DE performs competitively with these algorithms.

Overlapping Community Detection in Directed and Undirected Attributed Networks Using a Multiobjective Evolutionary Algorithm

In many real-world networks, the structural connections of networks and the attributes about each node are always available. We typically call such graphs attributed networks, in which attributes always play the same important role in community detection as the topological structure. It is shown that the very existence of overlapping communities is one of the most important characteristics of various complex networks, while the majority of the existing community detection methods was designed for detecting separated communities in attributed networks. Therefore, it is quite challenging to detect meaningful overlapping structures with the combination of node attributes and topological structures. Therefore, in this article, we propose a multiobjective evolutionary algorithm based on the similarity attribute for overlapping community detection in attributed networks (MOEA-SA _OV ). In MOEA-SA _OV , a modified extended modularity EQ_OV , dealing with both directed and undirected networks, is well designed as the first objective. Another objective employed is the attribute similarity S_A . Then, a novel encoding and decoding strategy is designed to realize the goal of representing overlapping communities efficiently. MOEA-SA _OV runs under the framework of the nondominated sorting genetic algorithm II (NSGA-II) and can automatically determine the number of communities. In the experiments, the performance of MOEA-SA _OV is validated on both synthetic and real-world networks, and the experimental results demonstrate that our method can effectively find Pareto fronts about overlapping community structures with practical significance in both directed and undirected attributed networks.

Estimation of Distribution Algorithms with Fuzzy Sampling for Stochastic Programming Problems

Generating practical methods for simulation-based optimization has attracted a great deal of attention recently. In this paper, the estimation of distribution algorithms are used to solve nonlinear continuous optimization problems that contain noise. One common approach to dealing with these problems is to combine sampling methods with optimal search methods. Sampling techniques have a serious problem when the sample size is small, so estimating the objective function values with noise is not accurate in this case. In this research, a new sampling technique is proposed based on fuzzy logic to deal with small sample sizes. Then, simulation-based optimization methods are designed by combining the estimation of distribution algorithms with the proposed sampling technique and other sampling techniques to solve the stochastic programming problems. Moreover, additive versions of the proposed methods are developed to optimize functions without noise in order to evaluate different efficiency levels of the proposed methods. In order to test the performance of the proposed methods, different numerical experiments were carried out using several benchmark test functions. Finally, three real-world applications are considered to assess the performance of the proposed methods.

Local Binary Pattern-Based Adaptive Differential Evolution for Multimodal Optimization Problems

The multimodal optimization problem (MMOP) requires the algorithm to find multiple global optima of the problem simultaneously. In order to solve MMOP efficiently, a novel differential evolution (DE) algorithm based on the local binary pattern (LBP) is proposed in this paper. The LBP makes use of the neighbors' information for extracting relevant pattern information, so as to identify the multiple regions of interests, which is similar to finding multiple peaks in MMOP. Inspired by the principle of LBP, this paper proposes an LBP-based adaptive DE (LBPADE) algorithm. It enables the LBP operator to form multiple niches, and further to locate multiple peak regions in MMOP. Moreover, based on the LBP niching information, we develop a niching and global interaction (NGI) mutation strategy and an adaptive parameter strategy (APS) to fully search the niching areas and maintain multiple peak regions. The proposed NGI mutation strategy incorporates information from both the niching and the global areas for effective exploration, while APS adjusts the parameters of each individual based on its own LBP information and guides the individual to the promising direction. The proposed LBPADE algorithm is evaluated on the extensive MMOPs test functions. The experimental results show that LBPADE outperforms or at least remains competitive with some state-of-the-art algorithms.

A Distributed Swarm Optimizer With Adaptive Communication for Large-Scale Optimization

Large-scale optimization with high dimensionality and high computational cost becomes ubiquitous nowadays. To tackle such challenging problems efficiently, devising distributed evolutionary computation algorithms is imperative. To this end, this paper proposes a distributed swarm optimizer based on a special master-slave model. Specifically, in this distributed optimizer, the master is mainly responsible for communication with slaves, while each slave iterates a swarm to traverse the solution space. An asynchronous and adaptive communication strategy based on the request-response mechanism is especially devised to let the slaves communicate with the master efficiently. Particularly, the communication between the master and each slave is adaptively triggered during the iteration. To aid the slaves to search the space efficiently, an elite-guided learning strategy is especially designed via utilizing elite particles in the current swarm and historically best solutions found by different slaves to guide the update of particles. Together, this distributed optimizer asynchronously iterates multiple swarms to collaboratively seek the optimum in parallel. Extensive experiments on a widely used large-scale benchmark set substantiate that the distributed optimizer could: 1) achieve competitive effectiveness in terms of solution quality as compared to the state-of-the-art large-scale methods; 2) accelerate the execution of the algorithm in comparison with the sequential one and obtain almost linear speedup as the number of cores increases; and 3) preserve a good scalability to solve higher dimensional problems.

Simulation-Based EDAs for Stochastic Programming Problems

With the rapid growth of simulation software packages, generating practical tools for simulation-based optimization has attracted a lot of interest over the last decades. In this paper, a modified method of Estimation of Distribution Algorithms (EDAs) is constructed by a combination with variable-sample techniques to deal with simulation-based optimization problems. Moreover, a new variable-sample technique is introduced to support the search process whenever the sample sizes are small, especially in the beginning of the search process. The proposed method shows efficient results by simulating several numerical experiments.

Enhancing Gaussian Estimation of Distribution Algorithm by Exploiting Evolution Direction With Archive

As a typical model-based evolutionary algorithm, estimation of distribution algorithm (EDA) possesses unique characteristics and has been widely applied in global optimization. However, the commonly used Gaussian EDA (GEDA) usually suffers from premature convergence, which severely limits its search efficiency. This paper first systematically analyzes the reasons for the deficiency of traditional GEDA, then tries to enhance its performance by exploiting the evolution direction, and finally develops a new GEDA variant named EDA². Instead of only utilizing some good solutions produced in the current generation to estimate the Gaussian model, EDA² preserves a certain number of high-quality solutions generated in the previous generations into an archive and employs these historical solutions to assist estimating the covariance matrix of Gaussian model. By this means, the evolution direction information hidden in the archive is naturally integrated into the estimated model, which in turn can guide EDA² toward more promising solution regions. Moreover, the new estimation method significantly reduces the population size of EDA² since it needs fewer individuals in the current population for model estimation. As a result, a fast convergence can be achieved. To verify the efficiency of EDA², we tested it on a variety of benchmark functions and compared it with several state-of-the-art EAs. The experimental results demonstrate that EDA² is efficient and competitive.