Feature Selection Problem and Metaheuristics: A Systematic Literature Review about Its Formulation, Evaluation and Applications

Feature selection is becoming a relevant problem within the field of machine learning. The feature selection problem focuses on the selection of the small, necessary, and sufficient subset of features that represent the general set of features, eliminating redundant and irrelevant information. Given the importance of the topic, in recent years there has been a boom in the study of the problem, generating a large number of related investigations. Given this, this work analyzes 161 articles published between 2019 and 2023 (20 April 2023), emphasizing the formulation of the problem and performance measures, and proposing classifications for the objective functions and evaluation metrics. Furthermore, an in-depth description and analysis of metaheuristics, benchmark datasets, and practical real-world applications are presented. Finally, in light of recent advances, this review paper provides future research opportunities.

A Controlled Strengthened Dominance Relation for Evolutionary Many-Objective Optimization

Maintaining a balance between convergence and diversity is particularly crucial in evolutionary multiobjective optimization. Recently, a novel dominance relation called "strengthened dominance relation" (SDR) is proposed, which outperforms the existing dominance relations in balancing convergence and diversity. In this article, two points that influence the performance of SDR are studied and a new dominance relation, which is mainly based on SDR, is proposed (CSDR). An adaptation strategy is presented to dynamically adjust the dominance relation according to the current generation number. The CSDR is embedded into NSGA-II to substitute the Pareto dominance, labeled as NSGA-II/CSDR. The performance of our proposed method is validated by comparing it with five state-of-the-art algorithms on commonly used benchmark problems. NSGA-II/CSDR outperforms other algorithms in the most test instances considering both convergence and diversity.

Multipath Error Modeling Methodology for GNSS Integrity Monitoring Using a Global Optimization Strategy

Valid multipath error model is the prerequisite for high-performance GNSS integrity monitoring. It is indispensable to civil aviation and other Safety-of-Life (SoL) users. The model must perfectly bound multipath error while preventing the constructed model from being too conservative. Nevertheless, no sound methodologies to meet both the requirements have been introduced in previous literatures, and subsequently, practices always require iterative manual trade-offs. To improve the efficiency of multipath modeling, we propose a new automatic multipath error modeling methodology. It quantifies the above requirements in the objective function of multiobjective genetic algorithm (GA) so that multipath modeling can be managed automatically. Moreover, through introducing a new model that is based on two inflation factors, conservatism of modeling results can be significantly reduced. Experiments based on a 4-month dataset of BDS-3 Medium Earth Orbit (MEO) satellites show that constructed multipath models effectively bound actual error in each elevation bin. In addition, the new model form with two inflation factors brings average CDF difference reduction of 67.4% at B1I and 50.6% at B3I, which means significantly mitigation in terms of conservatism.

An Ensemble Framework of Evolutionary Algorithm for Constrained Multi-Objective Optimization

In the real-world, symmetry or asymmetry widely exists in various problems. Some of them can be formulated as constrained multi-objective optimization problems (CMOPs). During the past few years, handling CMOPs by evolutionary algorithms has become more popular. Lots of constrained multi-objective optimization evolutionary algorithms (CMOEAs) have been proposed. Whereas different CMOEAs may be more suitable for different CMOPs, it is difficult to choose the best one for a CMOP at hand. In this paper, we propose an ensemble framework of CMOEAs that aims to achieve better versatility on handling diverse CMOPs. In the proposed framework, the hypervolume indicator is used to evaluate the performance of CMOEAs, and a decreasing mechanism is devised to delete the poorly performed CMOEAs and to gradually determine the most suitable CMOEA. A new CMOEA, namely ECMOEA, is developed based on the framework and three state-of-the-art CMOEAs. Experimental results on five benchmarks with totally 52 instances demonstrate the effectiveness of our approach. In addition, the superiority of ECMOEA is verified through comparisons to seven state-of-the-art CMOEAs. Moreover, the effectiveness of ECMOEA on the real-world problems is also evaluated for eight instances.

Improved Sparrow Search Algorithm Based on Iterative Local Search

This paper solves the shortcomings of sparrow search algorithm in poor utilization to the current individual and lack of effective search, improves its search performance, achieves good results on 23 basic benchmark functions and CEC 2017, and effectively improves the problem that the algorithm falls into local optimal solution and has low search accuracy. This paper proposes an improved sparrow search algorithm based on iterative local search (ISSA). In the global search phase of the followers, the variable helix factor is introduced, which makes full use of the individual's opposite solution about the origin, reduces the number of individuals beyond the boundary, and ensures the algorithm has a detailed and flexible search ability. In the local search phase of the followers, an improved iterative local search strategy is adopted to increase the search accuracy and prevent the omission of the optimal solution. By adding the dimension by dimension lens learning strategy to scouters, the search range is more flexible and helps jump out of the local optimal solution by changing the focusing ability of the lens and the dynamic boundary of each dimension. Finally, the boundary control is improved to effectively utilize the individuals beyond the boundary while retaining the randomness of the individuals. The ISSA is compared with PSO, SCA, GWO, WOA, MWOA, SSA, BSSA, CSSA, and LSSA on 23 basic functions to verify the optimization performance of the algorithm. In addition, in order to further verify the optimization performance of the algorithm when the optimal solution is not 0, the above algorithms are compared in CEC 2017 test function. The simulation results show that the ISSA has good universality. Finally, this paper applies ISSA to PID parameter tuning and robot path planning, and the results show that the algorithm has good practicability and effect.

Running Time Analysis of MOEA/D on Pseudo-Boolean Functions

Decomposition-based multiobjective evolutionary algorithms (MOEAs) are a class of popular methods for solving the multiobjective optimization problems (MOPs), and have been widely studied in numerical experiments and successfully applied in practice. However, we know little about these algorithms from the theoretical aspect. In this paper, we present running time analysis of a simple MOEA with mutation and crossover based on the MOEA/D framework (MOEA/D-C) on five pseudo-Boolean functions. Our rigorous theoretical analysis shows that by properly setting the number of subproblems, the upper bounds of expected running time of MOEA/D-C obtaining a set of solutions to cover the Pareto fronts (PFs) of these problems are apparently lower than those of the one with mutation-only (MOEA/D-M). Moreover, to effectively obtain a set of solutions to cover the PFs of these problem, MOEA/D-C only needs to decompose these MOPs into several subproblems with a set of simple weight vectors while MOEA/D-M needs to find Ω(n) optimally decomposed weight vectors. This result suggests that the use of crossover in decomposition-based MOEA can simplify the setting of weight vectors for different problems and make the algorithm more efficient. This paper provides some insights into the working principles of MOEA/D and explains why some existing decomposition-based MOEAs work well in computational experiments.

Deep Reinforcement Learning for Multiobjective Optimization

This article proposes an end-to-end framework for solving multiobjective optimization problems (MOPs) using deep reinforcement learning (DRL), that we call DRL-based multiobjective optimization algorithm (DRL-MOA). The idea of decomposition is adopted to decompose the MOP into a set of scalar optimization subproblems. Then, each subproblem is modeled as a neural network. Model parameters of all the subproblems are optimized collaboratively according to a neighborhood-based parameter-transfer strategy and the DRL training algorithm. Pareto-optimal solutions can be directly obtained through the trained neural-network models. Specifically, the multiobjective traveling salesman problem (MOTSP) is solved in this article using the DRL-MOA method by modeling the subproblem as a Pointer Network. Extensive experiments have been conducted to study the DRL-MOA and various benchmark methods are compared with it. It is found that once the trained model is available, it can scale to newly encountered problems with no need for retraining the model. The solutions can be directly obtained by a simple forward calculation of the neural network; thereby, no iteration is required and the MOP can be always solved in a reasonable time. The proposed method provides a new way of solving the MOP by means of DRL. It has shown a set of new characteristics, for example, strong generalization ability and fast solving speed in comparison with the existing methods for multiobjective optimizations. The experimental results show the effectiveness and competitiveness of the proposed method in terms of model performance and running time.

Overlapping Community Detection in Directed and Undirected Attributed Networks Using a Multiobjective Evolutionary Algorithm

In many real-world networks, the structural connections of networks and the attributes about each node are always available. We typically call such graphs attributed networks, in which attributes always play the same important role in community detection as the topological structure. It is shown that the very existence of overlapping communities is one of the most important characteristics of various complex networks, while the majority of the existing community detection methods was designed for detecting separated communities in attributed networks. Therefore, it is quite challenging to detect meaningful overlapping structures with the combination of node attributes and topological structures. Therefore, in this article, we propose a multiobjective evolutionary algorithm based on the similarity attribute for overlapping community detection in attributed networks (MOEA-SA _OV ). In MOEA-SA _OV , a modified extended modularity EQ_OV , dealing with both directed and undirected networks, is well designed as the first objective. Another objective employed is the attribute similarity S_A . Then, a novel encoding and decoding strategy is designed to realize the goal of representing overlapping communities efficiently. MOEA-SA _OV runs under the framework of the nondominated sorting genetic algorithm II (NSGA-II) and can automatically determine the number of communities. In the experiments, the performance of MOEA-SA _OV is validated on both synthetic and real-world networks, and the experimental results demonstrate that our method can effectively find Pareto fronts about overlapping community structures with practical significance in both directed and undirected attributed networks.

A review on genetic algorithm: past, present, and future

In this paper, the analysis of recent advances in genetic algorithms is discussed. The genetic algorithms of great interest in research community are selected for analysis. This review will help the new and demanding researchers to provide the wider vision of genetic algorithms. The well-known algorithms and their implementation are presented with their pros and cons. The genetic operators and their usages are discussed with the aim of facilitating new researchers. The different research domains involved in genetic algorithms are covered. The future research directions in the area of genetic operators, fitness function and hybrid algorithms are discussed. This structured review will be helpful for research and graduate teaching.

Efficient Large-Scale Multiobjective Optimization Based on a Competitive Swarm Optimizer

There exist many multiobjective optimization problems (MOPs) containing a large number of decision variables in real-world applications, which are known as large-scale MOPs. Due to the ineffectiveness of existing operators in finding optimal solutions in a huge decision space, some decision variable division-based algorithms have been tailored for improving the search efficiency in solving large-scale MOPs. However, these algorithms will encounter difficulties when solving problems with complicated landscapes, as the decision variable division is likely to be inaccurate and time consuming. In this paper, we propose a competitive swarm optimizer (CSO)-based efficient search for solving large-scale MOPs. The proposed algorithm adopts a new particle updating strategy that suggests a two-stage strategy to update position, which can highly improve the search efficiency. The experimental results on large-scale benchmark MOPs and an application example demonstrate the superiority of the proposed algorithm over several state-of-the-art multiobjective evolutionary algorithms, including problem transformation-based algorithm, decision variable clustering-based algorithm, particle swarm optimization algorithm, and estimation of distribution algorithm.

A Cross-Reference Line Method Based Multiobjective Evolutionary Algorithm to Enhance Population Diversity

Multiobjective evolutionary algorithms (MOEAs) with higher population diversity have been extensively presented in literature studies and shown great potential in the approximate Pareto front (PF). Especially, in the recent development of MOEAs, the reference line method is increasingly favored due to its diversity enhancement nature and auxiliary selection mechanism based on the uniformly distributed reference line. However, the existing reference line method ignores the nadir point and consequently causes the Pareto incompatibility problem, which makes the algorithm convergence worse. To address this issue, a multiobjective evolutionary algorithm based on the adaptive cross-reference line method, called MOEA-CRL, is proposed under the framework of the indicator-based MOEAs. Based on the dominant penalty distance (DPD) indicator, the cross-reference line method can not only solve the Pareto incompatibility problem but also enhance the population diversity on the convex PF and improve the performances of MOEA-CRL for irregular PF. In addition, the MOEA-CRL adjusts the distribution of the cross-reference lines directly defined by the DPD indicator according to the contributing solutions. Therefore, the adaptation of cross-reference lines will not be affected by the population size and the uniform distribution of cross-reference lines can be maintained. The MOEA-CRL is examined and compared with other MOEAs on several benchmark problems. The experimental results show that the MOEA-CRL is superior to several advanced MOEAs, especially on the convex PF. The MOEA-CRL exhibits the flexibility in population size setting and the great versatility in various multiobjective optimization problems (MOPs) and many-objective optimization problems (MaOPs).

Effect of Objective Normalization and Penalty Parameter on Penalty Boundary Intersection Decomposition-Based Evolutionary Many-Objective Optimization Algorithms

An objective normalization strategy is essential in any evolutionary multiobjective or many-objective optimization (EMO or EMaO) algorithm, due to the distance calculations between objective vectors required to compute diversity and convergence of population members. For the decomposition-based EMO/EMaO algorithms involving the Penalty Boundary Intersection (PBI) metric, normalization is an important matter due to the computation of two distance metrics. In this article, we make a theoretical analysis of the effect of instabilities in the normalization process on the performance of PBI-based MOEA/D and a proposed PBI-based NSGA-III procedure. Although the effect is well recognized in the literature, few theoretical studies have been done so far to understand its true nature and the choice of a suitable penalty parameter value for an arbitrary problem. The developed theoretical results have been corroborated with extensive experimental results on three to 15-objective convex and non-convex instances of DTLZ and WFG problems. The article, makes important theoretical conclusions on PBI-based decomposition algorithms derived from the study.

What Weights Work for You? Adapting Weights for Any Pareto Front Shape in Decomposition-Based Evolutionary Multiobjective Optimisation

The quality of solution sets generated by decomposition-based evolutionary multi-objective optimisation (EMO) algorithms depends heavily on the consistency between a given problem's Pareto front shape and the specified weights' distribution. A set of weights distributed uniformly in a simplex often leads to a set of well-distributed solutions on a Pareto front with a simplex-like shape, but may fail on other Pareto front shapes. It is an open problem on how to specify a set of appropriate weights without the information of the problem's Pareto front beforehand. In this article, we propose an approach to adapt weights during the evolutionary process (called AdaW). AdaW progressively seeks a suitable distribution of weights for the given problem by elaborating several key parts in weight adaptation-weight generation, weight addition, weight deletion, and weight update frequency. Experimental results have shown the effectiveness of the proposed approach. AdaW works well for Pareto fronts with very different shapes: 1) the simplex-like, 2) the inverted simplex-like, 3) the highly nonlinear, 4) the disconnect, 5) the degenerate, 6) the scaled, and 7) the high-dimensional.

Evolutionary Many-Objective Algorithm Using Decomposition-Based Dominance Relationship

Decomposition-based evolutionary algorithms have shown great potential in many-objective optimization. However, the lack of theoretical studies on decomposition methods has hindered their further development and application. In this paper, we first theoretically prove that weight sum, Tchebycheff, and penalty boundary intersection decomposition methods are essentially interconnected. Inspired by this, we further show that highly customized dominance relationship can be derived from decomposition for any given decomposition vector. A new evolutionary algorithm is then proposed by applying the customized dominance relationship with adaptive strategy to each subpopulation of multiobjective to multiobjective framework. Experiments are conducted to compare the proposed algorithm with five state-of-the-art decomposition-based evolutionary algorithms on a set of well-known scaled many-objective test problems with 5 to 15 objectives. Simulation results have shown that the proposed algorithm can make better use of the decomposition vectors to achieve better performance. Further investigations on unscaled many-objective test problems verify the robust and generality of the proposed algorithm.

Service-oriented Application Composition with Evolutionary Heuristics and Multiple Criteria

The need to create and deploy business application systems rapidly has sparked interest in using web services to compose them. When creating mission-critical business applications through web service compositions, in addition to ensuring that functional requirements are met, designers need to consider the end-to-end reliability, security, performance, and overall cost of the application. As the number of available coarse-grain business services grows, the problem of selecting appropriate services quickly becomes combinatorially explosive for realistic-sized business applications. This article develops a business-process-driven approach for composing service-oriented applications. We use a combination of weights to explore the entire QoS criteria landscape through the use of a multi-criteria genetic algorithm (GA) to identify a Pareto-optimal multidimensional frontier that permits managers to trade off conflicting objectives when selecting a set of services. We illustrate the effectiveness of the approach by applying it to a real-world drop-ship business application and compare its performance to another GA-based approach for service composition.

A Collaborative Neurodynamic Approach to Multiobjective Optimization

There are two ultimate goals in multiobjective optimization. The primary goal is to obtain a set of Pareto-optimal solutions while the secondary goal is to obtain evenly distributed solutions to characterize the efficient frontier. In this paper, a collaborative neurodynamic approach to multiobjective optimization is presented to attain both goals of Pareto optimality and solution diversity. The multiple objectives are first scalarized using a weighted Chebyshev function. Multiple projection neural networks are employed to search for Pareto-optimal solutions with the help of a particle swarm optimization (PSO) algorithm in reintialization. To diversify the Pareto-optimal solutions, a holistic approach is proposed by maximizing the hypervolume (HV) using again a PSO algorithm. The experimental results show that the proposed approach outperforms three other state-of-the-art multiobjective algorithms (i.e., HMOEA/D, MOEA/DD, and NSGAIII) most of times on 37 benchmark datasets in terms of HV and inverted generational distance.

An Enhanced Decomposition-Based Evolutionary Algorithm With Adaptive Reference Vectors

Multiobjective optimization problems with more than three objectives are commonly referred to as many-objective optimization problems (MaOPs). Development of algorithms to solve MaOPs has garnered significant research attention in recent years. "Decomposition" is a commonly adopted approach toward this aim, wherein the problem is divided into a set of simpler subproblems guided by a set of reference vectors. The reference vectors are often predefined and distributed uniformly in the objective space. Use of such uniform distribution of reference vectors has shown commendable performance on problems with "regular" Pareto optimal front (POF), i.e., those that are nondegenerate, smooth, continuous, and easily mapped by a unit simplex of reference vectors. However, the performance deteriorates for problems with "irregular" POF (i.e., which deviate from above properties), since a number of reference vectors may not have a solution on the POF along them. While adaptive approaches have been suggested in the literature that attempt to delete/insert reference directions conforming to the geometry of the evolving front, their performance may in turn be compromised for problems with regular POFs. This paper presents a generalized version of previously proposed decomposition-based evolutionary algorithm with adaptive reference vectors, intended toward achieving competitive performance for both types of problems. The proposed approach starts off with a set of uniform reference vectors and collects information about feasibility and nondominance of solutions that associate with the reference vectors over a learning period. Subsequently, new reference directions are inserted/deleted, while the original directions may assume an active or inactive role during the course of evolution. Numerical experiments are conducted over a wide range of problems with regular and irregular POFs with up to 15 objectives to demonstrate the competence of the proposed approach with the state-of-the-art methods.

A Multiobjective Evolutionary Algorithm Based on Structural and Attribute Similarities for Community Detection in Attributed Networks

Most of the existing community detection algorithms are based on vertex connectivity. While in many real networks, each vertex usually has one or more attributes describing its properties which are often homogeneous in a cluster. Such networks can be modeled as attributed graphs, whose attributes sometimes are equally important to topological structure in graph clustering. One important challenge is to detect communities considering both topological structure and vertex properties simultaneously. To this propose, a multiobjective evolutionary algorithm based on structural and attribute similarities (MOEA-SA) is first proposed to solve the attributed graph clustering problems in this paper. In MOEA-SA, a new objective named as attribute similarity is proposed and another objective employed is the modularity . A hybrid representation is used and a neighborhood correction strategy is designed to repair the wrongly assigned genes through making balance between structural and attribute information. Moreover, an effective multi-individual-based mutation operator is designed to guide the evolution toward the good direction. The performance of MOEA-SA is validated on several real Facebook attributed graphs and several ego-networks with multiattribute. Two measurements, namely density and entropy , are used to evaluate the quality of communities obtained. Experimental results demonstrate the effectiveness of MOEA-SA and the systematic comparisons with existing methods show that MOEA-SA can get better values of and in each graph and find more relevant communities with practical meanings. Knee points corresponding to the best compromise solutions are calculated to guide decision makers to make convenient choices.

Test Problems for Large-Scale Multiobjective and Many-Objective Optimization

The interests in multiobjective and many-objective optimization have been rapidly increasing in the evolutionary computation community. However, most studies on multiobjective and many-objective optimization are limited to small-scale problems, despite the fact that many real-world multiobjective and many-objective optimization problems may involve a large number of decision variables. As has been evident in the history of evolutionary optimization, the development of evolutionary algorithms (EAs) for solving a particular type of optimization problems has undergone a co-evolution with the development of test problems. To promote the research on large-scale multiobjective and many-objective optimization, we propose a set of generic test problems based on design principles widely used in the literature of multiobjective and many-objective optimization. In order for the test problems to be able to reflect challenges in real-world applications, we consider mixed separability between decision variables and nonuniform correlation between decision variables and objective functions. To assess the proposed test problems, six representative evolutionary multiobjective and many-objective EAs are tested on the proposed test problems. Our empirical results indicate that although the compared algorithms exhibit slightly different capabilities in dealing with the challenges in the test problems, none of them are able to efficiently solve these optimization problems, calling for the need for developing new EAs dedicated to large-scale multiobjective and many-objective optimization.