A New Co-Evolution Binary Particle Swarm Optimization with Multiple Inertia Weight Strategy for Feature Selection

A feature selection method based on the Golden Jackal-Grey Wolf Hybrid Optimization Algorithm

This paper proposes a feature selection method based on a hybrid optimization algorithm that combines the Golden Jackal Optimization (GJO) and Grey Wolf Optimizer (GWO). The primary objective of this method is to create an effective data dimensionality reduction technique for eliminating redundant, irrelevant, and noisy features within high-dimensional datasets. Drawing inspiration from the Chinese idiom "Chai Lang Hu Bao," hybrid algorithm mechanisms, and cooperative behaviors observed in natural animal populations, we amalgamate the GWO algorithm, the Lagrange interpolation method, and the GJO algorithm to propose the multi-strategy fusion GJO-GWO algorithm. In Case 1, the GJO-GWO algorithm addressed eight complex benchmark functions. In Case 2, GJO-GWO was utilized to tackle ten feature selection problems. Experimental results consistently demonstrate that under identical experimental conditions, whether solving complex benchmark functions or addressing feature selection problems, GJO-GWO exhibits smaller means, lower standard deviations, higher classification accuracy, and reduced execution times. These findings affirm the superior optimization performance, classification accuracy, and stability of the GJO-GWO algorithm.

Feature Selection Problem and Metaheuristics: A Systematic Literature Review about Its Formulation, Evaluation and Applications

Feature selection is becoming a relevant problem within the field of machine learning. The feature selection problem focuses on the selection of the small, necessary, and sufficient subset of features that represent the general set of features, eliminating redundant and irrelevant information. Given the importance of the topic, in recent years there has been a boom in the study of the problem, generating a large number of related investigations. Given this, this work analyzes 161 articles published between 2019 and 2023 (20 April 2023), emphasizing the formulation of the problem and performance measures, and proposing classifications for the objective functions and evaluation metrics. Furthermore, an in-depth description and analysis of metaheuristics, benchmark datasets, and practical real-world applications are presented. Finally, in light of recent advances, this review paper provides future research opportunities.

Cauchy non-convex sparse feature selection method for the high-dimensional small-sample problem in motor imagery EEG decoding

Introduction

The time, frequency, and space information of electroencephalogram (EEG) signals is crucial for motor imagery decoding. However, these temporal-frequency-spatial features are high-dimensional small-sample data, which poses significant challenges for motor imagery decoding. Sparse regularization is an effective method for addressing this issue. However, the most commonly employed sparse regularization models in motor imagery decoding, such as the least absolute shrinkage and selection operator (LASSO), is a biased estimation method and leads to the loss of target feature information.

Methods

In this paper, we propose a non-convex sparse regularization model that employs the Cauchy function. By designing a proximal gradient algorithm, our proposed model achieves closer-to-unbiased estimation than existing sparse models. Therefore, it can learn more accurate, discriminative, and effective feature information. Additionally, the proposed method can perform feature selection and classification simultaneously, without requiring additional classifiers.

Results

We conducted experiments on two publicly available motor imagery EEG datasets. The proposed method achieved an average classification accuracy of 82.98% and 64.45% in subject-dependent and subject-independent decoding assessment methods, respectively.

Conclusion

The experimental results show that the proposed method can significantly improve the performance of motor imagery decoding, with better classification performance than existing feature selection and deep learning methods. Furthermore, the proposed model shows better generalization capability, with parameter consistency over different datasets and robust classification across different training sample sizes. Compared with existing sparse regularization methods, the proposed method converges faster, and with shorter model training time.

A multistart tabu search-based method for feature selection in medical applications

In the design of classification models, irrelevant or noisy features are often generated. In some cases, there may even be negative interactions among features. These weaknesses can degrade the performance of the models. Feature selection is a task that searches for a small subset of relevant features from the original set that generate the most efficient models possible. In addition to improving the efficiency of the models, feature selection confers other advantages, such as greater ease in the generation of the necessary data as well as clearer and more interpretable models. In the case of medical applications, feature selection may help to distinguish which characteristics, habits, and factors have the greatest impact on the onset of diseases. However, feature selection is a complex task due to the large number of possible solutions. In the last few years, methods based on different metaheuristic strategies, mainly evolutionary algorithms, have been proposed. The motivation of this work is to develop a method that outperforms previous methods, with the benefits that this implies especially in the medical field. More precisely, the present study proposes a simple method based on tabu search and multistart techniques. The proposed method was analyzed and compared to other methods by testing their performance on several medical databases. Specifically, eight databases belong to the well-known repository of the University of California in Irvine and one of our own design were used. In these computational tests, the proposed method outperformed other recent methods as gauged by various metrics and classifiers. The analyses were accompanied by statistical tests, the results of which showed that the superiority of our method is significant and therefore strengthened these conclusions. In short, the contribution of this work is the development of a method that, on the one hand, is based on different strategies than those used in recent methods, and on the other hand, improves the performance of these methods.

Application of Machine Learning to Predict COVID-19 Spread via an Optimized BPSO Model

During the pandemic of the coronavirus disease (COVID-19), statistics showed that the number of affected cases differed from one country to another and also from one city to another. Therefore, in this paper, we provide an enhanced model for predicting COVID-19 samples in different regions of Saudi Arabia (high-altitude and sea-level areas). The model is developed using several stages and was successfully trained and tested using two datasets that were collected from Taif city (high-altitude area) and Jeddah city (sea-level area) in Saudi Arabia. Binary particle swarm optimization (BPSO) is used in this study for making feature selections using three different machine learning models, i.e., the random forest model, gradient boosting model, and naive Bayes model. A number of predicting evaluation metrics including accuracy, training score, testing score, F-measure, recall, precision, and receiver operating characteristic (ROC) curve were calculated to verify the performance of the three machine learning models on these datasets. The experimental results demonstrated that the gradient boosting model gives better results than the random forest and naive Bayes models with an accuracy of 94.6% using the Taif city dataset. For the dataset of Jeddah city, the results demonstrated that the random forest model outperforms the gradient boosting and naive Bayes models with an accuracy of 95.5%. The dataset of Jeddah city achieved better results than the dataset of Taif city in Saudi Arabia using the enhanced model for the term of accuracy.

Binary Sand Cat Swarm Optimization Algorithm for Wrapper Feature Selection on Biological Data

In large datasets, irrelevant, redundant, and noisy attributes are often present. These attributes can have a negative impact on the classification model accuracy. Therefore, feature selection is an effective pre-processing step intended to enhance the classification performance by choosing a small number of relevant or significant features. It is important to note that due to the NP-hard characteristics of feature selection, the search agent can become trapped in the local optima, which is extremely costly in terms of time and complexity. To solve these problems, an efficient and effective global search method is needed. Sand cat swarm optimization (SCSO) is a newly introduced metaheuristic algorithm that solves global optimization algorithms. Nevertheless, the SCSO algorithm is recommended for continuous problems. bSCSO is a binary version of the SCSO algorithm proposed here for the analysis and solution of discrete problems such as wrapper feature selection in biological data. It was evaluated on ten well-known biological datasets to determine the effectiveness of the bSCSO algorithm. Moreover, the proposed algorithm was compared to four recent binary optimization algorithms to determine which algorithm had better efficiency. A number of findings demonstrated the superiority of the proposed approach both in terms of high prediction accuracy and small feature sizes.

Semisupervised Bacterial Heuristic Feature Selection Algorithm for High-Dimensional Classification with Missing Labels

Feature selection is a crucial method for discovering relevant features in high-dimensional data. However, most studies primarily focus on completely labeled data, ignoring the frequent occurrence of missing labels in real-world problems. To address high-dimensional and label-missing problems in data classification simultaneously, we proposed a semisupervised bacterial heuristic feature selection algorithm. To track the label-missing problem, a k-nearest neighbor semisupervised learning strategy is designed to reconstruct missing labels. In addition, the bacterial heuristic algorithm is improved using hierarchical population initialization, dynamic learning, and elite population evolution strategies to enhance the search capacity for various feature combinations. To verify the effectiveness of the proposed algorithm, three groups of comparison experiments based on eight datasets are employed, including two traditional feature selection methods, four bacterial heuristic feature selection algorithms, and two swarm-based heuristic feature selection algorithms. Experimental results demonstrate that the proposed algorithm has obvious advantages in terms of classification accuracy and selected feature numbers.

A Novel Methodology for Classifying EMG Movements Based on SVM and Genetic Algorithms

Electromyography (EMG) processing is a fundamental part of medical research. It offers the possibility of developing new devices and techniques for the diagnosis, treatment, care, and rehabilitation of patients, in most cases non-invasively. However, EMG signals are random, non-stationary, and non-linear, making their classification difficult. Due to this, it is of vital importance to define which factors are helpful for the classification process. In order to improve this process, it is possible to apply algorithms capable of identifying which features are most important in the categorization process. Algorithms based on metaheuristic methods have demonstrated an ability to search for suitable subsets of features for optimization problems. Therefore, this work proposes a methodology based on genetic algorithms for feature selection to find the parameter space that offers the slightest classification error in 250 ms signal segments. For classification, a support vector machine is used. For this work, two databases were used, the first corresponding to the right upper extremity and the second formed by movements of the right lower extremity. For both databases, a feature space reduction of over 65% was obtained, with a higher average classification efficiency of 91% for the best subset of parameters. In addition, particle swarm optimization (PSO) was applied based on right upper extremity data, obtaining an 88% average error and a 46% reduction for the best subset of parameters. Finally, a sensitivity analysis was applied to the characteristics selected by PSO and genetic algorithms for the database of the right upper extremity, obtaining that the parameters determined by the genetic algorithms show greater sensitivity for the classification process.

A review of swarm-based metaheuristic optimization techniques and their application to doubly fed induction generator

In this paper, a review of Metaheuristic Optimization Techniques (MOT) which are currently in use for optimization in a vast range of problems, is presented. MOT are known for their simplicity and stochastic nature and successfully applied to solve complex engineering problems. Although there exist various categories of MOT, the techniques from swarm intelligence is reviewed in this paper. An explanation of the theoretical foundation upon which each algorithm is based is provided, along with the relevant mathematical models that explain how an algorithm attempts to obtain the best solution to a problem. The paper also reviews the applications of swarm-based MOT to the control of the doubly fed induction generator (DFIG). Particular attention is given to control of the DFIG for wind energy applications. Control of the DFIG is generally realized via the use of PI controllers. While various PI controller tuning methods are well established (such as the Ziegler–Nichols and Cohen–Coon methods), these methods produce satisfactory results, and often fail to meet the stringent levels of control presently required. Due to this fact, as well as the current success of MOT in engineering, the application of MOT to the control of the DFIG could be promising area of research. The results of the study show that although the various swarm-based MOT differ from each other in terms of aspects such as complexity and advantages, they are all based on the concept of randomness, and always attempt to produce the best possible solution. It was also observed that various swarm-based MOT displays the demerit of getting easily trapped in the local optimum, however various advancements have been proposed to correct such an issue. Based on the results of the application of these techniques to other engineering problems, their application to the DFIG could yield exceptional results.

Bibliometric Survey on Particle Swarm Optimization Algorithms (2001–2021)

Particle swarm optimization algorithms (PSOA) is a metaheuristic algorithm used to optimize computational problems using candidate solutions or particles based on selected quality measures. Despite the extensive research published, studies that critically examine its recent scientific developments and research impact are lacking. Therefore, the publication trends and research landscape on PSOA research were examined. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and bibliometric analysis techniques were applied to identify and analyze the published documents indexed in Scopus from 2001 to 2021. The published documents on PSOA increased from 8 to 1,717 (21,362.50%) due to the growing applications of PSOA in solving computational problems. “Conference papers” is the most common document type, whereas the most prolific researcher on PSOA is Andries P. Engelbrecht (South Africa). The most active affiliation (Ministry of Education) and funding organization (National Natural Science Foundation) are based in China. The research landscape on PSOA revealed high levels of publications, citations, and collaborations among the top authors, institutions, and countries worldwide. Keywords co-occurrence analysis revealed that “particle swarm optimization (PSO)” occurred more frequently than others. The findings of the study could provide researchers and policymakers with insights into the prospects and challenges of PSOA research relative to similar algorithms in the literature.

Thermal-Economic Optimization of Plate–Fin Heat Exchanger Using Improved Gaussian Quantum-Behaved Particle Swarm Algorithm

Heat exchangers are usually designed using a sophisticated process of trial-and-error to find proper values of unknown parameters which satisfy given requirements. Recently, the design of heat exchangers using evolutionary optimization algorithms has received attention. The major aim of the present study is to propose an improved Gaussian quantum-behaved particle swarm optimization (GQPSO) algorithm for enhanced optimization performance and its verification through application to a multivariable thermal-economic optimization problem of a crossflow plate–fin heat exchanger (PFHE). Three single objective functions: the number of entropy generation units (NEGUs), total annual cost (TAC), and heat exchanger surface area (A), were minimized separately by evaluating optimal values of seven unknown variables using four different PSO-based methods. By comparing the obtained best fitness values, the improved GQPSO approach could search quickly for better global optimal solutions by preventing particles from falling to the local minimum due to its modified local attractor scheme based on the Gaussian distributed random numbers. For example, the proposed GQPSO could predict further improved best fitness values of 40% for NEGUs, 17% for TAC, and 4.5% for A, respectively. Consequently, the present study suggests that the improved GQPSO approach with the modified local attractor scheme can be efficient in rapidly finding more suitable solutions for optimizing the thermal-economic problem of the crossflow PFHE.

A2BCF: An Automated ABC-Based Feature Selection Algorithm for Classification Models in an Education Application

Feature selection is an essential step of preprocessing in Machine Learning (ML) algorithms that can significantly impact the performance of ML models. It is considered one of the most crucial phases of automated ML (AutoML). Feature selection aims to find the optimal subset of features and remove the noninformative features from the dataset. Feature selection also reduces the computational time and makes the data more understandable to the learning model. There are various heuristic search strategies to address combinatorial optimization challenges. This paper develops an Automated Artificial Bee Colony-based algorithm for Feature Selection (A2BCF) to solve a classification problem. The application domain evaluating our proposed algorithm is education science, which solves a binary classification problem, namely, undergraduate student success. The modifications made to the original Artificial Bee Colony algorithm make the algorithm a well-performed approach.

An Efficient Parallel Hybrid Feature Selection Approach for Big Data Analysis

Classification algorithms face runtime complexity due to high data dimension, especially in the context of big data. Feature selection (FS) is a technique for reducing dimensions and improving learning performance. In this paper, the authors proposed a hybrid FS algorithm for classification in the context of big data. Firstly, only the most relevant features are selected using symmetric uncertainty (SU) as a measure of correlation. The features are distributed into subsets using Apache Spark to calculate SU between each feature and target class in parallel. Then a Binary PSO (BPSO) algorithm is used to find the optimal FS. The BPSO has limited convergence and restricted inertial weight adjustment, so the authors suggested using a multiple inertia weight strategy to influence the changes in particle motions so that the search process is more varied. Also, the authors proposed a parallel fitness evaluation for particles under Spark to accelerate the algorithm. The results showed that the proposed FS achieved higher classification performance with a smaller size in reasonable time.

Multi-objective grasshopper optimization algorithm based on multi-group and co-evolution

The balance between exploration and exploitation is critical to the performance of a Meta-heuristic optimization method. At different stages, a proper tradeoff between exploration and exploitation can drive the search process towards better performance. This paper develops a multi-objective grasshopper optimization algorithm (MOGOA) with a new proposed framework called the Multi-group and Co-evolution Framework which can archive a fine balance between exploration and exploitation. For the purpose, a grouping mechanism and a co-evolution mechanism are designed and integrated into the framework for ameliorating the convergence and the diversity of multi-objective optimization solutions and keeping the exploration and exploitation of swarm intelligence algorithm in balance. The grouping mechanism is employed to improve the diversity of search agents for increasing coverage of search space. The co-evolution mechanism is used to improve the convergence to the true Pareto optimal front by the interaction of search agents. Quantitative and qualitative outcomes prove that the framework prominently ameliorate the convergence accuracy and convergence speed of MOGOA. The performance of the presented algorithm has been benchmarked by several standard test functions, such as CEC2009, ZDT and DTLZ. The diversity and convergence of the obtained multi-objective optimization solutions are quantitatively and qualitatively compared with the original MOGOA by using two performance indicators (GD and IGD). The results on test suits show that the diversity and convergence of the obtained solutions are significantly improved. On several test functions, some statistical indicators are more than doubled. The validity of the results has been verified by the Wilcoxon rank-sum test.

MH-COVIDNet: Diagnosis of COVID-19 using deep neural networks and meta-heuristic-based feature selection on X-ray images

COVID-19 is a disease that causes symptoms in the lungs and causes deaths around the world. Studies are ongoing for the diagnosis and treatment of this disease, which is defined as a pandemic. Early diagnosis of this disease is important for human life. This process is progressing rapidly with diagnostic studies based on deep learning. Therefore, to contribute to this field, a deep learning-based approach that can be used for early diagnosis of the disease is proposed in our study. In this approach, a data set consisting of 3 classes of COVID19, normal and pneumonia lung X-ray images was created, with each class containing 364 images. Pre-processing was performed using the image contrast enhancement algorithm on the prepared data set and a new data set was obtained. Feature extraction was completed from this data set with deep learning models such as AlexNet, VGG19, GoogleNet, and ResNet. For the selection of the best potential features, two metaheuristic algorithms of binary particle swarm optimization and binary gray wolf optimization were used. After combining the features obtained in the feature selection of the enhancement data set, they were classified using SVM. The overall accuracy of the proposed approach was obtained as 99.38%. The results obtained by verification with two different metaheuristic algorithms proved that the approach we propose can help experts during COVID-19 diagnostic studies.

The CSP-Based New Features Plus Non-Convex Log Sparse Feature Selection for Motor Imagery EEG Classification

The common spatial pattern (CSP) is a very effective feature extraction method in motor imagery based brain computer interface (BCI), but its performance depends on the selection of the optimal frequency band. Although a lot of research works have been proposed to improve CSP, most of these works have the problems of large computation costs and long feature extraction time. To this end, three new feature extraction methods based on CSP and a new feature selection method based on non-convex log regularization are proposed in this paper. Firstly, EEG signals are spatially filtered by CSP, and then three new feature extraction methods are proposed. We called them CSP-wavelet, CSP-WPD and CSP-FB, respectively. For CSP-Wavelet and CSP-WPD, the discrete wavelet transform (DWT) or wavelet packet decomposition (WPD) is used to decompose the spatially filtered signals, and then the energy and standard deviation of the wavelet coefficients are extracted as features. For CSP-FB, the spatially filtered signals are filtered into multiple bands by a filter bank (FB), and then the logarithm of variances of each band are extracted as features. Secondly, a sparse optimization method regularized with a non-convex log function is proposed for the feature selection, which we called LOG, and an optimization algorithm for LOG is given. Finally, ensemble learning is used for secondary feature selection and classification model construction. Combing feature extraction and feature selection methods, a total of three new EEG decoding methods are obtained, namely CSP-Wavelet+LOG, CSP-WPD+LOG, and CSP-FB+LOG. Four public motor imagery datasets are used to verify the performance of the proposed methods. Compared to existing methods, the proposed methods achieved the highest average classification accuracy of 88.86, 83.40, 81.53, and 80.83 in datasets 1–4, respectively. The feature extraction time of CSP-FB is the shortest. The experimental results show that the proposed methods can effectively improve the classification accuracy and reduce the feature extraction time. With comprehensive consideration of classification accuracy and feature extraction time, CSP-FB+LOG has the best performance and can be used for the real-time BCI system.

Adapted Binary Particle Swarm Optimization for Efficient Features Selection in the Case of Imbalanced Sensor Data

Daily living activities (DLAs) classification using data collected from wearable monitoring sensors is very challenging due to the imbalance characteristics of the monitored data. A major research challenge is to determine the best combination of features that returns the best accuracy results using minimal computational resources, when the data is heterogeneous and not fitted for classical algorithms that are designed for balanced low-dimensional datasets. This research article: (1) presents a modification of the classical version of the binary particle swarm optimization (BPSO) algorithm that introduces a particular type of particles called sensor particles, (2) describes the adaptation of this algorithm for data generated by sensors that monitor DLAs to determine the best positions and features of the monitoring sensors that lead to the best classification results, and (3) evaluates and validates the proposed approach using a machine learning methodology that integrates the modified version of the algorithm. The methodology is tested and validated on the Daily Life Activities (DaLiAc) dataset.

An Adapting Chemotaxis Bacterial Foraging Optimization Algorithm for Feature Selection in Classification

Efficient classification methods can improve the data quality or relevance to better optimize some Internet applications such as fast searching engine and accurate identification. However, in the big data era, difficulties and volumes of data processing increase drastically. To decrease the huge computational cost, heuristic algorithms have been used. In this paper, an Adapting Chemotaxis Bacterial Foraging Optimization (ACBFO) algorithm is proposed based on basic Bacterial Foraging Optimization (BFO) algorithm. The aim of this work is to design a modified algorithm which is more suitable for data classification. The proposed algorithm has two updating strategies and one structural changing. First, the adapting chemotaxis step updating strategy is responsible to increase the flexibility of searching. Second, the feature subsets updating strategy better combines the proposed heuristic algorithm with the KNN classifier. Third, the nesting structure of BFO has been simplified to reduce the computation complexity. The ACBFO has been compared with BFO, BFOLIW and BPSO by testing on 12 widely used benchmark datasets. The result shows that ACBFO has a good ability of solving classification problems and gets higher accuracy than the other comparation algorithm.

A New Quadratic Binary Harris Hawk Optimization for Feature Selection

Harris hawk optimization (HHO) is one of the recently proposed metaheuristic algorithms that has proven to be work more effectively in several challenging optimization tasks. However, the original HHO is developed to solve the continuous optimization problems, but not to the problems with binary variables. This paper proposes the binary version of HHO (BHHO) to solve the feature selection problem in classification tasks. The proposed BHHO is equipped with an S-shaped or V-shaped transfer function to convert the continuous variable into a binary one. Moreover, another variant of HHO, namely quadratic binary Harris hawk optimization (QBHHO), is proposed to enhance the performance of BHHO. In this study, twenty-two datasets collected from the UCI machine learning repository are used to validate the performance of proposed algorithms. A comparative study is conducted to compare the effectiveness of QBHHO with other feature selection algorithms such as binary differential evolution (BDE), genetic algorithm (GA), binary multi-verse optimizer (BMVO), binary flower pollination algorithm (BFPA), and binary salp swarm algorithm (BSSA). The experimental results show the superiority of the proposed QBHHO in terms of classification performance, feature size, and fitness values compared to other algorithms.

Hybrid Binary Particle Swarm Optimization Differential Evolution-Based Feature Selection for EMG Signals Classification

To date, the usage of electromyography (EMG) signals in myoelectric prosthetics allows patients to recover functional rehabilitation of their upper limbs. However, the increment in the number of EMG features has been shown to have a great impact on performance degradation. Therefore, feature selection is an essential step to enhance classification performance and reduce the complexity of the classifier. In this paper, a hybrid method, namely, binary particle swarm optimization differential evolution (BPSODE) was proposed to tackle feature selection problems in EMG signals classification. The performance of BPSODE was validated using the EMG signals of 10 healthy subjects acquired from a publicly accessible EMG database. First, discrete wavelet transform was applied to decompose the signals into wavelet coefficients. The features were then extracted from each coefficient and formed into the feature vector. Afterward, BPSODE was used to evaluate the most informative feature subset. To examine the effectiveness of the proposed method, four state-of-the-art feature selection methods were used for comparison. The parameters, including accuracy, feature selection ratio, precision, F-measure, and computation time were used for performance measurement. Our results showed that BPSODE was superior, in not only offering a high classification performance, but also in having the smallest feature size. From the empirical results, it can be inferred that BPSODE-based feature selection is useful for EMG signals classification.

Binary Competitive Swarm Optimizer Approaches for Feature Selection

Feature selection is known as an NP-hard combinatorial problem in which the possible feature subsets increase exponentially with the number of features. Due to the increment of the feature size, the exhaustive search has become impractical. In addition, a feature set normally includes irrelevant, redundant, and relevant information. Therefore, in this paper, binary variants of a competitive swarm optimizer are proposed for wrapper feature selection. The proposed approaches are used to select a subset of significant features for classification purposes. The binary version introduced here is performed by employing the S-shaped and V-shaped transfer functions, which allows the search agents to move on the binary search space. The proposed approaches are tested by using 15 benchmark datasets collected from the UCI machine learning repository, and the results are compared with other conventional feature selection methods. Our results prove the capability of the proposed binary version of the competitive swarm optimizer not only in terms of high classification performance, but also low computational cost.