Unsupervised feature selection by self-paced learning regularization

An Improved Binary Walrus Optimizer with Golden Sine Disturbance and Population Regeneration Mechanism to Solve Feature Selection Problems

Feature selection (FS) is a significant dimensionality reduction technique in machine learning and data mining that is adept at managing high-dimensional data efficiently and enhancing model performance. Metaheuristic algorithms have become one of the most promising solutions in FS owing to their powerful search capabilities as well as their performance. In this paper, the novel improved binary walrus optimizer (WO) algorithm utilizing the golden sine strategy, elite opposition-based learning (EOBL), and population regeneration mechanism (BGEPWO) is proposed for FS. First, the population is initialized using an iterative chaotic map with infinite collapses (ICMIC) chaotic map to improve the diversity. Second, a safe signal is obtained by introducing an adaptive operator to enhance the stability of the WO and optimize the trade-off between exploration and exploitation of the algorithm. Third, BGEPWO innovatively designs a population regeneration mechanism to continuously eliminate hopeless individuals and generate new promising ones, which keeps the population moving toward the optimal solution and accelerates the convergence process. Fourth, EOBL is used to guide the escape behavior of the walrus to expand the search range. Finally, the golden sine strategy is utilized for perturbing the population in the late iteration to improve the algorithm's capacity to evade local optima. The BGEPWO algorithm underwent evaluation on 21 datasets of different sizes and was compared with the BWO algorithm and 10 other representative optimization algorithms. The experimental results demonstrate that BGEPWO outperforms these competing algorithms in terms of fitness value, number of selected features, and F1-score in most datasets. The proposed algorithm achieves higher accuracy, better feature reduction ability, and stronger convergence by increasing population diversity, continuously balancing exploration and exploitation processes and effectively escaping local optimal traps.

Multitask-Guided Deep Clustering With Boundary Adaptation

Multitask learning uses external knowledge to improve internal clustering and single-task learning. Existing multitask learning algorithms mostly use shallow-level correlation to aid judgment, and the boundary factors on high-dimensional datasets often lead algorithms to poor performance. The initial parameters of these algorithms cause the border samples to fall into a local optimal solution. In this study, a multitask-guided deep clustering (DC) with boundary adaptation (MTDC-BA) based on a convolutional neural network autoencoder (CNN-AE) is proposed. In the first stage, dubbed multitask pretraining (M-train), we construct an autoencoder (AE) named CNN-AE using the DenseNet-like structure, which performs deep feature extraction and stores captured multitask knowledge into model parameters. In the second phase, the parameters of the M-train are shared for CNN-AE, and clustering results are obtained by deep features, which is termed as single-task fitting (S-fit). To eliminate the boundary effect, we use data augmentation and improved self-paced learning to construct the boundary adaptation. We integrate boundary adaptors into the M-train and S-fit stages appropriately. The interpretability of MTDC-BA is accomplished by data transformation. The model relies on the principle that features become important as the reconfiguration loss decreases. Experiments on a series of typical datasets confirm the performance of the proposed MTDC-BA. Compared with other traditional clustering methods, including single-task DC algorithms and the latest multitask clustering algorithms, our MTDC-BA achieves better clustering performance with higher computational efficiency. Deep features clustering results demonstrate the stability of MTDC-BA by visualization and convergence verification. Through the visualization experiment, we explain and analyze the whole model data input and the middle characteristic layer. Further understanding of the principle of MTDC-BA. Through additional experiments, we know that the proposed MTDC-BA is efficient in the use of multitask knowledge. Finally, we carry out sensitivity experiments on the hyper-parameters to verify their optimal performance.

Hypergraph convolutional network for longitudinal data analysis in Alzheimer's disease

Alzheimer's disease (AD) is an irreversible and progressive neurodegenerative disease. Longitudinal structural magnetic resonance imaging (sMRI) data have been widely used for tracking AD pathogenesis and diagnosis. However, existing methods tend to treat each time point equally without considering the temporal characteristics of longitudinal data. In this paper, we propose a weighted hypergraph convolution network (WHGCN) to use the internal correlations among different time points and leverage high-order relationships between subjects for AD detection. Specifically, we construct hypergraphs for sMRI data at each time point using the K-nearest neighbor (KNN) method to represent relationships between subjects, and then fuse the hypergraphs according to the importance of the data at each time point to obtain the final hypergraph. Subsequently, we use hypergraph convolution to learn high-order information between subjects while performing feature dimensionality reduction. Finally, we conduct experiments on 518 subjects selected from the Alzheimer's disease neuroimaging initiative (ADNI) database, and the results show that the WHGCN can get higher AD detection performance and has the potential to improve our understanding of the pathogenesis of AD.

A Survey on Curriculum Learning

Curriculum learning (CL) is a training strategy that trains a machine learning model from easier data to harder data, which imitates the meaningful learning order in human curricula. As an easy-to-use plug-in, the CL strategy has demonstrated its power in improving the generalization capacity and convergence rate of various models in a wide range of scenarios such as computer vision and natural language processing etc. In this survey article, we comprehensively review CL from various aspects including motivations, definitions, theories, and applications. We discuss works on curriculum learning within a general CL framework, elaborating on how to design a manually predefined curriculum or an automatic curriculum. In particular, we summarize existing CL designs based on the general framework of Difficulty Measurer + Training Scheduler and further categorize the methodologies for automatic CL into four groups, i.e., Self-paced Learning, Transfer Teacher, RL Teacher, and Other Automatic CL. We also analyze principles to select different CL designs that may benefit practical applications. Finally, we present our insights on the relationships connecting CL and other machine learning concepts including transfer learning, meta-learning, continual learning and active learning, etc., then point out challenges in CL as well as potential future research directions deserving further investigations.

Fuzzy Sparse Deviation Regularized Robust Principal Component Analysis

Robust principal component analysis (RPCA) is a technique that aims to make principal component analysis (PCA) robust to noise samples. The current modeling approaches of RPCA were proposed by analyzing the prior distribution of the reconstruction error terms. However, these methods ignore the influence of samples with large reconstruction errors, as well as the valid information of these samples in principal component space, which will degrade the ability of PCA to extract the principal component of data. In order to solve this problem, Fuzzy sparse deviation regularized robust principal component Analysis (FSD-PCA) is proposed in this paper. First, FSD-PCA learns the principal components by minimizing the square of l₂ -norm-based reconstruction error. Then, FSD-PCA introduces sparse deviation on reconstruction error term to relax the samples with large bias, thus FSD-PCA can process noise and principal components of samples separately as well as improve the ability of FSD-PCA for retaining the principal component information. Finally, FSD-PCA estimates the prior probability of each sample by fuzzy weighting based on the relaxed reconstruction error, which can improve the robustness of the model. The experimental results indicate that the proposed model performs excellent robustness against different types of noise than the state-of-art algorithms, and the sparse deviation term enables FSD-PCA to process noise information and principal component information separately, so FSD-PCA can filter the noise information of an image and restore the corrupted image.

A reawakening of Machine Learning Application in Unmanned Aerial Vehicle: Future Research Motivation

Machine learning (ML) entails artificial procedures that improve robotically through experience and using data. Supervised, unsupervised, semi-supervised, and Reinforcement Learning (RL) are the main types of ML. This study mainly focuses on RL and Deep learning, since necessitates mainly sequential and consecutive decision-making context. This is a comparison to supervised and non-supervised learning due to the interactive nature of the environment. Exploiting a forthcoming accumulative compensation and its stimulus of machines, complex policy decisions. The study further analyses and presents ML perspectives depicting state-of-the-art developments with advancement, relatively depicting the future trend of RL based on its applicability in technology. It's a challenge to an Internet of Things (IoT) and demonstrates what possibly can be adopted as a solution. This study presented a summarized perspective on identified arenas on the analysis of RL. The study scrutinized that a reasonable number of the techniques engrossed in alternating policy values instead of modifying other gears in an exact state of intellectual. The study presented a strong foundation for the current studies to be adopted by the researchers from different research backgrounds to develop models, and architectures that are relevant.

Single Cell Self-Paced Clustering with Transcriptome Sequencing Data

Single cell RNA sequencing (scRNA-seq) allows researchers to explore tissue heterogeneity, distinguish unusual cell identities, and find novel cellular subtypes by providing transcriptome profiling for individual cells. Clustering analysis is usually used to predict cell class assignments and infer cell identities. However, the performance of existing single-cell clustering methods is extremely sensitive to the presence of noise data and outliers. Existing clustering algorithms can easily fall into local optimal solutions. There is still no consensus on the best performing method. To address this issue, we introduce a single cell self-paced clustering (scSPaC) method with F-norm based nonnegative matrix factorization (NMF) for scRNA-seq data and a sparse single cell self-paced clustering (sscSPaC) method with l21-norm based nonnegative matrix factorization for scRNA-seq data. We gradually add single cells from simple to complex to our model until all cells are selected. In this way, the influences of noisy data and outliers can be significantly reduced. The proposed method achieved the best performance on both simulation data and real scRNA-seq data. A case study about human clara cells and ependymal cells scRNA-seq data clustering shows that scSPaC is more advantageous near the clustering dividing line.

A feature selection strategy for improving software maintainability prediction

Software maintainability is a significant contributor while choosing particular software. It is helpful in estimation of the efforts required after delivering the software to the customer. However, issues like imbalanced distribution of datasets, and redundant and irrelevant occurrence of various features degrade the performance of maintainability prediction models. Therefore, current study applies ImpS algorithm to handle imbalanced data and extensively investigates several Feature Selection (FS) techniques including Symmetrical Uncertainty (SU), RandomForest filter, and Correlation-based FS using one open-source, three proprietaries and two commercial datasets. Eight different machine learning algorithms are utilized for developing prediction models. The performance of models is evaluated using Accuracy, G-Mean, Balance, & Area under the ROC Curve. Two statistical tests, Friedman Test and Wilcoxon Signed Ranks Test are conducted for assessing different FS techniques. The results substantiate that FS techniques significantly improve the performance of various prediction models with an overall improvement of 18.58%, 129.73%, 80.00%, and 45.76% in the median values of Accuracy, G-Mean, Balance, & AUC, respectively for all the datasets taken together. Friedman test advocates the supremacy of SU FS technique. Wilcoxon Signed Ranks test showcases that SU FS technique is significantly superior to the CFS technique for three out of six datasets.

Determining and forecasting drought susceptibility in southwestern Iran using multi-criteria decision-making (MCDM) coupled with CA-Markov model

Forecasting drought and determining relevant data to predict drought are an important topic for decision-makers and planners. It is critical to predicting drought in the south of Fars province, an important agricultural center in Iran located in arid and semi-arid climates. The purpose of this study was to generate a drought map in 2019 using 12 parameters: altitude, aridity index, erosion, groundwater depth, land use, PET (Potential evapotranspiration), precipitation days, precipitation, slope, soil texture, soil salinity, and distance to river, and predict drought maps in 2030 and 2040 using the cellular automata (CA)-Markov model spatially. The fuzzy method was first used to homogenize the data. Next, by evaluating each parameter, the weight of each parameter was calculated using the analytic hierarchy process (AHP), and a map of drought-prone areas was generated. The results of the fuzzy-AHP method showed that the eastern and southeastern regions of the study area were prone to drought. The four most predictive parameters in causing drought, i.e., aridity index, PET, precipitation, and soil texture, were selected using the Best search method and were then chosen as the input to determine drought mapping using the fuzzy and AHP methods. Finally, the CA-Markov model was used to predict future drought maps, and the results showed that in 2030 and 2040 the drought situation in the east and south of the study area would intensify.

Diagnosis of early Alzheimer's disease based on dynamic high order networks

Machine learning methods have been widely used for early diagnosis of Alzheimer's disease (AD) via functional connectivity networks (FCNs) analysis from neuroimaging data. The conventional low-order FCNs are obtained by time-series correlation of the whole brain based on resting-state functional magnetic resonance imaging (R-fMRI). However, FCNs overlook inter-region interactions, which limits application to brain disease diagnosis. To overcome this drawback, we develop a novel framework to exploit the high-level dynamic interactions among brain regions for early AD diagnosis. Specifically, a sliding window approach is employed to generate some R-fMRI sub-series. The correlations among these sub-series are then used to construct a series of dynamic FCNs. High-order FCNs based on the topographical similarity between each pair of the dynamic FCNs are then constructed. Afterward, a local weight clustering method is used to extract effective features of the network, and the least absolute shrinkage and selection operation method is chosen for feature selection. A support vector machine is employed for classification, and the dynamic high-order network approach is evaluated on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. Our experimental results demonstrate that the proposed approach not only achieves promising results for AD classification, but also successfully recognizes disease-related biomarkers.

Using a Text Mining Approach to Hear Voices of Customers from Social Media toward the Fast-Food Restaurant Industry

Due to the COVID-19 pandemic, the sales of fast-food businesses have dropped sharply. Customer satisfaction has always been one of the key factors for the sustainable development of enterprises. However, in the fast-food restaurant business, gaining the knowledge of customer satisfaction is one of the critical tasks. Moreover, text reviews in social media have become one of important reference sources for customers’ decisions in buying services and products. Therefore, the main purpose of this study is to explore whether customer voices from social media reviews are different during the COVID-19 outbreak and to propose a new method to reduce interpersonal contact when collecting data. A text mining scheme which includes least absolute shrinkage and selection operator (LASSO) and decision trees (DT) are presented to discover the essential factors for customers to increase their satisfaction from unstructured online customer reviews. Finally, three real world review sets were employed to validate the effectiveness of the presented text mining scheme. Experimental results can help companies to properly adapt to similar epidemic situations in the future and facilitate their sustainable development.

Application of Machine Learning Algorithms in Plant Breeding: Predicting Yield From Hyperspectral Reflectance in Soybean

Recent substantial advances in high-throughput field phenotyping have provided plant breeders with affordable and efficient tools for evaluating a large number of genotypes for important agronomic traits at early growth stages. Nevertheless, the implementation of large datasets generated by high-throughput phenotyping tools such as hyperspectral reflectance in cultivar development programs is still challenging due to the essential need for intensive knowledge in computational and statistical analyses. In this study, the robustness of three common machine learning (ML) algorithms, multilayer perceptron (MLP), support vector machine (SVM), and random forest (RF), were evaluated for predicting soybean (Glycine max) seed yield using hyperspectral reflectance. For this aim, the hyperspectral reflectance data for the whole spectra ranged from 395 to 1005 nm, which were collected at the R4 and R5 growth stages on 250 soybean genotypes grown in four environments. The recursive feature elimination (RFE) approach was performed to reduce the dimensionality of the hyperspectral reflectance data and select variables with the largest importance values. The results indicated that R5 is more informative stage for measuring hyperspectral reflectance to predict seed yields. The 395 nm reflectance band was also identified as the high ranked band in predicting the soybean seed yield. By considering either full or selected variables as the input variables, the ML algorithms were evaluated individually and combined-version using the ensemble-stacking (E-S) method to predict the soybean yield. The RF algorithm had the highest performance with a value of 84% yield classification accuracy among all the individual tested algorithms. Therefore, by selecting RF as the metaClassifier for E-S method, the prediction accuracy increased to 0.93, using all variables, and 0.87, using selected variables showing the success of using E-S as one of the ensemble techniques. This study demonstrated that soybean breeders could implement E-S algorithm using either the full or selected spectra reflectance to select the high-yielding soybean genotypes, among a large number of genotypes, at early growth stages.

Robust and Discriminative Brain Genome Association Study

Brain Genome Association (BGA) study, which investigates the associations between brain structure/function (characterized by neuroimaging phenotypes) and genetic variations (characterized by Single Nucleotide Polymorphisms (SNPs)), is important in pathological analysis of neurological disease. However, the current BGA studies are limited as they did not explicitly consider the disease labels, source importance, and sample importance in their formulations. We address these issues by proposing a robust and discriminative BGA formulation. Specifically, we learn two transformation matrices for mapping two heterogeneous data sources (i.e., neuroimaging data and genetic data) into a common space, so that the samples from the same subject (but diffrent sources) are close to each other, and also the samples with diffrent labels are separable. In addition, we add a sparsity constraint on the transformation matrices to enable feature selection on both data sources. Furthermore, both sample importance and source importance are also considered in the formulation via adaptive parameter-free sample and source weightings. We have conducted various experiments, using Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, to test how well the neuroimaging phenotypes and SNPs can represent each other in the common space.

Group sparse reduced rank regression for neuroimaging genetic study

The neuroimaging genetic study usually needs to deal with high dimensionality of both brain imaging data and genetic data, so that often resulting in the issue of curse of dimensionality. In this paper, we propose a group sparse reduced rank regression model to take the relations of both the phenotypes and the genotypes for the neuroimaging genetic study. Specifically, we propose designing a graph sparsity constraint as well as a reduced rank constraint to simultaneously conduct subspace learning and feature selection. The group sparsity constraint conducts feature selection to identify genotypes highly related to neuroimaging data, while the reduced rank constraint considers the relations among neuroimaging data to conduct subspace learning in the feature selection model. Furthermore, an alternative optimization algorithm is proposed to solve the resulting objective function and is proved to achieve fast convergence. Experimental results on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset showed that the proposed method has superiority on predicting the phenotype data by the genotype data, than the alternative methods under comparison.