A survey on feature selection approaches for clustering

Cluster-based radiomics reveal spatial heterogeneity of bevacizumab response for treatment of radiotherapy-induced cerebral necrosis

Background

Bevacizumab is used in the treatment of radiation necrosis (RN), which is a debilitating toxicity following head and neck radiotherapy. However, there is no biomarker to predict if a patient would respond to bevacizumab.

Purpose

We aimed to develop a cluster-based radiomics approach to characterize the spatial heterogeneity of RN and map their responses to bevacizumab.

Methods

118 consecutive nasopharyngeal carcinoma patients diagnosed with RN were enrolled. We divided 152 lesions from the patients into 101 for training, and 51 for validation. We extracted voxel-level radiomics features from each lesion segmented on T1-weighted+contrast and T2 FLAIR sequences of pre- and post-bevacizumab magnetic resonance images, followed by a three-step analysis involving individual- and population-level clustering, before delta-radiomics to derive five radiomics clusters within the lesions. We tested the association of each cluster with response to bevacizumab and developed a clinico-radiomics model using clinical predictors and cluster-specific features.

Results

71 (70.3%) and 34 (66.7%) lesions had responded to bevacizumab in the training and validation datasets, respectively. Two radiomics clusters were spatially mapped to the edema region, and the volume changes were significantly associated with bevacizumab response (OR:11.12 [95% CI: 2.54-73.47], P = 0.004; and 1.63[1.07-2.78], P = 0.042). The combined clinico-radiomics model based on textural features extracted from the most significant cluster improved the prediction of bevacizumab response, compared with a clinical-only model (AUC:0.755 [0.645-0.865] to 0.852 [0.764-0.940], training; 0.708 [0.554-0.861] to 0.816 [0.699-0.933], validation).

Conclusion

Our radiomics approach yielded intralesional resolution, enabling a more refined feature selection for predicting bevacizumab efficacy in the treatment of RN.

An orchestra of machine learning methods reveals landmarks in single-cell data exemplified with aging fibroblasts

In this work, a Python framework for characteristic feature extraction is developed and applied to gene expression data of human fibroblasts. Unlabeled feature selection objectively determines groups and minimal gene sets separating groups. ML explainability methods transform the features correlating with phenotypic differences into causal reasoning, supported by further pipeline and visualization tools, allowing user knowledge to boost causal reasoning. The purpose of the framework is to identify characteristic features that are causally related to phenotypic differences of single cells. The pipeline consists of several data science methods enriched with purposeful visualization of the intermediate results in order to check them systematically and infuse the domain knowledge about the investigated process. A specific focus is to extract a small but meaningful set of genes to facilitate causal reasoning for the phenotypic differences. One application could be drug target identification. For this purpose, the framework follows different steps: feature reduction (PFA), low dimensional embedding (UMAP), clustering ((H)DBSCAN), feature correlation (chi-square, mutual information), ML validation and explainability (SHAP, tree explainer). The pipeline is validated by identifying and correctly separating signature genes associated with aging in fibroblasts from single-cell gene expression measurements: PLK3, polo-like protein kinase 3; CCDC88A, Coiled-Coil Domain Containing 88A; STAT3, signal transducer and activator of transcription-3; ZNF7, Zinc Finger Protein 7; SLC24A2, solute carrier family 24 member 2 and lncRNA RP11-372K14.2. The code for the preprocessing step can be found in the GitHub repository https://github.com/AC-PHD/NoLabelPFA, along with the characteristic feature extraction https://github.com/LauritzR/characteristic-feature-extraction.

EEG-based emotion recognition using dual tree complex wavelet transform and random subspace ensemble classifier

Emotions are strongly admitted as a main source to establish meaningful interactions between humans and computers. Thanks to the advancements in electroencephalography (EEG), especially in the usage of portable and cheap wearable EEG devices, the demand for identifying emotions has extremely increased. However, the overall scientific knowledge and works concerning EEG-based emotion recognition is still limited. To cover this issue, we introduce an EEG-based emotion recognition framework in this study. The proposed framework involves the following stages: preprocessing, feature extraction, feature selection and classification. For the preprocessing stage, multi scale principle component analysis and sysmlets-4 filter are used. A version of discrete wavelet transform (DWT), namely dual tree complex wavelet transform (DTCWT) is utilized for the feature extraction stage. To reduce the feature dimension size, a variety of statistical criteria are employed. For the final stage, we adopt ensemble classifiers due to their promising performance in classification problems. The proposed framework achieves nearly 96.8% accuracy by using random subspace ensemble classifier. It can therefore be resulted that the proposed EEG-based framework performs well in terms of identifying emotions.

A study on feature selection using multi-domain feature extraction for automated k-complex detection

Background

K-complex detection plays a significant role in the field of sleep research. However, manual annotation for electroencephalography (EEG) recordings by visual inspection from experts is time-consuming and subjective. Therefore, there is a necessity to implement automatic detection methods based on classical machine learning algorithms. However, due to the complexity of EEG signal, current feature extraction methods always produce low relevance to k-complex detection, which leads to a great performance loss for the detection. Hence, finding compact yet effective integrated feature vectors becomes a crucially core task in k-complex detection.

Method

In this paper, we first extract multi-domain features based on time, spectral analysis, and chaotic theory. Those features are extracted from a 0.5-s EEG segment, which is obtained using the sliding window technique. As a result, a vector containing twenty-two features is obtained to represent each segment. Next, we explore several feature selection methods and compare their performance in detecting k-complex. Based on the analysis of the selected features, we identify compact features which are fewer than twenty-two features and deemed as relevant and proceeded to the next step. Additionally, three classical classifiers are employed to evaluate the performance of the feature selection models.

Results

The results demonstrate that combining different features significantly improved the k-complex detection performance. The best performance is achieved by applying the feature selection method, which results in an accuracy of 93.03%± 7.34, sensitivity of 93.81%± 5.62%, and specificity 94.13± 5.81, respectively, using a smaller number of the combined feature sets.

Conclusion

The proposed method in this study can serve as an efficient tool for the automatic detection of k-complex, which is useful for neurologists or doctors in the diagnosis of sleep research.

Leading consumption patterns of psychoactive substances in Colombia: A deep neural network-based clustering-oriented embedding approach

The number of health-related incidents caused using illegal and legal psychoactive substances (PAS) has dramatically increased over two decades worldwide. In Colombia, the use of illicit substances has increased up to 10.3%, while the consumption alcohol and tobacco has increased to 84% and 12%, respectively. It is well-known that identifying drug consumption patterns in the general population is essential in reducing overall drug consumption. However, existing approaches do not incorporate Machine Learning and/or Deep Data Mining methods in combination with spatial techniques. To enhance our understanding of mental health issues related to PAS and assist in the development of national policies, here we present a novel Deep Neural Network-based Clustering-oriented Embedding Algorithm that incorporates an autoencoder and spatial techniques. The primary goal of our model is to identify general and spatial patterns of drug consumption and abuse, while also extracting relevant features from the input data and identifying clusters during the learning process. As a test case, we used the largest publicly available database of legal and illegal PAS consumption comprising 49,600 Colombian households. We estimated and geographically represented the prevalence of consumption and/or abuse of both PAS and non-PAS, while achieving statistically significant goodness-of-fit values. Our results indicate that region, sex, housing type, socioeconomic status, age, and variables related to household finances contribute to explaining the patterns of consumption and/or abuse of PAS. Additionally, we identified three distinct patterns of PAS consumption and/or abuse. At the spatial level, these patterns indicate concentrations of drug consumption in specific regions of the country, which are closely related to specific geographic locations and the prevailing social and environmental contexts. These findings can provide valuable insights to facilitate decision-making and develop national policies targeting specific groups given their cultural, geographic, and social conditions.

Computer-aided diagnosis systems: a comparative study of classical machine learning versus deep learning-based approaches

The diagnostic phase of the treatment process is essential for patient guidance and follow-up. The accuracy and effectiveness of this phase can determine the life or death of a patient. For the same symptoms, different doctors may come up with different diagnoses whose treatments may, instead of curing a patient, be fatal. Machine learning (ML) brings new solutions to healthcare professionals to save time and optimize the appropriate diagnosis. ML is a data analysis method that automates the creation of analytical models and promotes predictive data. There are several ML models and algorithms that rely on features extracted from, for example, a patient's medical images to indicate whether a tumor is benign or malignant. The models differ in the way they operate and the method used to extract the discriminative features of the tumor. In this article, we review different ML models for tumor classification and COVID-19 infection to evaluate the different works. The computer-aided diagnosis (CAD) systems, which we referred to as classical, are based on accurate feature identification, usually performed manually or with other ML techniques that are not involved in classification. The deep learning-based CAD systems automatically perform the identification and extraction of discriminative features. The results show that the two types of DAC have quite close performances but the use of one or the other type depends on the datasets. Indeed, manual feature extraction is necessary when the size of the dataset is small; otherwise, deep learning is used.

Semisupervised Bacterial Heuristic Feature Selection Algorithm for High-Dimensional Classification with Missing Labels

Feature selection is a crucial method for discovering relevant features in high-dimensional data. However, most studies primarily focus on completely labeled data, ignoring the frequent occurrence of missing labels in real-world problems. To address high-dimensional and label-missing problems in data classification simultaneously, we proposed a semisupervised bacterial heuristic feature selection algorithm. To track the label-missing problem, a k-nearest neighbor semisupervised learning strategy is designed to reconstruct missing labels. In addition, the bacterial heuristic algorithm is improved using hierarchical population initialization, dynamic learning, and elite population evolution strategies to enhance the search capacity for various feature combinations. To verify the effectiveness of the proposed algorithm, three groups of comparison experiments based on eight datasets are employed, including two traditional feature selection methods, four bacterial heuristic feature selection algorithms, and two swarm-based heuristic feature selection algorithms. Experimental results demonstrate that the proposed algorithm has obvious advantages in terms of classification accuracy and selected feature numbers.

A Fast Hybrid Feature Selection Based on Correlation-Guided Clustering and Particle Swarm Optimization for High-Dimensional Data

The "curse of dimensionality" and the high computational cost have still limited the application of the evolutionary algorithm in high-dimensional feature selection (FS) problems. This article proposes a new three-phase hybrid FS algorithm based on correlation-guided clustering and particle swarm optimization (PSO) (HFS-C-P) to tackle the above two problems at the same time. To this end, three kinds of FS methods are effectively integrated into the proposed algorithm based on their respective advantages. In the first and second phases, a filter FS method and a feature clustering-based method with low computational cost are designed to reduce the search space used by the third phase. After that, the third phase applies oneself to finding an optimal feature subset by using an evolutionary algorithm with the global searchability. Moreover, a symmetric uncertainty-based feature deletion method, a fast correlation-guided feature clustering strategy, and an improved integer PSO are developed to improve the performance of the three phases, respectively. Finally, the proposed algorithm is validated on 18 publicly available real-world datasets in comparison with nine FS algorithms. Experimental results show that the proposed algorithm can obtain a good feature subset with the lowest computational cost.

Decoding clinical biomarker space of COVID-19: Exploring matrix factorization-based feature selection methods

One of the most critical challenges in managing complex diseases like COVID-19 is to establish an intelligent triage system that can optimize the clinical decision-making at the time of a global pandemic. The clinical presentation and patients' characteristics are usually utilized to identify those patients who need more critical care. However, the clinical evidence shows an unmet need to determine more accurate and optimal clinical biomarkers to triage patients under a condition like the COVID-19 crisis. Here we have presented a machine learning approach to find a group of clinical indicators from the blood tests of a set of COVID-19 patients that are predictive of poor prognosis and morbidity. Our approach consists of two interconnected schemes: Feature Selection and Prognosis Classification. The former is based on different Matrix Factorization (MF)-based methods, and the latter is performed using Random Forest algorithm. Our model reveals that Arterial Blood Gas (ABG) O₂ Saturation and C-Reactive Protein (CRP) are the most important clinical biomarkers determining the poor prognosis in these patients. Our approach paves the path of building quantitative and optimized clinical management systems for COVID-19 and similar diseases.

An Intelligent ECG-Based Tool for Diagnosing COVID-19 via Ensemble Deep Learning Techniques

Diagnosing COVID-19 accurately and rapidly is vital to control its quick spread, lessen lockdown restrictions, and decrease the workload on healthcare structures. The present tools to detect COVID-19 experience numerous shortcomings. Therefore, novel diagnostic tools are to be examined to enhance diagnostic accuracy and avoid the limitations of these tools. Earlier studies indicated multiple structures of cardiovascular alterations in COVID-19 cases which motivated the realization of using ECG data as a tool for diagnosing the novel coronavirus. This study introduced a novel automated diagnostic tool based on ECG data to diagnose COVID-19. The introduced tool utilizes ten deep learning (DL) models of various architectures. It obtains significant features from the last fully connected layer of each DL model and then combines them. Afterward, the tool presents a hybrid feature selection based on the chi-square test and sequential search to select significant features. Finally, it employs several machine learning classifiers to perform two classification levels. A binary level to differentiate between normal and COVID-19 cases, and a multiclass to discriminate COVID-19 cases from normal and other cardiac complications. The proposed tool reached an accuracy of 98.2% and 91.6% for binary and multiclass levels, respectively. This performance indicates that the ECG could be used as an alternative means of diagnosis of COVID-19.

Efficient palmprint biometric identification systems using deep learning and feature selection methods

Over the past two decades, several studies have paid great attention to biometric palmprint recognition. Recently, most methods in literature adopted deep learning due to their high recognition accuracy and the capability to adapt with different acquisition palmprint images. However, high-dimensional data with a large number of uncorrelated and redundant features remain a challenge due to computational complexity issues. Feature selection is a process of selecting a subset of relevant features, which aims to decrease the dimensionality, reduce the running time, and improve the accuracy. In this paper, we propose efficient unimodal and multimodal biometric systems based on deep learning and feature selection. Our approach called simplified PalmNet–Gabor concentrates on the improvement of the PalmNet for fast recognition of multispectral and contactless palmprint images. Therefore, we used Log-Gabor filters in the preprocessing to increase the contrast of palmprint features. Then, we reduced the number of features using feature selection and dimensionality reduction procedures. For the multimodal system, we fused modalities at the matching score level to improve system performance. The proposed method effectively improves the accuracy of the PalmNet and reduces the number of features as well the computational time. We validated the proposed method on four public palmprint databases, two multispectral databases, CASIA and PolyU, and two contactless databases, Tongji and PolyU 2D/3D. Experiments show that our approach achieves a high recognition rate while using a substantially lower number of features.

High dimensionality reduction by matrix factorization for systems pharmacology

The extraction of predictive features from the complex high-dimensional multi-omic data is necessary for decoding and overcoming the therapeutic responses in systems pharmacology. Developing computational methods to reduce high-dimensional space of features in in vitro, in vivo and clinical data is essential to discover the evolution and mechanisms of the drug responses and drug resistance. In this paper, we have utilized the matrix factorization (MF) as a modality for high dimensionality reduction in systems pharmacology. In this respect, we have proposed three novel feature selection methods using the mathematical conception of a basis for features. We have applied these techniques as well as three other MF methods to analyze eight different gene expression datasets to investigate and compare their performance for feature selection. Our results show that these methods are capable of reducing the feature spaces and find predictive features in terms of phenotype determination. The three proposed techniques outperform the other methods used and can extract a 2-gene signature predictive of a tyrosine kinase inhibitor treatment response in the Cancer Cell Line Encyclopedia.

Distance-based clustering challenges for unbiased benchmarking studies

Benchmark datasets with predefined cluster structures and high-dimensional biomedical datasets outline the challenges of cluster analysis: clustering algorithms are limited in their clustering ability in the presence of clusters defining distance-based structures resulting in a biased clustering solution. Data sets might not have cluster structures. Clustering yields arbitrary labels and often depends on the trial, leading to varying results. Moreover, recent research indicated that all partition comparison measures can yield the same results for different clustering solutions. Consequently, algorithm selection and parameter optimization by unsupervised quality measures (QM) are always biased and misleading. Only if the predefined structures happen to meet the particular clustering criterion and QM, can the clusters be recovered. Results are presented based on 41 open-source algorithms which are particularly useful in biomedical scenarios. Furthermore, comparative analysis with mirrored density plots provides a significantly more detailed benchmark than that with the typically used box plots or violin plots.

Determining the number of hidden layer and hidden neuron of neural network for wind speed prediction

Artificial neural network (ANN) is one of the techniques in artificial intelligence, which has been widely applied in many fields for prediction purposes, including wind speed prediction. The aims of this research is to determine the topology of neural network that are used to predict wind speed. Topology determination means finding the hidden layers number and the hidden neurons number for corresponding hidden layer in the neural network. The difference between this research and previous research is that the objective function of this research is regression, while the objective function of previous research is classification. Determination of the topology of the neural network using principal component analysis (PCA) and K-means clustering. PCA is used to determine the hidden layers number, while clustering is used to determine the hidden neurons number for corresponding hidden layer. The selected topology is then used to predict wind speed. Then the performance of topology determination using PCA and clustering is then compared with several other methods. The results of the experiment show that the performance of the neural network topology determined using PCA and clustering has better performance than the other methods being compared. Performance is determined based on the RMSE value, the smaller the RMSE value, the better the neural network performance. In future research, it is necessary to apply a correlation or relationship between input attribute and output attribute and then analyzed, prior to conducting PCA and clustering analysis.

Decoding Clinical Biomarker Space of COVID-19: Exploring Matrix Factorization-based Feature Selection Methods

One of the most critical challenges in managing complex diseases like COVID-19 is to establish an intelligent triage system that can optimize the clinical decision-making at the time of a global pandemic. The clinical presentation and patients’ characteristics are usually utilized to identify those patients who need more critical care. However, the clinical evidence shows an unmet need to determine more accurate and optimal clinical biomarkers to triage patients under a condition like the COVID-19 crisis. Here we have presented a machine learning approach to find a group of clinical indicators from the blood tests of a set of COVID-19 patients that are predictive of poor prognosis and morbidity. Our approach consists of two interconnected schemes: Feature Selection and Prognosis Classification. The former is based on different Matrix Factorization (MF)-based methods, and the latter is performed using Random Forest algorithm. Our model reveals that Arterial Blood Gas (ABG) O 2 Saturation and C-Reactive Protein (CRP) are the most important clinical biomarkers determining the poor prognosis in these patients. Our approach paves the path of building quantitative and optimized clinical management systems for COVID-19 and similar diseases.

Identification of transcriptional subtypes in lung adenocarcinoma and squamous cell carcinoma through integrative analysis of microarray and RNA sequencing data

Classification of tumors into subtypes can inform personalized approaches to treatment including the choice of targeted therapies. The two most common lung cancer histological subtypes, lung adenocarcinoma and lung squamous cell carcinoma, have been previously divided into transcriptional subtypes using microarray data, and corresponding signatures were subsequently used to classify RNA-seq data. Cross-platform unsupervised classification facilitates the identification of robust transcriptional subtypes by combining vast amounts of publicly available microarray and RNA-seq data. However, cross-platform classification is challenging because of intrinsic differences in data generated using the two gene expression profiling technologies. In this report, we show that robust gene expression subtypes can be identified in integrated data representing over 3500 normal and tumor lung samples profiled using two widely used platforms, Affymetrix HG-U133 Plus 2.0 Array and Illumina HiSeq RNA sequencing. We tested and analyzed consensus clustering for 384 combinations of data processing methods. The agreement between subtypes identified in single-platform and cross-platform normalized data was then evaluated using a variety of statistics. Results show that unsupervised learning can be achieved with combined microarray and RNA-seq data using selected preprocessing, cross-platform normalization, and unsupervised feature selection methods. Our analysis confirmed three lung adenocarcinoma transcriptional subtypes, but only two consistent subtypes in squamous cell carcinoma, as opposed to four subtypes previously identified. Further analysis showed that tumor subtypes were associated with distinct patterns of genomic alterations in genes coding for therapeutic targets. Importantly, by integrating quantitative proteomics data, we were able to identify tumor subtype biomarkers that effectively classify samples on the basis of both gene and protein expression. This study provides the basis for further integrative data analysis across gene and protein expression profiling platforms.

cola: an R/Bioconductor package for consensus partitioning through a general framework

Classification of high-throughput genomic data is a powerful method to assign samples to subgroups with specific molecular profiles. Consensus partitioning is the most widely applied approach to reveal subgroups by summarizing a consensus classification from a list of individual classifications generated by repeatedly executing clustering on random subsets of the data. It is able to evaluate the stability of the classification. We implemented a new R/Bioconductor package, cola, that provides a general framework for consensus partitioning. With cola, various parameters and methods can be user-defined and easily integrated into different steps of an analysis, e.g., feature selection, sample classification or defining signatures. cola provides a new method named ATC (ability to correlate to other rows) to extract features and recommends spherical k-means clustering (skmeans) for subgroup classification. We show that ATC and skmeans have better performance than other commonly used methods by a comprehensive benchmark on public datasets. We also benchmark key parameters in the consensus partitioning procedure, which helps users to select optimal parameter values. Moreover, cola provides rich functionalities to apply multiple partitioning methods in parallel and directly compare their results, as well as rich visualizations. cola can automate the complete analysis and generates a comprehensive HTML report.

Edge Computing for Data Anomaly Detection of Multi-Sensors in Underground Mining

There is a growing interest in safety warning of underground mining due to the huge threat being faced by those working in underground mining. Data acquisition of sensors based on Internet of Things (IoT) is currently the main method, but the data anomaly detection and analysis of multi-sensors is a challenging task: firstly, the data that are collected by different sensors of underground mining are heterogeneous; secondly, real-time is required for the data anomaly detection of safety warning. Currently, there are many anomaly detection methods, such as traditional clustering methods K-means and C-means. Meanwhile, Artificial Intelligence (AI) is widely used in data analysis and prediction. However, K-means and C-means cannot directly process heterogeneous data, and AI algorithms require equipment with high computing and storage capabilities. IoT equipment of underground mining cannot perform complex calculation due to the limitation of energy consumption. Therefore, many existing methods cannot be directly used for IoT applications in underground mining. In this paper, a multi-sensors data anomaly detection method based on edge computing is proposed. Firstly, an edge computing model is designed, and according to the computing capabilities of different types of devices, anomaly detection tasks are migrated to different edge devices, which solve the problem of insufficient computing capabilities of the devices. Secondly, according to the requirements of different anomaly detection tasks, edge anomaly detection algorithms for sensor nodes and sink nodes are designed respectively. Lastly, an experimental platform is built for performance comparison analysis, and the experimental results show that the proposed algorithm has better performance in anomaly detection accuracy, delay, and energy consumption.