An Improved Ensemble Learning Method for Classifying High-Dimensional and Imbalanced Biomedicine Data

Specialty detection in the context of telemedicine in a highly imbalanced multi-class distribution

The Covid-19 pandemic has led to an increase in the awareness of and demand for telemedicine services, resulting in a need for automating the process and relying on machine learning (ML) to reduce the operational load. This research proposes a specialty detection classifier based on a machine learning model to automate the process of detecting the correct specialty for each question and routing it to the correct doctor. The study focuses on handling multiclass and highly imbalanced datasets for Arabic medical questions, comparing some oversampling techniques, developing a Deep Neural Network (DNN) model for specialty detection, and exploring the hidden business areas that rely on specialty detection such as customizing and personalizing the consultation flow for different specialties. The proposed module is deployed in both synchronous and asynchronous medical consultations to provide more real-time classification, minimize the doctor effort in addressing the correct specialty, and give the system more flexibility in customizing the medical consultation flow. The evaluation and assessment are based on accuracy, precision, recall, and F1-score. The experimental results suggest that combining multiple techniques, such as SMOTE and reweighing with keyword identification, is necessary to achieve improved performance in detecting rare classes in imbalanced multiclass datasets. By using these techniques, specialty detection models can more accurately detect rare classes in real-world scenarios where imbalanced data is common.

Improved prediction of glioma-related aphasia by diffusion MRI metrics, machine learning, and automated fiber bundle segmentation

White matter impairments caused by gliomas can lead to functional disorders. In this study, we predicted aphasia in patients with gliomas infiltrating the language network using machine learning methods. We included 78 patients with left-hemispheric perisylvian gliomas. Aphasia was graded preoperatively using the Aachen aphasia test (AAT). Subsequently, we created bundle segmentations based on automatically generated tract orientation mappings using TractSeg. To prepare the input for the support vector machine (SVM), we first preselected aphasia-related fiber bundles based on the associations between relative tract volumes and AAT subtests. In addition, diffusion magnetic resonance imaging (dMRI)-based metrics [axial diffusivity (AD), apparent diffusion coefficient (ADC), fractional anisotropy (FA), and radial diffusivity (RD)] were extracted within the fiber bundles' masks with their mean, standard deviation, kurtosis, and skewness values. Our model consisted of random forest-based feature selection followed by an SVM. The best model performance achieved 81% accuracy (specificity = 85%, sensitivity = 73%, and AUC = 85%) using dMRI-based features, demographics, tumor WHO grade, tumor location, and relative tract volumes. The most effective features resulted from the arcuate fasciculus (AF), middle longitudinal fasciculus (MLF), and inferior fronto-occipital fasciculus (IFOF). The most effective dMRI-based metrics were FA, ADC, and AD. We achieved a prediction of aphasia using dMRI-based features and demonstrated that AF, IFOF, and MLF were the most important fiber bundles for predicting aphasia in this cohort.

Adaptive Subspace Optimization Ensemble Method for High-Dimensional Imbalanced Data Classification

It is hard to construct an optimal classifier for high-dimensional imbalanced data, on which the performance of classifiers is seriously affected and becomes poor. Although many approaches, such as resampling, cost-sensitive, and ensemble learning methods, have been proposed to deal with the skewed data, they are constrained by high-dimensional data with noise and redundancy. In this study, we propose an adaptive subspace optimization ensemble method (ASOEM) for high-dimensional imbalanced data classification to overcome the above limitations. To construct accurate and diverse base classifiers, a novel adaptive subspace optimization (ASO) method based on adaptive subspace generation (ASG) process and rotated subspace optimization (RSO) process is designed to generate multiple robust and discriminative subspaces. Then a resampling scheme is applied on the optimized subspace to build a class-balanced data for each base classifier. To verify the effectiveness, our ASOEM is implemented based on different resampling strategies on 24 real-world high-dimensional imbalanced datasets. Experimental results demonstrate that our proposed methods outperform other mainstream imbalance learning approaches and classifier ensemble methods.

A new hybrid algorithm for three-stage gene selection based on whale optimization

In biomedical data mining, the gene dimension is often much larger than the sample size. To solve this problem, we need to use a feature selection algorithm to select feature gene subsets with a strong correlation with phenotype to ensure the accuracy of subsequent analysis. This paper presents a new three-stage hybrid feature gene selection method, that combines a variance filter, extremely randomized tree, and whale optimization algorithm. First, a variance filter is used to reduce the dimension of the feature gene space, and an extremely randomized tree is used to further reduce the feature gene set. Finally, the whale optimization algorithm is used to select the optimal feature gene subset. We evaluate the proposed method with three different classifiers in seven published gene expression profile datasets and compare it with other advanced feature selection algorithms. The results show that the proposed method has significant advantages in a variety of evaluation indicators.

A novel hybrid algorithm based on Harris Hawks for tumor feature gene selection

Background

Gene expression data are often used to classify cancer genes. In such high-dimensional datasets, however, only a few feature genes are closely related to tumors. Therefore, it is important to accurately select a subset of feature genes with high contributions to cancer classification.

Methods

In this article, a new three-stage hybrid gene selection method is proposed that combines a variance filter, extremely randomized tree and Harris Hawks (VEH). In the first stage, we evaluated each gene in the dataset through the variance filter and selected the feature genes that meet the variance threshold. In the second stage, we use extremely randomized tree to further eliminate irrelevant genes. Finally, we used the Harris Hawks algorithm to select the gene subset from the previous two stages to obtain the optimal feature gene subset.

Results

We evaluated the proposed method using three different classifiers on eight published microarray gene expression datasets. The results showed a 100% classification accuracy for VEH in gastric cancer, acute lymphoblastic leukemia and ovarian cancer, and an average classification accuracy of 95.33% across a variety of other cancers. Compared with other advanced feature selection algorithms, VEH has obvious advantages when measured by many evaluation criteria.

A hybrid sampling algorithm combining synthetic minority over-sampling technique and edited nearest neighbor for missed abortion diagnosis

BACKGROUND

Clinical diagnosis based on machine learning usually uses case samples as training samples, and uses machine learning to construct disease prediction models characterized by descriptive texts of clinical manifestations. However, the problem of sample imbalance often exists in the medical field, which leads to a decrease in classification performance of the machine learning.

METHODS

To solve the problem of sample imbalance in medical dataset, we propose a hybrid sampling algorithm combining synthetic minority over-sampling technique (SMOTE) and edited nearest neighbor (ENN). Firstly, the SMOTE is used to over-sampling missed abortion and diabetes datasets, so that the number of samples of the two classes is balanced. Then, ENN is used to under-sampling the over-sampled dataset to delete the "noisy sample" in the majority. Finally, Random forest is used to model and predict the sampled missed abortion and diabetes datasets to achieve an accurate clinical diagnosis.

RESULTS

Experimental results show that Random forest has the best classification performance on missed abortion and diabetes datasets after SMOTE-ENN sampled, and the MCC index is 95.6% and 90.0%, respectively. In addition, the results of pairwise comparison and multiple comparisons show that the SMOTE-ENN is significantly better than other sampling algorithms.

CONCLUSION

Random forest has significantly improved all indexes on the missed abortion dataset after SMOTE-ENN sampled.

Is handling unbalanced datasets for machine learning uplifts system performance?: A case of diabetic prediction

BACKGROUND AND AIMS

Healthcare is a sensitive sector, and addressing the class imbalance in the healthcare domain is a time-consuming task for machine learning-based systems due to the vast amount of data. This study looks into the impact of socioeconomic disparities on the healthcare data of diabetic patients to make accurate disease predictions.

METHODS

This study proposed a systematic approach of Closest Distance Ranking and Principal Component Analysis to deal with the unbalanced dataset. A typical machine learning technique was used to analyze the proposed approach. The data set of pregnant diabetic women is analysed for accurate detection.

RESULTS

The results of the case are analysed using sensitivity, which demonstrates that the minority class's lack of information makes it impossible to forecast the results. On the other hand, the unbalanced dataset was treated using the proposed technique and evaluated with the machine learning algorithm which significantly increased the performance of the system.

CONCLUSION

The performance of the machine learning-based system was significantly enhanced by the unbalanced dataset which was processed with the proposed technique and evaluated with the machine learning algorithm. For the first time, an unbalanced dataset was treated with a combination of Closest Distance Ranking and Principal Component Analysis.

Prediction of lymphedema occurrence in patients with breast cancer using the optimized combination of ensemble learning algorithm and feature selection

BACKGROUND

Breast cancer-related lymphedema is one of the most important complications that adversely affect patients' quality of life. Lymphedema can be managed if its risk factors are known and can be modified. This study aimed to select an appropriate model to predict the risk of lymphedema and determine the factors affecting lymphedema.

METHOD

This study was conducted on data of 970 breast cancer patients with lymphedema referred to a lymphedema clinic. This study was designed in two phases: developing an appropriate model to predict the risk of lymphedema and identifying the risk factors. The first phase included data preprocessing, optimizing feature selection for each base learner by the Genetic algorithm, optimizing the combined ensemble learning method, and estimating fitness function for evaluating an appropriate model. In the second phase, the influential variables were assessed and introduced based on the average number of variables in the output of the proposed algorithm.

RESULT

Once the sensitivity and accuracy of the algorithms were evaluated and compared, the Support Vector Machine algorithm showed the highest sensitivity and was found to be the superior model for predicting lymphedema. Meanwhile, the combined method had an accuracy coefficient of 91%. The extracted significant features in the proposed model were the number of lymph nodes to the number of removed lymph nodes ratio (68%), feeling of heaviness (67%), limited range of motion in the affected limb (65%), the number of the removed lymph nodes ( 64%), receiving radiotherapy (63%), misalignment of the dominant and the involved limb (62%), presence of fibrotic tissue (62%), type of surgery (62%), tingling sensation (62%), the number of the involved lymph nodes (61%), body mass index (61%), the number of chemotherapy sessions (60%), age (58%), limb injury (53%), chemotherapy regimen (53%), and occupation (50%).

CONCLUSION

Applying a combination of ensemble learning approach with the selected classification algorithms, feature selection, and optimization by Genetic algorithm, Lymphedema can be predicted with appropriate accuracy. Developing applications by effective variables to determine the risk of lymphedema can help lymphedema clinics choose the proper preventive and therapeutic method.

Classifier Ensemble Based on Multiview Optimization for High-Dimensional Imbalanced Data Classification

High-dimensional class imbalanced data have plagued the performance of classification algorithms seriously. Because of a large number of redundant/invalid features and the class imbalanced issue, it is difficult to construct an optimal classifier for high-dimensional imbalanced data. Classifier ensemble has attracted intensive attention since it can achieve better performance than an individual classifier. In this work, we propose a multiview optimization (MVO) to learn more effective and robust features from high-dimensional imbalanced data, based on which an accurate and robust ensemble system is designed. Specifically, an optimized subview generation (OSG) in MVO is first proposed to generate multiple optimized subviews from different scenarios, which can strengthen the classification ability of features and increase the diversity of ensemble members simultaneously. Second, a new evaluation criterion that considers the distribution of data in each optimized subview is developed based on which a selective ensemble of optimized subviews (SEOS) is designed to perform the subview selective ensemble. Finally, an oversampling approach is executed on the optimized view to obtain a new class rebalanced subset for the classifier. Experimental results on 25 high-dimensional class imbalanced datasets indicate that the proposed method outperforms other mainstream classifier ensemble methods.

Hybrid gene selection approach using XGBoost and multi-objective genetic algorithm for cancer classification

Microarray gene expression data are often accompanied by a large number of genes and a small number of samples. However, only a few of these genes are relevant to cancer, resulting in significant gene selection challenges. Hence, we propose a two-stage gene selection approach by combining extreme gradient boosting (XGBoost) and a multi-objective optimization genetic algorithm (XGBoost-MOGA) for cancer classification in microarray datasets. In the first stage, the genes are ranked using an ensemble-based feature selection using XGBoost. This stage can effectively remove irrelevant genes and yield a group comprising the most relevant genes related to the class. In the second stage, XGBoost-MOGA searches for an optimal gene subset based on the most relevant genes' group using a multi-objective optimization genetic algorithm. We performed comprehensive experiments to compare XGBoost-MOGA with other state-of-the-art feature selection methods using two well-known learning classifiers on 14 publicly available microarray expression datasets. The experimental results show that XGBoost-MOGA yields significantly better results than previous state-of-the-art algorithms in terms of various evaluation criteria, such as accuracy, F-score, precision, and recall.

Microclustering-Based Multi-Class Classification on Imbalanced Multi-Relational Datasets

In a relational database, classification algorithms are used to look for patterns across several interconnected relations. Most of the methods for multi-relational classification algorithms implicitly assume that the classes in the target relation are equally represented. Thus, they tend to produce poor predictive performance over the imbalanced dataset. In this paper, the authors propose an algorithm-level method MCMRC_IB for the classification of imbalanced multi-relational dataset. The proposed method extends MCMRC which is for balanced datasets. MCMRC_IB exploits the property of the imbalanced datasets that the minority class is represented by a smaller number of records usually 20-30% of the total records and is to be dealt accordingly by giving them weightages. The proposed method is able to handle multiple classes. Experimental results conðrm the efficiency of the proposed method in terms of predictive accuracy, F-measure, and G-mean.

Customer lifetime value prediction for gaming industry: fuzzy clustering based approach

The use of fuzzy logic in machine learning is becoming widespread. In machine learning problems, the data, which have different characteristics, are trained and predicted together. Training the model consisting of data with different characteristics can increase the rate of error in prediction. In this study, we suggest a new approach to assembling prediction with fuzzy clustering. Our approach aims to cluster the data according to their fuzzy membership value and model it with similar characteristics. This approach allows for efficient clustering of objects with more than one cluster characteristic. On the other hand, our approach will enable us to combine boosting type ensemble algorithms, which are various forms of assemblies that are widely used in machine learning due to their excellent success in the literature. We used a mobile game’s customers’ marketing and gameplay data for predicting their customer lifetime value for testing our approach. Customer lifetime value prediction for users is crucial for determining the marketing cost cap for companies. The findings reveal that using a fuzzy method to ensemble the algorithms outperforms implementing the algorithms individually.

Precision health data: Requirements, challenges and existing techniques for data security and privacy

Precision health leverages information from various sources, including omics, lifestyle, environment, social media, medical records, and medical insurance claims to enable personalized care, prevent and predict illness, and precise treatments. It extensively uses sensing technologies (e.g., electronic health monitoring devices), computations (e.g., machine learning), and communication (e.g., interaction between the health data centers). As health data contain sensitive private information, including the identity of patient and carer and medical conditions of the patient, proper care is required at all times. Leakage of these private information affects the personal life, including bullying, high insurance premium, and loss of job due to the medical history. Thus, the security, privacy of and trust on the information are of utmost importance. Moreover, government legislation and ethics committees demand the security and privacy of healthcare data. Besides, the public, who is the data source, always expects the security, privacy, and trust of their data. Otherwise, they can avoid contributing their data to the precision health system. Consequently, as the public is the targeted beneficiary of the system, the effectiveness of precision health diminishes. Herein, in the light of precision health data security, privacy, ethical and regulatory requirements, finding the best methods and techniques for the utilization of the health data, and thus precision health is essential. In this regard, firstly, this paper explores the regulations, ethical guidelines around the world, and domain-specific needs. Then it presents the requirements and investigates the associated challenges. Secondly, this paper investigates secure and privacy-preserving machine learning methods suitable for the computation of precision health data along with their usage in relevant health projects. Finally, it illustrates the best available techniques for precision health data security and privacy with a conceptual system model that enables compliance, ethics clearance, consent management, medical innovations, and developments in the health domain.

Feature Selection for High-Dimensional and Imbalanced Biomedical Data Based on Robust Correlation Based Redundancy and Binary Grasshopper Optimization Algorithm

The training machine learning algorithm from an imbalanced data set is an inherently challenging task. It becomes more demanding with limited samples but with a massive number of features (high dimensionality). The high dimensional and imbalanced data set has posed severe challenges in many real-world applications, such as biomedical data sets. Numerous researchers investigated either imbalanced class or high dimensional data sets and came up with various methods. Nonetheless, few approaches reported in the literature have addressed the intersection of the high dimensional and imbalanced class problem due to their complicated interactions. Lately, feature selection has become a well-known technique that has been used to overcome this problem by selecting discriminative features that represent minority and majority class. This paper proposes a new method called Robust Correlation Based Redundancy and Binary Grasshopper Optimization Algorithm (rCBR-BGOA); rCBR-BGOA has employed an ensemble of multi-filters coupled with the Correlation-Based Redundancy method to select optimal feature subsets. A binary Grasshopper optimisation algorithm (BGOA) is used to construct the feature selection process as an optimisation problem to select the best (near-optimal) combination of features from the majority and minority class. The obtained results, supported by the proper statistical analysis, indicate that rCBR-BGOA can improve the classification performance for high dimensional and imbalanced datasets in terms of G-mean and the Area Under the Curve (AUC) performance metrics.

Imbalanced biomedical data classification using self-adaptive multilayer ELM combined with dynamic GAN

BACKGROUND

Imbalanced data classification is an inevitable problem in medical intelligent diagnosis. Most of real-world biomedical datasets are usually along with limited samples and high-dimensional feature. This seriously affects the classification performance of the model and causes erroneous guidance for the diagnosis of diseases. Exploring an effective classification method for imbalanced and limited biomedical dataset is a challenging task.

METHODS

In this paper, we propose a novel multilayer extreme learning machine (ELM) classification model combined with dynamic generative adversarial net (GAN) to tackle limited and imbalanced biomedical data. Firstly, principal component analysis is utilized to remove irrelevant and redundant features. Meanwhile, more meaningful pathological features are extracted. After that, dynamic GAN is designed to generate the realistic-looking minority class samples, thereby balancing the class distribution and avoiding overfitting effectively. Finally, a self-adaptive multilayer ELM is proposed to classify the balanced dataset. The analytic expression for the numbers of hidden layer and node is determined by quantitatively establishing the relationship between the change of imbalance ratio and the hyper-parameters of the model. Reducing interactive parameters adjustment makes the classification model more robust.

RESULTS

To evaluate the classification performance of the proposed method, numerical experiments are conducted on four real-world biomedical datasets. The proposed method can generate authentic minority class samples and self-adaptively select the optimal parameters of learning model. By comparing with W-ELM, SMOTE-ELM, and H-ELM methods, the quantitative experimental results demonstrate that our method can achieve better classification performance and higher computational efficiency in terms of ROC, AUC, G-mean, and F-measure metrics.

CONCLUSIONS

Our study provides an effective solution for imbalanced biomedical data classification under the condition of limited samples and high-dimensional feature. The proposed method could offer a theoretical basis for computer-aided diagnosis. It has the potential to be applied in biomedical clinical practice.

A novel adaptive ensemble classification framework for ADME prediction

AECF is a GA based ensemble method. It includes four components which are (1) data balancing, (2) generating individual models, (3) combining individual models, and (4) optimizing the ensemble.

Multiclass imbalance learning: Improving classification of pediatric brain tumors from magnetic resonance spectroscopy

Purpose

Classification of pediatric brain tumors from ¹H‐magnetic resonance spectroscopy (MRS) can aid diagnosis and management of brain tumors. However, varied incidence of the different tumor types leads to imbalanced class sizes and introduces difficulties in classifying rare tumor groups. This study assessed different imbalanced multiclass learning techniques and compared the use of complete spectra and quantified metabolite profiles for classification of three main childhood brain tumor types.

Methods

Single‐voxel, Short echo time MRS data were collected from 90 patients with pilocytic astrocytoma (n = 42), medulloblastoma (n = 38), or ependymoma (n = 10). Both spectra and metabolite profiles were used to develop the learning algorithms. The borderline synthetic minority oversampling technique and AdaboostM1 were used to correct for the skewed distribution. Classifiers were trained using five different pattern recognition algorithms.

Results

Use of imbalanced learning techniques improved the balanced accuracy rate (BAR) of all classification methods (average BAR over all classification methods for spectra: oversampled data = 0.81, original = 0.63, P < 0.001; metabolite concentration: oversampled‐data = 0.91, original = 0.75, P < 0.0001). Performance of all classifiers in discriminating ependymomas increased when oversampled data were used compared with original data for both complete spectra (F‐measure P < 0.01) and metabolite profile (F‐measure P < 0.001).

Conclusion

Imbalanced learning techniques improve the classification accuracy of childhood brain tumors from MRS where group sizes differ and facilitate the inclusion of rarer tumor types into clinical decision support systems. Magn Reson Med 77:2114–2124, 2017. © 2016 The Authors Magnetic Resonance in Medicine published by Wiley Periodicals, Inc. on behalf of International Society for Magnetic Resonance in Medicine. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

An Ensemble Learning Based Framework for Traditional Chinese Medicine Data Analysis with ICD-10 Labels

Objective. This study aims to establish a model to analyze clinical experience of TCM veteran doctors. We propose an ensemble learning based framework to analyze clinical records with ICD-10 labels information for effective diagnosis and acupoints recommendation. Methods. We propose an ensemble learning framework for the analysis task. A set of base learners composed of decision tree (DT) and support vector machine (SVM) are trained by bootstrapping the training dataset. The base learners are sorted by accuracy and diversity through nondominated sort (NDS) algorithm and combined through a deep ensemble learning strategy. Results. We evaluate the proposed method with comparison to two currently successful methods on a clinical diagnosis dataset with manually labeled ICD-10 information. ICD-10 label annotation and acupoints recommendation are evaluated for three methods. The proposed method achieves an accuracy rate of 88.2%  ±  2.8% measured by zero-one loss for the first evaluation session and 79.6%  ±  3.6% measured by Hamming loss, which are superior to the other two methods. Conclusion. The proposed ensemble model can effectively model the implied knowledge and experience in historic clinical data records. The computational cost of training a set of base learners is relatively low.