Attribute selection based on information gain ratio in fuzzy rough set theory with application to tumor classification

An Integrated Machine Learning Approach for Congestive Heart Failure Prediction

Congestive heart failure (CHF) is one of the primary sources of mortality and morbidity among the global population. Over 26 million individuals globally are affected by heart disease, and its prevalence is rising by 2% yearly. With advances in healthcare technologies, if we predict CHF in the early stages, one of the leading global mortality factors can be reduced. Therefore, the main objective of this study is to use machine learning applications to enhance the diagnosis of CHF and to reduce the cost of diagnosis by employing minimum features to forecast the possibility of a CHF occurring. We employ a deep neural network (DNN) classifier for CHF classification and compare the performance of DNN with various machine learning classifiers. In this research, we use a very challenging dataset, called the Cardiovascular Health Study (CHS) dataset, and a unique pre-processing technique by integrating C4.5 and K-nearest neighbor (KNN). While the C4.5 technique is used to find significant features and remove the outlier data from the dataset, the KNN algorithm is employed for missing data imputation. For classification, we compare six state-of-the-art machine learning (ML) algorithms (KNN, logistic regression (LR), naive Bayes (NB), random forest (RF), support vector machine (SVM), and decision tree (DT)) with DNN. To evaluate the performance, we use seven statistical measurements (i.e., accuracy, specificity, sensitivity, F1-score, precision, Matthew's correlation coefficient, and false positive rate). Overall, our results reflect our proposed integrated approach, which outperformed other machine learning algorithms in terms of CHF prediction, reducing patient expenses by reducing the number of medical tests. The proposed model obtained 97.03% F1-score, 95.30% accuracy, 96.49% sensitivity, and 97.58% precision.

Taxonomy of hybrid architectures involving rule-based reasoning and machine learning in clinical decision systems: A scoping review

BACKGROUND

As the application of Artificial Intelligence (AI) technologies increases in the healthcare sector, the industry faces a need to combine medical knowledge, often expressed as clinical rules, with advances in machine learning (ML), which offer high prediction accuracy at the expense of transparency of decision making.

PURPOSE

This paper seeks to review the present literature, identify hybrid architecture patterns that incorporate rules and machine learning, and evaluate the rationale behind their selection to inform future development and research on the design of transparent and precise clinical decision systems.

METHODS

PubMed, IEEE Explore, and Google Scholar were queried in search for papers from 1992 to 2022, with the keywords: "clinical decision system", "hybrid clinical architecture", "machine learning and clinical rules". Excluded articles did not use both ML and rules or did not provide any explanation of employed architecture. A proposed taxonomy was used to organize the results, analyze them, and depict them in graphical and tabular form. Two researchers, one with expertise in rule-based systems and another in ML, reviewed identified papers and discussed the work to minimize bias, and the third one re-reviewed the work to ensure consistency of reporting.

RESULTS

The authors screened 957 papers and reviewed 71 that met their criteria. Five distinct architecture archetypes were determined: Rules are Embedded in ML architecture (REML) (most used), ML pre-processes input data for Rule-Based inference (MLRB), Rule-Based method pre-processes input data for ML prediction (RBML), Rules influence ML training (RMLT), Parallel Ensemble of Rules and ML (PERML), which was rarely observed in clinical contexts.

CONCLUSIONS

Most architectures in the reviewed literature prioritize prediction accuracy over explainability and trustworthiness, which has led to more complex embedded approaches. Alternatively, parallel (PERML) architectures may be employed, allowing for a more transparent system that is easier to explain to patients and clinicians. The potential of this approach warrants further research.

OTHER

A limitation of the study may be that it reviews scientific literature, while algorithms implemented in clinical practice may present different distributions of motivations and implementations of hybrid architectures.

A Novel Variable Selection Method Based on Binning-Normalized Mutual Information for Multivariate Calibration

Variable (wavelength) selection is essential in the multivariate analysis of near-infrared spectra to improve model performance and provide a more straightforward interpretation. This paper proposed a new variable selection method named binning-normalized mutual information (B-NMI) based on information entropy theory. "Data binning" was applied to reduce the effects of minor measurement errors and increase the features of near-infrared spectra. "Normalized mutual information" was employed to calculate the correlation between each wavelength and the reference values. The performance of B-NMI was evaluated by two experimental datasets (ideal ternary solvent mixture dataset, fluidized bed granulation dataset) and two public datasets (gasoline octane dataset, corn protein dataset). Compared with classic methods of backward and interval PLS (BIPLS), variable importance projection (VIP), correlation coefficient (CC), uninformative variables elimination (UVE), and competitive adaptive reweighted sampling (CARS), B-NMI not only selected the most featured wavelengths from the spectra of complex real-world samples but also improved the stability and robustness of variable selection results.

An Efficient Iterative Approach to Explainable Feature Learning

This article introduces a new iterative approach to explainable feature learning. During each iteration, new features are generated, first by applying arithmetic operations on the input set of features. These are then evaluated in terms of probability distribution agreements between values of samples belonging to different classes. Finally, a graph-based approach for feature selection is proposed, which allows for selecting high-quality and uncorrelated features to be used in feature generation during the next iteration. As shown by the results, the proposed method improved the accuracy of all tested classifiers, where the best accuracies were achieved using random forest. In addition, the method turned out to be insensitive to both of the input parameters, while superior performances in comparison to the state of the art were demonstrated on nine out of 15 test sets and achieving comparable results in the others. Finally, we demonstrate the explainability of the learned feature representation for knowledge discovery.

Ensemble filters with harmonize PSO-SVM algorithm for optimal hearing disorder prediction

Discovering a hearing disorder at an earlier intervention is critical for reducing the effects of hearing loss and the approaches to increase the remaining hearing ability can be implemented to achieve the successful development of human communication. Recently, the explosive dataset features have increased the complexity for audiologists to decide the proper treatment for the patient. In most cases, data with irrelevant features and improper classifier parameters causes a crucial influence on the audiometry system in terms of accuracy. This is due to the dependent processes of these two, where the classification accuracy performance could be worsened if both processes are conducted independently. Although the filter algorithm is capable of eliminating irrelevant features, it still lacks the ability to consider feature reliance and results in a poor selection of significant features. Improper kernel parameter settings may also contribute to poor accuracy performance. In this paper, an ensemble filters feature selection based on Information Gain (IG), Gain Ratio (GR), Chi-squared (CS), and Relief-F (RF) with harmonize optimization of Particle Swarm Optimization (PSO) and Support Vector Machine (SVM) is presented to mitigate these problems. Ensemble filters are utilized so that the initial top dominant features relevant for classification can be considered. Then, PSO and SVM are optimized simultaneously to achieve the optimal solution. The results on a standard Audiology dataset show that the proposed method produces 96.50% accuracy with optimal solution compared to classical SVM, which signifies the proposed method is effective in handling high dimensional data for hearing disorder prediction.

Attribute Selection Based Genetic Network Programming for Intrusion Detection System

Intrusion detection, as a technology used to monitor abnormal behavior and maintain network security, has attracted many researchers’ attention in recent years. Thereinto, association rule mining is one of the mainstream methods to construct intrusion detection systems (IDS). However, the existing association rule algorithms face the challenges of high false positive rate and low detection rate. Meanwhile, too many rules might lead to the uncertainty increase that affects the performance of IDS. In order to tackle the above problems, a modified genetic network programming (GNP) is proposed for class association rule mining. Specifically, based on the property that node connections in the directed graph structure of GNP can be used to construct attribute associations, we propose to introduce information gain into GNP node selection. The most important attributes are thus selected, and the irrelevant attributes are removed before the rule is extracted. Moreover, not only the uncertainty among the class association rules is alleviated and also time consumption is reduced. The extracted rules can be applied to any classifier without affecting the detection performance. Experiment results based on NSL-KDD and KDDCup99 verify the performance of our proposed algorithm.

Gene Selection in a Single Cell Gene Space Based on D-S Evidence Theory

If the samples, features and information values in a real-valued information system are cells, genes and gene expression values, respectively, then for convenience, this system is said to be a single cell gene space. In the era of big data, people are faced with high dimensional gene expression data with redundancy and noise causing its strong uncertainty. D-S evidence theory excels at tackling the problem of uncertainty, and its conditions to be met are weaker than Bayesian probability theory. Therefore, this paper studies the gene selection in a single cell gene space to remove noise and redundancy with D-S evidence theory. The distance between two cells in each gene is first defined. Then, the tolerance relation is established according to the defined distance. In addition, the belief and plausibility functions to grasp the uncertainty of a single cell gene space are introduced on the basis of the tolerance classes. Statistical analysis shows that they can effectively measure the uncertainty of a single cell gene space. Furthermore, several gene selection algorithms in a single cell gene space are presented using the proposed belief and plausibility. Finally, the performance of the proposed algorithm is compared to other algorithms on some published single-cell data sets. Experimental results and statistical tests show that the classification and clustering performance of the presented algorithm not only exceeds the other three state-of-the-art algorithms, but also its gene reduction rate is very high.

Information entropy based attribute reduction for incomplete heterogeneous data

Attribute reduction in an information system (IS) is an important research topic in rough set theory (RST). This paper investigates attribute reduction for incomplete heterogeneous data based on information entropy. Information entropy in an incomplete IS with heterogeneous data (IISH) is first defined. Then, some derived notions of information entropy, such as joint information entropy, conditional information entropy, mutual information entropy, gain and gain ratio in an incomplete decision IS with heterogeneous data (IDISH), are presented. Next, information entropy is applied to perform attribute reduction in an IDISH. Two attribute reduction algorithms, based on gain and gain ratio, respectively, are proposed. Finally, in order to illustrate the feasibility and efficiency of the proposed algorithms, experimental analysis is carried out and comparisons are done. It is worth mentioning that the incomplete rate is used to deal with incomplete heterogeneous data.

Cancer Detection and Prediction Using Genetic Algorithms

Cancer is a wide category of diseases that is caused by the abnormal, uncontrollable growth of cells, and it is the second leading cause of death globally. Screening, early diagnosis, and prediction of recurrence give patients the best possible chance for successful treatment. However, these tests can be expensive and invasive and the results have to be interpreted by experts. Genetic algorithms (GAs) are metaheuristics that belong to the class of evolutionary algorithms. GAs can find the optimal or near-optimal solutions in huge, difficult search spaces and are widely used for search and optimization. This makes them ideal for detecting cancer by creating models to interpret the results of tests, especially noninvasive. In this article, we have comprehensively reviewed the existing literature, analyzed them critically, provided a comparative analysis of the state-of-the-art techniques, and identified the future challenges in the development of such techniques by medical professionals.

Rough fermatean fuzzy ideals in semigroups

In multiple criteria decision making (MCDM) problem, the decision maker deal with a situation where the sum of membership and non-membership grade of an attributes does not belongs to [0, 1]. To avoid such a situation, we proposed a new type of fuzzy system known as fermatean fuzzy system. More precisely, we presented the notion of fermatean fuzzy ideal theory and rough fermatean fuzzy sets in semigroups. The idea of lower and upper approximation in fermatean fuzzy sets has been initiated. The study has been further extended to rough fermatean fuzzy left(resp. right, interior) ideals in semigroup. Several results related to this notion are determined.

An Accelerating Reduction Approach for Incomplete Decision Table Using Positive Approximation Set

Due to the explosive growth of data collected by various sensors, it has become a difficult problem determining how to conduct feature selection more efficiently. To address this problem, we offer a fresh insight into rough set theory from the perspective of a positive approximation set. It is found that a granularity domain can be used to characterize the target knowledge, because of its form of a covering with respect to a tolerance relation. On the basis of this fact, a novel heuristic approach ARIPA is proposed to accelerate representative reduction algorithms for incomplete decision table. As a result, ARIPA in classical rough set model and ARIPA-IVPR in variable precision rough set model are realized respectively. Moreover, ARIPA is adopted to improve the computational efficiency of two existing state-of-the-art reduction algorithms. To demonstrate the effectiveness of the improved algorithms, a variety of experiments utilizing four UCI incomplete data sets are conducted. The performances of improved algorithms are compared with those of original ones as well. Numerical experiments justify that our accelerating approach enhances the existing algorithms to accomplish the reduction task more quickly. In some cases, they fulfill attribute reduction even more stably than the original algorithms do.

Gain-Loss Evaluation-Based Generic Selection for Steganalysis Feature

Fewer contribution feature components in the image high-dimensional steganalysis feature are able to increase the spatio-temporal complexity of detecting the stego images, and even reduce the detection accuracy. In order to maintain or even improve the detection accuracy while effectively reducing the dimension of the DCTR steganalysis feature, this paper proposes a new selection approach for DCTR feature. First, the asymmetric distortion factor and information gain ratio of each feature component are improved to measure the difference between the symmetric cover and stego features, which provides the theoretical basis for selecting the feature components that contribute to a great degree to detecting the stego images. Additionally, the feature components are arranged in descending order rely on the two measurement criteria, which provides the basis for deleting the components. Based on the above, removing feature components that are ranked larger differently according to two criteria. Ultimately, the preserved feature components are used as the final selected feature for training and detection. Comparison experiments with existing classical approaches indicate that this approach can effectively reduce the feature dimension while maintaining or even improving the detection accuracy. At the same time, it can reduce the detection spatio-temporal complexity of the stego images.

Reduction in a fuzzy probabilistic information system

An information system (IS) is an important mathematical tool for artificial intelligence. A fuzzy probabilistic information system (FPIS), the combination of some fuzzy relations in the same universe which satisfies the probability distribution, can be seen as an IS with fuzzy relations. A FPIS overcomes the shortcoming that rough set theory assumes elements in the universe with equal probability and leads to lose some useful information. This paper integrates the probability distribution into the fuzzy relations in a FPIS and studies its reduction. Firstly, the concept of a FPIS is introduced and its reduction is proposed. Then, the fuzzy relations in a FPIS are divided into three categories (P-necessary, P-relatively necessary and P-unnecessary fuzzy relations) according to their importance. Next, entropy measurement for a FPIS is investigated. Moreover, some reduction algorithms are constructed. Finally, an example is given to verify the effectiveness of these proposed algorithms.

Cancer Cell Intrinsic and Immunologic Phenotypes Determine Clinical Outcomes in Basal-like Breast Cancer

PURPOSE

Basal-like breast cancer (BLBC) is a particularly aggressive intrinsic molecular subtype of breast cancer that lacks targeted therapies. There is also no clinically useful test to risk stratify patients with BLBC. We hypothesized that a transcriptome-based phenotypic characterization of BLBC tumors and their microenvironments may overcome these challenges.

EXPERIMENTAL DESIGN

We conducted a retrospective correlative genomic sequencing study using a matched pairs design with validation in five independent cohorts. The study was conducted on a large population-based prospective cohort of the major molecular subtypes of breast cancer conducted in the greater Seattle-Puget Sound metropolitan area. Cases consisted of women 20-69 years of age first diagnosed with invasive breast cancer identified through the population-based Surveillance Epidemiology and End Results program. Patients for this analysis (n = 949) were identified from the 1,408 patients with stage I-III triple-negative breast cancer [estrogen receptor-negative (ER^-), progesterone receptor-negative (PR^-), HER2^-]. Of the 949 women, 248 developed a recurrence after their initial diagnosis. A matched set of 67 recurrent and nonrecurrent BLBC tumors was subjected to transcriptome sequencing. Through RNA sequencing of the matched sets of recurrent and nonrecurrent BLBC tumors, we aimed to identify prognostic phenotypes.To identify nonredundant and uncorrelated prognostic genes, we used an ensemble of variable selection algorithms, which resulted in a ranking of genes on the basis of their expected utility in classification. Using leave-one-out cross-validation, we trained a random forest classifier on the basis of the top 21 genes (BRAVO-DX). Validations were performed in five independent triple-negative or BLBC cohorts, and biomarker robustness and transferability were demonstrated by employing real-time PCR.

RESULTS

We found that cancer cell intrinsic and immunologic phenotypes are independent predictors of recurrence. By simultaneously interrogating the tumor and its microenvironment, we developed a compound risk model that stratified patients into low-, medium-, and high-risk groups, with a 14%/56%/74% chance of recurrence, respectively. Biologically, the primary tumors of patients who developed a recurrence had increased growth factor signaling and stem-like features, while nonrecurrent tumors showed high lymphocyte infiltration with clonal expansion of T and B cells, as well as antitumor polarization of macrophages. We validated our model in five independent cohorts, including three large cohorts, where BRAVO-DX was highly informative in identifying patients with disease recurrence [HR, 6.79 (95% confidence interval (CI), 1.89-24.37); HR, 3.45 (95% CI, 2.41-4.93); and HR, 1.69 (95% CI, 1.17-2.46)]. A smaller gene set focused on the tumor immunophenotype, BRAVO-IMMUNE, was highly prognostic in all five cohorts.

CONCLUSIONS

Together, these results indicate that phenotypic characteristics of BLBCs and their microenvironment are associated with recurrence-free survival and demonstrate the utility of intrinsic and extrinsic phenotypes as independent prognostic biomarkers in BLBC. Pending further evaluation and validation, our prognostic model has the potential to inform clinical decision-making for patients with BLBC as it identifies those at high risk of rapidly progressing on standard chemotherapy, as well as those who may benefit from alternative first-line therapies.

Integrated Learning: Screening Optimal Biomarkers for Identifying Preeclampsia in Placental mRNA Samples

Preeclampsia (PE) is a maternal disease that causes maternal and child death. Treatment and preventive measures are not sound enough. The problem of PE screening has attracted much attention. The purpose of this study is to screen placental mRNA to obtain the best PE biomarkers for identifying patients with PE. We use Limma in the R language to screen out the 48 differentially expressed genes with the largest differences and used correlation-based feature selection algorithms to reduce the dimensionality and avoid attribute redundancy arising from too many mRNA samples participating in the classification. After reducing the mRNA attributes, the mRNA samples are sorted from large to small according to information gain. In this study, a classifier model is designed to identify whether samples had PE through mRNA in the placenta. To improve the accuracy of classification and avoid overfitting, three classifiers, including C4.5, AdaBoost, and multilayer perceptron, are used. We use the majority voting strategy integrated with the differentially expressed genes and the genes filtered by the best subset method as comparison methods to train the classifier. The results show that the classification accuracy rate has increased from 79% to 82.2%, and the number of mRNA features has decreased from 48 to 13. This study provides clues for the main PE biomarkers of mRNA in the placenta and provides ideas for the treatment and screening of PE.

Application of Rough Ant Colony Algorithm in Adolescent Psychology

With the rapid development of big data, big data research in the security protection industry has been increasingly regarded as a hot spot. This article mainly aims at solving the problem of predicting the tendency of juvenile delinquency based on the experimental data of juvenile blindly following psychological crime. To solve this problem, this paper proposes a rough ant colony classification algorithm, referred to as RoughAC, which first uses the concept of upper and lower approximate sets in rough sets to determine the degree of membership. In addition, in the ant colony algorithm, we use the membership value to update the pheromone. Experiments show that the algorithm can not only solve the premature convergence problem caused by stagnation near the local optimal solution but also solve the continuous domain and combinatorial optimization problems and achieve better classification results. Moreover, the algorithm has a good effect on predicting classification and can provide guidance for predicting the tendency of juvenile delinquency.

Predicting Postoperative Length of Stay for Isolated Coronary Artery Bypass Graft Patients Using Machine Learning

Purpose

Predictive analytics (PA) is a new trending approach in the field of healthcare that uses machine learning to build a prediction model using supervised learning algorithms. Isolated coronary artery bypass grafting (iCABG), an open-heart surgery, is commonly performed in the treatment of coronary heart disease.

Aim

The aim of this study was to develop and evaluate a model to predict postoperative length of stay (PLoS) for iCABG patients using supervised machine learning techniques, and to identify the features with the highest contribution to the model.

Methods

This is a retrospective study that uses historic data of adult patients who underwent isolated CABG (iCABG). After initial data pre-processing, data imputation using the kNN method was applied. The study used five prediction models using Naïve Bayes, Decision Tree, Random Forest, Logistic Regression and k Nearest Neighbor algorithms. Data imbalance was managed using the following widely used methods: oversampling, undersampling, "Both", and random over-sampling examples (ROSE). The features selection process was conducted using the Boruta method. Two techniques were applied to examine the performance of the models, (70%, 30%) split and cross-validation, respectively. Models were evaluated by comparing their performance using AUC and other metrics.

Results

In the final dataset, six distinct features and 621 instances were used to develop the models. A total of 20 models were developed using R statistical software. The model generated using Random Forest with "Both" resampling method and cross-validation technique was deemed the best fit (AUC=0.81; F1 score=0.82; and recall=0.82). Attributes found to be highly predictive of PLoS were pulmonary artery systolic, age, height, EuroScore II, intra-aortic balloon pump used, and complications during operation.

Conclusion

This study demonstrates the significance and effectiveness of building a model that predicts PLoS for iCABG patients using patient specifications and pre-/intra-operative measures.

Improving Model Performance on the Stratification of Breast Cancer Patients by Integrating Multiscale Genomic Features

In clinical cancer research, it is a hot topic on how to accurately stratify patients based on genomic data. With the development of next-generation sequencing technology, more and more types of genomic features, such as mRNA expression level, can be used to distinguish cancer patients. Previous studies commonly stratified patients by using a single type of genomic features, which can only reflect one aspect of the cancer. In fact, multiscale genomic features will provide more information and may be helpful for clinical prediction. In addition, most of the conventional machine learning algorithms use a handcrafted gene set as features to construct models, which is generally selected by a statistical method with an arbitrary cut-off, e.g., p value < 0.05. The genes in the gene set are not necessarily related to the cancer and will make the model unreliable. Therefore, in our study, we thoroughly investigated the performance of different machine learning methods on stratifying breast cancer patients with a single type of genomic features. Then, we proposed a strategy, which can take into account the degree of correlation between genes and cancer patients, to identify the features from mRNAs and microRNAs, and evaluated the performance of the models with the new combined features of the multiscale genomic features. The results showed that, compared with the models constructed with a single type of features, the models with the multiscale genomic features generated by our proposed method achieved better performance on stratifying the ER status of breast cancer patients. Moreover, we found that the identified multiscale genomic features were closely related to the cancer by gene set enrichment analysis, indicating that our proposed strategy can well reflect the biological relevance of the genes to breast cancer. In conclusion, modelling with multiscale genomic features closely related to the cancer not only can guarantee the prediction performance of the models but also can effectively provide candidate genes for interpreting the mechanisms of cancer.