Enhancing the effectiveness and interpretability of decision tree and rule induction classifiers with evolutionary training set selection over imbalanced problems

Using machine learning algorithm to analyse the hypothyroidism complications caused by radiotherapy in patients with head and neck cancer

Machine learning algorithms were used to analyze the odds and predictors of complications of thyroid damage after radiation therapy in patients with head and neck cancer. This study used decision tree (DT), random forest (RF), and support vector machine (SVM) algorithms to evaluate predictors for the data of 137 head and neck cancer patients. Candidate factors included gender, age, thyroid volume, minimum dose, average dose, maximum dose, number of treatments, and relative volume of the organ receiving X dose (X: 10, 20, 30, 40, 50, 60 Gy). The algorithm was optimized according to these factors and tenfold cross-validation to analyze the state of thyroid damage and select the predictors of thyroid dysfunction. The importance of the predictors identified by the three machine learning algorithms was ranked: the top five predictors were age, thyroid volume, average dose, V50 and V60. Of these, age and volume were negatively correlated with thyroid damage, indicating that the greater the age and thyroid volume, the lower the risk of thyroid damage; the average dose, V50 and V60 were positively correlated with thyroid damage, indicating that the larger the average dose, V50 and V60, the higher the risk of thyroid damage. The RF algorithm was most accurate in predicting the probability of thyroid damage among the three algorithms optimized using the above factors. The Area under the receiver operating characteristic curve (AUC) was 0.827 and the accuracy (ACC) was 0.824. This study found that five predictors (age, thyroid volume, mean dose, V50 and V60) are important factors affecting the chance that patients with head and neck cancer who received radiation therapy will develop hypothyroidism. Using these factors as the prediction basis of the algorithm and using RF to predict the occurrence of hypothyroidism had the highest ACC, which was 82.4%. This algorithm is quite helpful in predicting the probability of radiotherapy complications. It also provides references for assisting medical decision-making in the future.

A Pareto-based Ensemble with Feature and Instance Selection for Learning from Multi-Class Imbalanced Datasets

Imbalanced classification is related to those problems that have an uneven distribution among classes. In addition to the former, when instances are located into the overlapped areas, the correct modeling of the problem becomes harder. Current solutions for both issues are often focused on the binary case study, as multi-class datasets require an additional effort to be addressed. In this research, we overcome these problems by carrying out a combination between feature and instance selections. Feature selection will allow simplifying the overlapping areas easing the generation of rules to distinguish among the classes. Selection of instances from all classes will address the imbalance itself by finding the most appropriate class distribution for the learning task, as well as possibly removing noise and difficult borderline examples. For the sake of obtaining an optimal joint set of features and instances, we embedded the searching for both parameters in a Multi-Objective Evolutionary Algorithm, using the C4.5 decision tree as baseline classifier in this wrapper approach. The multi-objective scheme allows taking a double advantage: the search space becomes broader, and we may provide a set of different solutions in order to build an ensemble of classifiers. This proposal has been contrasted versus several state-of-the-art solutions on imbalanced classification showing excellent results in both binary and multi-class problems.

Evolutionary fuzzy ARTMAP neural networks for classification of semiconductor defects

Wafer defect detection using an intelligent system is an approach of quality improvement in semiconductor manufacturing that aims to enhance its process stability, increase production capacity, and improve yields. Occasionally, only few records that indicate defective units are available and they are classified as a minority group in a large database. Such a situation leads to an imbalanced data set problem, wherein it engenders a great challenge to deal with by applying machine-learning techniques for obtaining effective solution. In addition, the database may comprise overlapping samples of different classes. This paper introduces two models of evolutionary fuzzy ARTMAP (FAM) neural networks to deal with the imbalanced data set problems in a semiconductor manufacturing operations. In particular, both the FAM models and hybrid genetic algorithms are integrated in the proposed evolutionary artificial neural networks (EANNs) to classify an imbalanced data set. In addition, one of the proposed EANNs incorporates a facility to learn overlapping samples of different classes from the imbalanced data environment. The classification results of the proposed evolutionary FAM neural networks are presented, compared, and analyzed using several classification metrics. The outcomes positively indicate the effectiveness of the proposed networks in handling classification problems with imbalanced data sets.

Knowledge Discovery in a Community Data Set: Malnutrition among the Elderly

Objectives

The purpose of this study was to design a prediction model that explains the characteristics of elderly adults at risk of malnutrition.

Methods

Data were obtained from a large data set, 2008 Korean Elderly Survey, in which the data of 15,146 subjects were entered. With nutritional status a target variable, the input variables included the demographic and socioeconomic status of participants. The data were analyzed by using the SPSS Clementine 12.0 program's feature selection node to select meaningful variables.

Results

Among the C5.0, C&R Tree, QUEST, and CHAID models, the highest predictability was reported by C&R Tree with the accuracy rate of 77.1%. The presence of more than two comorbidities, living alone status, having severe difficulty in daily activities, and lower perceived economic status were identified as risk factors of malnutrition in elderly.

Conclusions

A reliable decision support model was designed to provide accurate information regarding the characteristics of elderly individuals with malnutrition. The findings demonstrated the good feasibility of data mining when used for a large community data set and its value in assisting health professionals and local decision makers to come up with effective strategies for achieving public health goals.

A scalable memetic algorithm for simultaneous instance and feature selection

Instance selection is becoming increasingly relevant due to the huge amount of data that is constantly produced in many fields of research. At the same time, most of the recent pattern recognition problems involve highly complex datasets with a large number of possible explanatory variables. For many reasons, this abundance of variables significantly harms classification or recognition tasks. There are efficiency issues, too, because the speed of many classification algorithms is largely improved when the complexity of the data is reduced. One of the approaches to address problems that have too many features or instances is feature or instance selection, respectively. Although most methods address instance and feature selection separately, both problems are interwoven, and benefits are expected from facing these two tasks jointly. This paper proposes a new memetic algorithm for dealing with many instances and many features simultaneously by performing joint instance and feature selection. The proposed method performs four different local search procedures with the aim of obtaining the most relevant subsets of instances and features to perform an accurate classification. A new fitness function is also proposed that enforces instance selection but avoids putting too much pressure on removing features. We prove experimentally that this fitness function improves the results in terms of testing error. Regarding the scalability of the method, an extension of the stratification approach is developed for simultaneous instance and feature selection. This extension allows the application of the proposed algorithm to large datasets. An extensive comparison using 55 medium to large datasets from the UCI Machine Learning Repository shows the usefulness of our method. Additionally, the method is applied to 30 large problems, with very good results. The accuracy of the method for class-imbalanced problems in a set of 40 datasets is shown. The usefulness of the method is also tested using decision trees and support vector machines as classification methods.

OligoIS: Scalable Instance Selection for Class-Imbalanced Data Sets

In current research, an enormous amount of information is constantly being produced, which poses a challenge for data mining algorithms. Many of the problems in extremely active research areas, such as bioinformatics, security and intrusion detection, or text mining, share the following two features: large data sets and class-imbalanced distribution of samples. Although many methods have been proposed for dealing with class-imbalanced data sets, most of these methods are not scalable to the very large data sets common to those research fields. In this paper, we propose a new approach to dealing with the class-imbalance problem that is scalable to data sets with many millions of instances and hundreds of features. This proposal is based on the divide-and-conquer principle combined with application of the selection process to balanced subsets of the whole data set. This divide-and-conquer principle allows the execution of the algorithm in linear time. Furthermore, the proposed method is easy to implement using a parallel environment and can work without loading the whole data set into memory. Using 40 class-imbalanced medium-sized data sets, we will demonstrate our method's ability to improve the results of state-of-the-art instance selection methods for class-imbalanced data sets. Using three very large data sets, we will show the scalability of our proposal to millions of instances and hundreds of features.

A hybrid decision support model to discover informative knowledge in diagnosing acute appendicitis

Background

The aim of this study is to develop a simple and reliable hybrid decision support model by combining statistical analysis and decision tree algorithms to ensure high accuracy of early diagnosis in patients with suspected acute appendicitis and to identify useful decision rules.

Methods

We enrolled 326 patients who attended an emergency medical center complaining mainly of acute abdominal pain. Statistical analysis approaches were used as a feature selection process in the design of decision support models, including the Chi-square test, Fisher's exact test, the Mann-Whitney U-test (p < 0.01), and Wald forward logistic regression (entry and removal criteria of 0.01 and 0.05, or 0.05 and 0.10, respectively). The final decision support models were constructed using the C5.0 decision tree algorithm of Clementine 12.0 after pre-processing.

Results

Of 55 variables, two subsets were found to be indispensable for early diagnostic knowledge discovery in acute appendicitis. The two subsets were as follows: (1) lymphocytes, urine glucose, total bilirubin, total amylase, chloride, red blood cell, neutrophils, eosinophils, white blood cell, complaints, basophils, glucose, monocytes, activated partial thromboplastin time, urine ketone, and direct bilirubin in the univariate analysis-based model; and (2) neutrophils, complaints, total bilirubin, urine glucose, and lipase in the multivariate analysis-based model. The experimental results showed that the model with univariate analysis (80.2%, 82.4%, 78.3%, 76.8%, 83.5%, and 80.3%) outperformed models using multivariate analysis (71.6%, 69.3%, 73.7%, 69.7%, 73.3%, and 71.5% with entry and removal criteria of 0.01 and 0.05; 73.5%, 66.0%, 80.0%, 74.3%, 72.9%, and 73.0% with entry and removal criteria of 0.05 and 0.10) in terms of accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and area under ROC curve, during a 10-fold cross validation. A statistically significant difference was detected in the pairwise comparison of ROC curves (p < 0.01, 95% CI, 3.13-14.5; p < 0.05, 95% CI, 1.54-13.1). The larger induced decision model was more effective for identifying acute appendicitis in patients with acute abdominal pain, whereas the smaller induced decision tree was less accurate with the test data.

Conclusions

The decision model developed in this study can be applied as an aid in the initial decision making of clinicians to increase vigilance in cases of suspected acute appendicitis.