From Clustering to Cluster Explanations via Neural Networks

Deep Neural Networks and Tabular Data: A Survey

Heterogeneous tabular data are the most commonly used form of data and are essential for numerous critical and computationally demanding applications. On homogeneous datasets, deep neural networks have repeatedly shown excellent performance and have therefore been widely adopted. However, their adaptation to tabular data for inference or data generation tasks remains highly challenging. To facilitate further progress in the field, this work provides an overview of state-of-the-art deep learning methods for tabular data. We categorize these methods into three groups: data transformations, specialized architectures, and regularization models. For each of these groups, our work offers a comprehensive overview of the main approaches. Moreover, we discuss deep learning approaches for generating tabular data and also provide an overview over strategies for explaining deep models on tabular data. Thus, our first contribution is to address the main research streams and existing methodologies in the mentioned areas while highlighting relevant challenges and open research questions. Our second contribution is to provide an empirical comparison of traditional machine learning methods with 11 deep learning approaches across five popular real-world tabular datasets of different sizes and with different learning objectives. Our results, which we have made publicly available as competitive benchmarks, indicate that algorithms based on gradient-boosted tree ensembles still mostly outperform deep learning models on supervised learning tasks, suggesting that the research progress on competitive deep learning models for tabular data is stagnating. To the best of our knowledge, this is the first in-depth overview of deep learning approaches for tabular data; as such, this work can serve as a valuable starting point to guide researchers and practitioners interested in deep learning with tabular data.

Using of artificial neural networks and different evolutionary algorithms to predict the viscosity and thermal conductivity of silica-alumina-MWCN/water nanofluid

This study predicts the parameters such as viscosity and thermal conductivity in silica-alumina-MWCN/water nanofluid using the artificial intelligence method and using design variables such as solid volume fraction and temperature. In this study, 6 optimization algorithms were used to predict and numerically model the μnf and TC of silica-alumina-MWCNT/water-NF. In this study, six measurement criteria were used to evaluate the estimates obtained from the coupling process of GMDH ANN with each of these 6 optimization algorithms. The results reveal that the influence of the φ is notably higher on both μnf and TC with values of 0.83 for μnf and 0.92 for TC, while Temp has a relatively weaker impact with -0.5 for μnf and 0.38 for TC. Among various algorithms, the coupling of the evolutionary algorithm NSGA II with ANN and GMDH performs best in predicting μnf and TC for the NF, with a maximum margin of deviation of -0.108 and an R2 evaluation criterion of 0.99996 for μnf and 1 for TC, indicating exceptional model accuracy. In the subsequent phase, a meta-heuristic Genetic Algorithm minimizes μnf and TC values. Four points (A, B, C, and D) along the Pareto front are selected, with point A representing the optimal state characterized by low values of φ and Temp (0.0002 and 50.8772, respectively) and corresponding target function values of 0.9988 for μnf and 0.6344 for TC. In contrast, point D represents the highest values of φ and Temp (0.49986 and 59.9775, respectively) and yields target function values of 2.382 for μnf and 0.8517 for TC. This analysis aids in identifying the optimal operating conditions for maximizing NF performance.

Explanation of clustering result based on multi-objective optimization

Clustering is an unsupervised machine learning technique whose goal is to cluster unlabeled data. But traditional clustering methods only output a set of results and do not provide any explanations of the results. Although in the literature a number of methods based on decision tree have been proposed to explain the clustering results, most of them have some disadvantages, such as too many branches and too deep leaves, which lead to complex explanations and make it difficult for users to understand. In this paper, a hypercube overlay model based on multi-objective optimization is proposed to achieve succinct explanations of clustering results. The model designs two objective functions based on the number of hypercubes and the compactness of instances and then uses multi-objective optimization to find a set of nondominated solutions. Finally, an Utopia point is defined to determine the most suitable solution, in which each cluster can be covered by as few hypercubes as possible. Based on these hypercubes, an explanations of each cluster is provided. Upon verification on synthetic and real datasets respectively, it shows that the model can provide a concise and understandable explanations to users.

Predictive Modeling and Integrated Risk Assessment of Postoperative Mortality and Pneumonia in Traumatic Brain Injury Patients through Clustering and Machine Learning: Retrospective Study

This study harnessed machine learning to forecast postoperative mortality (POM) and postoperative pneumonia (PPN) among surgical traumatic brain injury (TBI) patients. Our analysis centered on the following key variables: Glasgow Coma Scale (GCS), midline brain shift (MSB), and time from injury to emergency room arrival (TIE). Additionally, we introduced innovative clustered variables to enhance predictive accuracy and risk assessment. Exploring data from 617 patients spanning 2012 to 2022, we observed that 22.9% encountered postoperative mortality, while 30.0% faced postoperative pneumonia (PPN). Sensitivity for POM and PPN prediction, before incorporating clustering, was in the ranges of 0.43-0.82 (POM) and 0.54-0.76 (PPN). Following clustering, sensitivity values were 0.47-0.76 (POM) and 0.61-0.77 (PPN). Accuracy was in the ranges of 0.67-0.76 (POM) and 0.70-0.81 (PPN) prior to clustering and 0.42-0.73 (POM) and 0.55-0.73 (PPN) after clustering. Clusters characterized by low GCS, small MSB, and short TIE exhibited a 3.2-fold higher POM risk compared to clusters with high GCS, small MSB, and short TIE. In summary, leveraging clustered variables offers a novel avenue for predicting POM and PPN in TBI patients. Assessing the amalgamated impact of GCS, MSB, and TIE characteristics provides valuable insights for clinical decision making.