Machine Learning Interpretability: A Survey on Methods and Metrics

From explainable to interpretable deep learning for natural language processing in healthcare: How far from reality?

Deep learning (DL) has substantially enhanced natural language processing (NLP) in healthcare research. However, the increasing complexity of DL-based NLP necessitates transparent model interpretability, or at least explainability, for reliable decision-making. This work presents a thorough scoping review of explainable and interpretable DL in healthcare NLP. The term "eXplainable and Interpretable Artificial Intelligence" (XIAI) is introduced to distinguish XAI from IAI. Different models are further categorized based on their functionality (model-, input-, output-based) and scope (local, global). Our analysis shows that attention mechanisms are the most prevalent emerging IAI technique. The use of IAI is growing, distinguishing it from XAI. The major challenges identified are that most XIAI does not explore "global" modelling processes, the lack of best practices, and the lack of systematic evaluation and benchmarks. One important opportunity is to use attention mechanisms to enhance multi-modal XIAI for personalized medicine. Additionally, combining DL with causal logic holds promise. Our discussion encourages the integration of XIAI in Large Language Models (LLMs) and domain-specific smaller models. In conclusion, XIAI adoption in healthcare requires dedicated in-house expertise. Collaboration with domain experts, end-users, and policymakers can lead to ready-to-use XIAI methods across NLP and medical tasks. While challenges exist, XIAI techniques offer a valuable foundation for interpretable NLP algorithms in healthcare.

Exploring explainable AI features in the vocal biomarkers of lung disease

This review delves into the burgeoning field of explainable artificial intelligence (XAI) in the detection and analysis of lung diseases through vocal biomarkers. Lung diseases, often elusive in their early stages, pose a significant public health challenge. Recent advancements in AI have ushered in innovative methods for early detection, yet the black-box nature of many AI models limits their clinical applicability. XAI emerges as a pivotal tool, enhancing transparency and interpretability in AI-driven diagnostics. This review synthesizes current research on the application of XAI in analyzing vocal biomarkers for lung diseases, highlighting how these techniques elucidate the connections between specific vocal features and lung pathology. We critically examine the methodologies employed, the types of lung diseases studied, and the performance of various XAI models. The potential for XAI to aid in early detection, monitor disease progression, and personalize treatment strategies in pulmonary medicine is emphasized. Furthermore, this review identifies current challenges, including data heterogeneity and model generalizability, and proposes future directions for research. By offering a comprehensive analysis of explainable AI features in the context of lung disease detection, this review aims to bridge the gap between advanced computational approaches and clinical practice, paving the way for more transparent, reliable, and effective diagnostic tools.

Vertebral Column Pathology Diagnosis Using Ensemble Strategies Based on Supervised Machine Learning Techniques

One expanding area of bioinformatics is medical diagnosis through the categorization of biomedical characteristics. Automatic medical strategies to boost the diagnostic through machine learning (ML) methods are challenging. They require a formal examination of their performance to identify the best conditions that enhance the ML method. This work proposes variants of the Voting and Stacking (VC and SC) ensemble strategies based on diverse auto-tuning supervised machine learning techniques to increase the efficacy of traditional baseline classifiers for the automatic diagnosis of vertebral column orthopedic illnesses. The ensemble strategies are created by first combining a complete set of auto-tuned baseline classifiers based on different processes, such as geometric, probabilistic, logic, and optimization. Next, the three most promising classifiers are selected among k-Nearest Neighbors (kNN), Naïve Bayes (NB), Logistic Regression (LR), Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA), Support Vector Machine (SVM), Artificial Neural Networks (ANN), and Decision Tree (DT). The grid-search K-Fold cross-validation strategy is applied to auto-tune the baseline classifier hyperparameters. The performances of the proposed ensemble strategies are independently compared with the auto-tuned baseline classifiers. A concise analysis evaluates accuracy, precision, recall, F1-score, and ROC-ACU metrics. The analysis also examines the misclassified disease elements to find the most and least reliable classifiers for this specific medical problem. The results show that the VC ensemble strategy provides an improvement comparable to that of the best baseline classifier (the kNN). Meanwhile, when all baseline classifiers are included in the SC ensemble, this strategy surpasses 95% in all the evaluated metrics, standing out as the most suitable option for classifying vertebral column diseases.

Predicting pedalling metrics based on lower limb joint kinematics

This study aimed to predict the index of effectiveness (IE) and positive impulse proportion (PIP) to assess the cyclist's pedalling technique from lower limb kinematic variables. Several wrapped feature selection techniques were applied to select the best predictors. To predict IE and PIP two multiple linear regressions (MLR) composed of 11 predictors (R² = 0.81 ± 0.12, R² = 0.81 ± 0.05) and two artificial neural networks (ANN) composed of 21 and 28 predictors (R² = 0.95 ± 0.01, R² = 0.92 ± 0.02) were developed. The ANN predicts with accuracy, and the MLR shows the influence of each predictor.

Corn leaf disease: insightful diagnosis using VGG16 empowered by explainable AI

The agricultural sector is pivotal to food security and economic stability worldwide. Corn holds particular significance in the global food industry, especially in developing countries where agriculture is a cornerstone of the economy. However, corn crops are vulnerable to various diseases that can significantly reduce yields. Early detection and precise classification of these diseases are crucial to prevent damage and ensure high crop productivity. This study leverages the VGG16 deep learning (DL) model to classify corn leaves into four categories: healthy, blight, gray spot, and common rust. Despite the efficacy of DL models, they often face challenges related to the explainability of their decision-making processes. To address this, Layer-wise Relevance Propagation (LRP) is employed to enhance the model's transparency by generating intuitive and human-readable heat maps of input images. The proposed VGG16 model, augmented with LRP, outperformed previous state-of-the-art models in classifying corn leaf diseases. Simulation results demonstrated that the model not only achieved high accuracy but also provided interpretable results, highlighting critical regions in the images used for classification. By generating human-readable explanations, this approach ensures greater transparency and reliability in model performance, aiding farmers in improving their crop yields.

A Data-Driven Approach to Predicting Recreational Activity Participation Using Machine Learning

Purpose: With the popularity of recreational activities, the study aimed to develop prediction models for recreational activity participation and explore the key factors affecting participation in recreational activities. Methods: A total of 12,712 participants, excluding individuals under 20, were selected from the National Health and Nutrition Examination Survey (NHANES) from 2011 to 2018. The mean age of the sample was 46.86 years (±16.97), with a gender distribution of 6,721 males and 5,991 females. The variables included demographic, physical-related variables, and lifestyle variables. This study developed 42 prediction models using six machine learning methods, including logistic regression, Support Vector Machine (SVM), decision tree, random forest, eXtreme Gradient Boosting (XGBoost), and Light Gradient Boosting Machine (LightGBM). The relative importance of each variable was evaluated by permutation feature importance. Results: The results illustrated that the LightGBM was the most effective algorithm for predicting recreational activity participation (accuracy: .838, precision: .783, recall: .967, F1-score: .865, AUC: .826). In particular, prediction performance increased when the demographic and lifestyle datasets were used together. Next, as the result of the permutation feature importance based on the top models, education level and moderate-vigorous physical activity (MVPA) were found to be essential variables. Conclusion: These findings demonstrated the potential of a data-driven approach utilizing machine learning in a recreational discipline. Furthermore, this study interpreted the prediction model through feature importance analysis to overcome the limitation of machine learning interpretability.

How scanning probe microscopy can be supported by artificial intelligence and quantum computing?

The impact of Artificial Intelligence (AI) is rapidly expanding, revolutionizing both science and society. It is applied to practically all areas of life, science, and technology, including materials science, which continuously requires novel tools for effective materials characterization. One of the widely used techniques is scanning probe microscopy (SPM). SPM has fundamentally changed materials engineering, biology, and chemistry by providing tools for atomic-precision surface mapping. Despite its many advantages, it also has some drawbacks, such as long scanning times or the possibility of damaging soft-surface materials. In this paper, we focus on the potential for supporting SPM-based measurements, with an emphasis on the application of AI-based algorithms, especially Machine Learning-based algorithms, as well as quantum computing (QC). It has been found that AI can be helpful in automating experimental processes in routine operations, algorithmically searching for optimal sample regions, and elucidating structure-property relationships. Thus, it contributes to increasing the efficiency and accuracy of optical nanoscopy scanning probes. Moreover, the combination of AI-based algorithms and QC may have enormous potential to enhance the practical application of SPM. The limitations of the AI-QC-based approach were also discussed. Finally, we outline a research path for improving AI-QC-powered SPM. RESEARCH HIGHLIGHTS: Artificial intelligence and quantum computing as support for scanning probe microscopy. The analysis indicates a research gap in the field of scanning probe microscopy. The research aims to shed light into ai-qc-powered scanning probe microscopy.

Machine learning models' assessment: trust and performance

The common black box nature of machine learning models is an obstacle to their application in health care context. Their widespread application is limited by a significant "lack of trust." So, the main goal of this work is the development of an evaluation approach that can assess, simultaneously, trust and performance. Trust assessment is based on (i) model robustness (stability assessment), (ii) confidence (95% CI of geometric mean), and (iii) interpretability (comparison of respective features ranking with clinical evidence). Performance is assessed through geometric mean. For validation, in patients' stratification in cardiovascular risk assessment, a Portuguese dataset (N=1544) was applied. Five different models were compared: (i) GRACE score, the most common risk assessment tool in Portugal for patients with acute coronary syndrome; (ii) logistic regression; (iii) Naïve Bayes; (iv) decision trees; and (v) rule-based approach, previously developed by this team. The obtained results confirm that the simultaneous assessment of trust and performance can be successfully implemented. The rule-based approach seems to have potential for clinical application. It provides a high level of trust in the respective operation while outperformed the GRACE model's performance, enhancing the required physicians' acceptance. This may increase the possibility to effectively aid the clinical decision.

Ineffective Learning Behaviors and Their Psychological Mechanisms among Adolescents in Online Learning: A Narrative Review

During the COVID-19 pandemic, many countries and regions experienced a surge in online learning, but the public complained about and questioned its effectiveness. One of the most important reasons for this was the inadequate metacognitive abilities of adolescents. Studies in learning sciences have identified various inefficient learning behaviors among students in online learning, including help abuse, help avoidance, and wheel spinning; all closely related to metacognition. Despite concerns about ecological validity, researchers in psychology have proposed the agenda-based regulation framework, the COPES model, and MAPS model, which may help explain the inefficient learning behaviors among adolescents in online learning. Future studies should aim to verify these theoretical frameworks within the context of online learning and elucidate the causes of inefficient learning behaviors; the design and optimization of online learning systems should be informed by theories in cognitive psychology.

Identifying the risk of depression in a large sample of adolescents: An artificial neural network based on random forest

BACKGROUND

This study aims to develop an artificial neural network (ANN) prediction model incorporating random forest (RF) screening ability for predicting the risk of depression in adolescents and identifies key risk factors to provide a new approach for primary care screening of depression among adolescents.

METHODS

The data were from a large cross-sectional study conducted in China from July to September 2021, enrolling 8635 adolescents aged 10-17 with their parents. We used the Patient health questionnaire (PHQ-9) to rate adolescent depression symptoms, using scales and single-item questions to collect demographic information and other variables. Initial model variables screening used the RF importance assessment, followed by building prediction model using the screened variables through the ANN.

RESULTS

The rate of depression symptoms in adolescents was 24.6%, and the depression risk prediction model was built based on 70% of the training set and 30% of the test set. Ten variables were included in the final prediction model with a model accuracy of 85.03%, AUC of 0.892, specificity of 89.79%, and sensitivity of 70.81%. The top 10 significant factors of depression risk were adolescent rumination, adolescent self-esteem, adolescent mobile phone addiction, peer victimization, care in parenting styles, overprotection in parenting styles, academic pressure, conflict in parent-child relationship, parental rumination, and relationship between parents.

CONCLUSIONS

The ANN model based on the RF effectively identifies depression risk in adolescents and provides a methodological reference for large-scale primary screening. Cross-sectional studies and single-item scales limit further improvements in model accuracy.

Linking repeated subjective judgments and ConvNets for multimodal assessment of the immediate living environment

The integration of alternative data extraction approaches for multimodal data, can significantly reduce modeling difficulties for the automatic location assessment. We develop a method for assessing the quality of the immediate living environment by incorporating human judgments as ground truth into a neural network for generating new synthetic data and testing the effects in surrogate hedonic models. We expect that the quality of the data will be less biased if the annotation is performed by multiple independent persons applying repeated trials which should reduce the overall error variance and lead to more robust results. Experimental results show that linking repeated subjective judgements and Deep Learning can reliably determine the quality scores and thus expand the range of information for the quality assessment. The presented method is not computationally intensive, can be performed repetitively and can also be easily adapted to machine learning approaches in a broader sense or be transferred to other use cases. Following aspects are essential for the implementation of the method:•Sufficient amount of representative data for human assessment.•Repeated assessment trials by individuals.•Confident derivation of the effect of human judgments on property price as an approbation for further generation of synthetic data.

Brain tumor detection using proper orthogonal decomposition integrated with deep learning networks

BACKGROUND AND OBJECTIVE

The central organ of the human nervous system is the brain, which receives and sends stimuli to the various parts of the body to engage in daily activities. Uncontrolled growth of brain cells can result in tumors which affect the normal functions of healthy brain cells. An automatic reliable technique for detecting tumors is imperative to assist medical practitioners in the timely diagnosis of patients. Although machine learning models are being used, with minimal data availability to train, development of low-order based models integrated with machine learning are a tool for reliable detection.

METHODS

In this study, we focus on comparing a low-order model such as proper orthogonal decomposition (POD) coupled with convolutional neural network (CNN) on 2D images from magnetic resonance imaging (MRI) scans to effectively identify brain tumors. The explainability of the coupled POD-CNN prediction output as well as the state-of-the-art pre-trained transfer learning models such as MobileNetV2, Inception-v3, ResNet101, and VGG-19 were explored.

RESULTS

The results showed that CNN predicted tumors with an accuracy of 99.21% whereas POD-CNN performed better with about 1/3rd of computational time at an accuracy of 95.88%. Explainable AI with SHAP showed MobileNetV2 has better prediction in identifying the tumor boundaries.

CONCLUSIONS

Integration of POD with CNN is carried for the first time to detect brain tumor detection with minimal MRI scan data. This study facilitates low-model approaches in machine learning to improve the accuracy and performance of tumor detection.

Metastable grain boundaries: the roles of structural and chemical disorders in their energetics, non-equilibrium kinetic evolution, and mechanical behaviors

Complex environments in advanced manufacturing usually involve ultrafast laser or ion irradiation which leads to rapid heating and cooling and drives grain boundaries (GBs) to non-equilibrium states, featuring distinct energetics and kinetic behaviors compared to conventional equilibrium or near-equilibrium GBs. In this topical review, we provide an overview of both recent experimental and computational studies on metastable GBs, i.e. their energetics, kinetic behaviors, and mechanical properties. In contrast to GBs at thermodynamic equilibrium, the inherent structure energy of metastable GBs exhibits a spectrum instead of single value for a particular misorientation, due to the existence of microstructural and chemical disorder. The potential energy landscape governs the energetic and kinetic behaviors of metastable GBs, including the ageing/rejuvenating mechanism and activation barrier distributions. The unique energetics and structural disorder of metastable GBs lead to unique mechanical properties and tunability of interface-rich nanocrystalline materials. We also discuss that, in addition to structural disorder, chemical complexity in multi-components alloys could also drive the GBs away from their ground states and, subsequently, significantly impact on the GBs-mediated deformation. And under some extreme conditions such as irradiation, structural disorders and chemical complexity may simultaneously present at interfaces, further enriching of metastability of GBs and their physical and mechanical behaviors. Finally, we discuss the machine learning techniques, which have been increasingly employed to predict and understand the complex behaviors of metastable GBs in recent years. We highlight the potential of data-driven approaches to revolutionize the study of disorder systems by efficiently extracting the relationship between structural features and material properties. We hope this topical review paper could shed light and stimulate the development of new GBs engineering strategies that allow more flexibility and tunability for the design of nano-structured materials.

Human-annotated rationales and explainable text classification: a survey

Asking annotators to explain "why" they labeled an instance yields annotator rationales: natural language explanations that provide reasons for classifications. In this work, we survey the collection and use of annotator rationales. Human-annotated rationales can improve data quality and form a valuable resource for improving machine learning models. Moreover, human-annotated rationales can inspire the construction and evaluation of model-annotated rationales, which can play an important role in explainable artificial intelligence.

Novel embeddings improve the prediction of risk perception

We assess whether the classic psychometric paradigm of risk perception can be improved or supplanted by novel approaches relying on language embeddings. To this end, we introduce the Basel Risk Norms, a large data set covering 1004 distinct sources of risk (e.g., vaccination, nuclear energy, artificial intelligence) and compare the psychometric paradigm against novel text and free-association embeddings in predicting risk perception. We find that an ensemble model combining text and free association rivals the predictive accuracy of the psychometric paradigm, captures additional affect and frequency-related dimensions of risk perception not accounted for by the classic approach, and has greater range of applicability to real-world text data, such as news headlines. Overall, our results establish the ensemble of text and free-association embeddings as a promising new tool for researchers and policymakers to track real-world risk perception.

Supplementary Information

The online version contains supplementary material available at 10.1140/epjds/s13688-024-00478-x.

Cherry on Top or Real Need? A Review of Explainable Machine Learning in Kidney Transplantation

Research on solid organ transplantation has taken advantage of the substantial acquisition of medical data and the use of artificial intelligence (AI) and machine learning (ML) to answer diagnostic, prognostic, and therapeutic questions for many years. Nevertheless, despite the question of whether AI models add value to traditional modeling approaches, such as regression models, their "black box" nature is one of the factors that have hindered the translation from research to clinical practice. Several techniques that make such models understandable to humans were developed with the promise of increasing transparency in the support of medical decision-making. These techniques should help AI to close the gap between theory and practice by yielding trust in the model by doctors and patients, allowing model auditing, and facilitating compliance with emergent AI regulations. But is this also happening in the field of kidney transplantation? This review reports the use and explanation of "black box" models to diagnose and predict kidney allograft rejection, delayed graft function, graft failure, and other related outcomes after kidney transplantation. In particular, we emphasize the discussion on the need (or not) to explain ML models for biological discovery and clinical implementation in kidney transplantation. We also discuss promising future research paths for these computational tools.

Mpox-AISM: AI-mediated super monitoring for mpox and like-mpox

Swift and accurate diagnosis for earlier-stage monkeypox (mpox) patients is crucial to avoiding its spread. However, the similarities between common skin disorders and mpox and the need for professional diagnosis unavoidably impaired the diagnosis of earlier-stage mpox patients and contributed to mpox outbreak. To address the challenge, we proposed "Super Monitoring", a real-time visualization technique employing artificial intelligence (AI) and Internet technology to diagnose earlier-stage mpox cheaply, conveniently, and quickly. Concretely, AI-mediated "Super Monitoring" (mpox-AISM) integrates deep learning models, data augmentation, self-supervised learning, and cloud services. According to publicly accessible datasets, mpox-AISM's Precision, Recall, Specificity, and F1-score in diagnosing mpox reach 99.3%, 94.1%, 99.9%, and 96.6%, respectively, and it achieves 94.51% accuracy in diagnosing mpox, six like-mpox skin disorders, and normal skin. With the Internet and communication terminal, mpox-AISM has the potential to perform real-time and accurate diagnosis for earlier-stage mpox in real-world scenarios, thereby preventing mpox outbreak.

Potential suicide risk among the college student population: machine learning approaches for identifying predictors and different students' risk profiles

BACKGROUND

Suicide is one of the leading causes of death among young people and university students. Research has identified numerous socio-demographic, relational, and clinical factors as potential predictors of suicide risk, and machine learning techniques have emerged as promising ways to improve risk assessment.

OBJECTIVE

This cross-sectional observational study aimed at identifying predictors and college student profiles associated with suicide risk through a machine learning approach.

METHODS

A total of 3102 students were surveyed regarding potential suicide risk, socio-demographic characteristics, academic career, and physical/mental health and well-being. The classification tree technique and the multiple correspondence analysis were applied to define students' profiles in terms of suicide risk and to detect the main predictors of such a risk.

RESULTS

Among the participating students, 7% showed high potential suicide risk and 3.8% had a history of suicide attempts. Psychological distress and use of alcohol/substance were prominent predictors of suicide risk contributing to define the profile of high risk of suicide: students with significant psychological distress, and with medium/high-risk use of alcohol and psychoactive substances. Conversely, low psychological distress and low-risk use of alcohol and substances, together with religious practice, represented the profile of students with low risk of suicide.

CONCLUSIONS

Machine learning techniques could hold promise for assessing suicide risk in college students, potentially leading to the development of more effective prevention programs. These programs should address both risk and protective factors and be tailored to students' needs and to the different categories of risk.

Toward Fairness, Accountability, Transparency, and Ethics in AI for Social Media and Health Care: Scoping Review

BACKGROUND

The use of social media for disseminating health care information has become increasingly prevalent, making the expanding role of artificial intelligence (AI) and machine learning in this process both significant and inevitable. This development raises numerous ethical concerns. This study explored the ethical use of AI and machine learning in the context of health care information on social media platforms (SMPs). It critically examined these technologies from the perspectives of fairness, accountability, transparency, and ethics (FATE), emphasizing computational and methodological approaches that ensure their responsible application.

OBJECTIVE

This study aims to identify, compare, and synthesize existing solutions that address the components of FATE in AI applications in health care on SMPs. Through an in-depth exploration of computational methods, approaches, and evaluation metrics used in various initiatives, we sought to elucidate the current state of the art and identify existing gaps. Furthermore, we assessed the strength of the evidence supporting each identified solution and discussed the implications of our findings for future research and practice. In doing so, we made a unique contribution to the field by highlighting areas that require further exploration and innovation.

METHODS

Our research methodology involved a comprehensive literature search across PubMed, Web of Science, and Google Scholar. We used strategic searches through specific filters to identify relevant research papers published since 2012 focusing on the intersection and union of different literature sets. The inclusion criteria were centered on studies that primarily addressed FATE in health care discussions on SMPs; those presenting empirical results; and those covering definitions, computational methods, approaches, and evaluation metrics.

RESULTS

Our findings present a nuanced breakdown of the FATE principles, aligning them where applicable with the American Medical Informatics Association ethical guidelines. By dividing these principles into dedicated sections, we detailed specific computational methods and conceptual approaches tailored to enforcing FATE in AI-driven health care on SMPs. This segmentation facilitated a deeper understanding of the intricate relationship among the FATE principles and highlighted the practical challenges encountered in their application. It underscored the pioneering contributions of our study to the discourse on ethical AI in health care on SMPs, emphasizing the complex interplay and the limitations faced in implementing these principles effectively.

CONCLUSIONS

Despite the existence of diverse approaches and metrics to address FATE issues in AI for health care on SMPs, challenges persist. The application of these approaches often intersects with additional ethical considerations, occasionally leading to conflicts. Our review highlights the lack of a unified, comprehensive solution for fully and effectively integrating FATE principles in this domain. This gap necessitates careful consideration of the ethical trade-offs involved in deploying existing methods and underscores the need for ongoing research.

A new interpretable streamflow prediction approach based on SWAT-BiLSTM and SHAP

Streamflow is a crucial variable for assessing the available water resources for both human and environmental use. Accurate streamflow prediction plays a significant role in water resource management and assessing the impacts of climate change. This study explores the potential of coupling conceptual hydrological models based on physical processes with machine learning algorithms to enhance the performance of streamflow simulations. Four coupled models, namely SWAT-Transformer, SWAT-LSTM, SWAT-GRU, and SWAT-BiLSTM, were constructed in this research. SWAT served as a transfer function to convert four meteorological features, including precipitation, temperature, relative humidity, and wind speed, into six hydrological features: soil water content, lateral flow, percolation, groundwater discharge, surface runoff, and evapotranspiration. Machine learning algorithms were employed to capture the underlying relationships between these ten feature variables and the target variable (streamflow) to predict daily streamflow in the Sandu-River Basin (SRB). Among the four coupled models and the calibrated SWAT model, SWAT-BiLSTM exhibited the best streamflow simulation performance. During the calibration period (training period), it achieved R2 and NSE values of 0.92 and 0.91, respectively, and maintained them at 0.90 during the validation period (testing period). Additionally, the performance of all four coupled models surpassed that of the calibrated SWAT model. Compared to the tendency of the SWAT model to underestimate streamflow, the absolute values of PBIAS for all coupled models are below 10%, which indicates that there is no significant systematic bias evident. SHapley Additive exPlanations (SHAP) were used to analyze the impact of different feature variables on streamflow prediction. The results indicated that precipitation contributed the most to streamflow prediction, with a global importance of 29.7%. Hydrological feature variable output by the SWAT model played a dominant role in the Bi-LSTM's prediction process. Coupling conceptual hydrological models with machine learning algorithms can significantly enhance the predictive performance of streamflow. The application of SHAP improves the interpretability of the coupled models and enhances researchers' confidence in the prediction results.

An Investigation of Efficiency Issues in a Low-Pressure Steam Turbine Using Neural Modelling

This study utilizes neural networks to detect and locate thermal anomalies in low-pressure steam turbines, some of which experienced a drop in efficiency. Standard approaches relying on expert knowledge or statistical methods struggled to identify the anomalous steam line due to difficulty in capturing nonlinear and weak relations in the presence of linear and strong ones. In this research, some inputs that linearly relate to outputs have been intentionally neglected. The remaining inputs have been used to train shallow feedforward or long short-term memory neural networks using measured data. The resulting models have been analyzed by Shapley additive explanations, which can determine the impact of individual inputs or model features on outputs. This analysis identified unexpected relations between lines that should not be connected. Subsequently, during periodic plant shutdown, a leak was discovered in the indicated line.

Applying machine learning to anaerobic fermentation of waste sludge using two targeted modeling strategies

Anaerobic fermentation is an effective method to harvest volatile fatty acids (VFAs) from waste activated sludge (WAS). Accurately predicting and optimizing VFAs production is crucial for anaerobic fermentation engineering. In this study, we developed machine learning models using two innovative strategies to precisely predict the daily yield of VFAs in a laboratory anaerobic fermenter. Strategy-1 focuses on model interpretability to comprehend the influence of variables of interest on VFAs production, while Strategy-2 takes into account the cost of variable acquisition, making it more suitable for practical applications in prediction and optimization. The results showed that Support Vector Regression emerged as the most effective model in this study, with testing R2 values of 0.949 and 0.939 for the two strategies, respectively. We conducted feature importance analysis to identify the critical factors that influence VFAs production. Detailed explanations were provided using partial dependence plots and Shepley Additive Explanations analyses. To optimize VFAs production, we integrated the developed model with optimization algorithms, resulting in a maximum yield of 2997.282 mg/L. This value was 45.2 % higher than the average VFAs level in the operated fermenter. Our study offers valuable insights for predicting and optimizing VFAs production in sludge anaerobic fermentation, and it facilitates engineering practice in VFAs harvesting from WAS.

A multi-objective stacked regression method for distance based colour measuring device

Identifying colour from a distance is challenging due to the external noise associated with the measurement process. The present study focuses on developing a colour measuring system and a novel Multi-target Regression (MTR) model for accurate colour measurement from distance. Herein, a novel MTR method, referred as Multi-Objective Stacked Regression (MOSR) is proposed. The core idea behind MOSR is based on stacking as an ensemble approach with multi-objective evolutionary learning using NSGA-II. A multi-objective optimization approach is used for selecting base learners that maximises prediction accuracy while minimising ensemble complexity, which is further compared with six state-of-the-art methods over the colour dataset. Classification and regression tree (CART), Random Forest (RF) and Support Vector Machine (SVM) were used as regressor algorithms. MOSR outperformed all compared methods with the highest coefficient of determination values for all three targets of the colour dataset. Rigorous comparison with state-of-the-art methods over 18 benchmarked datasets showed MOSR outperformed in 15 datasets when CART was used as a regressor algorithm and 11 datasets when RF and SVM were used as regressor algorithms. The MOSR method was statistically superior to compared methods and can be effectively used to measure accurate colour values in the distance-based colour measuring device.

Interpretability of bi-level variable selection methods

Variable selection is usually performed to increase interpretability, as sparser models are easier to understand than full models. However, a focus on sparsity is not always suitable, for example, when features are related due to contextual similarities or high correlations. Here, it may be more appropriate to identify groups and their predictive members, a task that can be accomplished with bi-level selection procedures. To investigate whether such techniques lead to increased interpretability, group exponential LASSO (GEL), sparse group LASSO (SGL), composite minimax concave penalty (cMCP), and least absolute shrinkage, and selection operator (LASSO) as reference methods were used to select predictors in time-to-event, regression, and classification tasks in bootstrap samples from a cohort of 1001 patients. Different groupings based on prior knowledge, correlation structure, and random assignment were compared in terms of selection relevance, group consistency, and collinearity tolerance. The results show that bi-level selection methods are superior to LASSO in all criteria. The cMCP demonstrated superiority in selection relevance, while SGL was convincing in group consistency. An all-round capacity was achieved by GEL: the approach jointly selected correlated and content-related predictors while maintaining high selection relevance. This method seems recommendable when variables are grouped, and interpretation is of primary interest.

Hyperparameter Optimization with Genetic Algorithms and XGBoost: A Step Forward in Smart Grid Fraud Detection

This study provides a comprehensive analysis of the combination of Genetic Algorithms (GA) and XGBoost, a well-known machine-learning model. The primary emphasis lies in hyperparameter optimization for fraud detection in smart grid applications. The empirical findings demonstrate a noteworthy enhancement in the model's performance metrics following optimization, particularly emphasizing a substantial increase in accuracy from 0.82 to 0.978. The precision, recall, and AUROC metrics demonstrate a clear improvement, indicating the effectiveness of optimizing the XGBoost model for fraud detection. The findings from our study significantly contribute to the expanding field of smart grid fraud detection. These results emphasize the potential uses of advanced metaheuristic algorithms to optimize complex machine-learning models. This work showcases significant progress in enhancing the accuracy and efficiency of fraud detection systems in smart grids.

Enhancing heart failure treatment decisions: interpretable machine learning models for advanced therapy eligibility prediction using EHR data

Timely and accurate referral of end-stage heart failure patients for advanced therapies, including heart transplants and mechanical circulatory support, plays an important role in improving patient outcomes and saving costs. However, the decision-making process is complex, nuanced, and time-consuming, requiring cardiologists with specialized expertise and training in heart failure and transplantation. In this study, we propose two logistic tensor regression-based models to predict patients with heart failure warranting evaluation for advanced heart failure therapies using irregularly spaced sequential electronic health records at the population and individual levels. The clinical features were collected at the previous visit and the predictions were made at the very beginning of the subsequent visit. Patient-wise ten-fold cross-validation experiments were performed. Standard LTR achieved an average F1 score of 0.708, AUC of 0.903, and AUPRC of 0.836. Personalized LTR obtained an F1 score of 0.670, an AUC of 0.869 and an AUPRC of 0.839. The two models not only outperformed all other machine learning models to which they were compared but also improved the performance and robustness of the other models via weight transfer. The AUPRC scores of support vector machine, random forest, and Naive Bayes are improved by 8.87%, 7.24%, and 11.38%, respectively. The two models can evaluate the importance of clinical features associated with advanced therapy referral. The five most important medical codes, including chronic kidney disease, hypotension, pulmonary heart disease, mitral regurgitation, and atherosclerotic heart disease, were reviewed and validated with literature and by heart failure cardiologists. Our proposed models effectively utilize EHRs for potential advanced therapies necessity in heart failure patients while explaining the importance of comorbidities and other clinical events. The information learned from trained model training could offer further insight into risk factors contributing to the progression of heart failure at both the population and individual levels.

Sequential stacking link prediction algorithms for temporal networks

Link prediction algorithms are indispensable tools in many scientific applications by speeding up network data collection and imputing missing connections. However, in many systems, links change over time and it remains unclear how to optimally exploit such temporal information for link predictions in such networks. Here, we show that many temporal topological features, in addition to having high computational cost, are less accurate in temporal link prediction than sequentially stacked static network features. This sequential stacking link prediction method uses 41 static network features that avoid detailed feature engineering choices and is capable of learning a highly accurate predictive distribution of future connections from historical data. We demonstrate that this algorithm works well for both partially observed and completely unobserved target layers, and on two temporal stochastic block models achieves near-oracle-level performance when combined with other single predictor methods as an ensemble learning method. Finally, we empirically illustrate that stacking multiple predictive methods together further improves performance on 19 real-world temporal networks from different domains.

Explainable machine learning for breast cancer diagnosis from mammography and ultrasound images: a systematic review

BACKGROUND

Breast cancer is the most common disease in women. Recently, explainable artificial intelligence (XAI) approaches have been dedicated to investigate breast cancer. An overwhelming study has been done on XAI for breast cancer. Therefore, this study aims to review an XAI for breast cancer diagnosis from mammography and ultrasound (US) images. We investigated how XAI methods for breast cancer diagnosis have been evaluated, the existing ethical challenges, research gaps, the XAI used and the relation between the accuracy and explainability of algorithms.

METHODS

In this work, Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist and diagram were used. Peer-reviewed articles and conference proceedings from PubMed, IEEE Explore, ScienceDirect, Scopus and Google Scholar databases were searched. There is no stated date limit to filter the papers. The papers were searched on 19 September 2023, using various combinations of the search terms 'breast cancer', 'explainable', 'interpretable', 'machine learning', 'artificial intelligence' and 'XAI'. Rayyan online platform detected duplicates, inclusion and exclusion of papers.

RESULTS

This study identified 14 primary studies employing XAI for breast cancer diagnosis from mammography and US images. Out of the selected 14 studies, only 1 research evaluated humans' confidence in using the XAI system-additionally, 92.86% of identified papers identified dataset and dataset-related issues as research gaps and future direction. The result showed that further research and evaluation are needed to determine the most effective XAI method for breast cancer.

CONCLUSION

XAI is not conceded to increase users' and doctors' trust in the system. For the real-world application, effective and systematic evaluation of its trustworthiness in this scenario is lacking.

PROSPERO REGISTRATION NUMBER

CRD42023458665.

Exploring Convolutional Neural Network Architectures for EEG Feature Extraction

The main purpose of this paper is to provide information on how to create a convolutional neural network (CNN) for extracting features from EEG signals. Our task was to understand the primary aspects of creating and fine-tuning CNNs for various application scenarios. We considered the characteristics of EEG signals, coupled with an exploration of various signal processing and data preparation techniques. These techniques include noise reduction, filtering, encoding, decoding, and dimension reduction, among others. In addition, we conduct an in-depth analysis of well-known CNN architectures, categorizing them into four distinct groups: standard implementation, recurrent convolutional, decoder architecture, and combined architecture. This paper further offers a comprehensive evaluation of these architectures, covering accuracy metrics, hyperparameters, and an appendix that contains a table outlining the parameters of commonly used CNN architectures for feature extraction from EEG signals.

Interpretable deep learning for deconvolutional analysis of neural signals

The widespread adoption of deep learning to build models that capture the dynamics of neural populations is typically based on "black-box" approaches that lack an interpretable link between neural activity and function. Here, we propose to apply algorithm unrolling, a method for interpretable deep learning, to design the architecture of sparse deconvolutional neural networks and obtain a direct interpretation of network weights in relation to stimulus-driven single-neuron activity through a generative model. We characterize our method, referred to as deconvolutional unrolled neural learning (DUNL), and show its versatility by applying it to deconvolve single-trial local signals across multiple brain areas and recording modalities. To exemplify use cases of our decomposition method, we uncover multiplexed salience and reward prediction error signals from midbrain dopamine neurons in an unbiased manner, perform simultaneous event detection and characterization in somatosensory thalamus recordings, and characterize the responses of neurons in the piriform cortex. Our work leverages the advances in interpretable deep learning to gain a mechanistic understanding of neural dynamics.

Interpretable Diabetic Retinopathy Diagnosis Based on Biomarker Activation Map

OBJECTIVE

Deep learning classifiers provide the most accurate means of automatically diagnosing diabetic retinopathy (DR) based on optical coherence tomography (OCT) and its angiography (OCTA). The power of these models is attributable in part to the inclusion of hidden layers that provide the complexity required to achieve a desired task. However, hidden layers also render algorithm outputs difficult to interpret. Here we introduce a novel biomarker activation map (BAM) framework based on generative adversarial learning that allows clinicians to verify and understand classifiers' decision-making.

METHODS

A data set including 456 macular scans were graded as non-referable or referable DR based on current clinical standards. A DR classifier that was used to evaluate our BAM was first trained based on this data set. The BAM generation framework was designed by combing two U-shaped generators to provide meaningful interpretability to this classifier. The main generator was trained to take referable scans as input and produce an output that would be classified by the classifier as non-referable. The BAM is then constructed as the difference image between the output and input of the main generator. To ensure that the BAM only highlights classifier-utilized biomarkers an assistant generator was trained to do the opposite, producing scans that would be classified as referable by the classifier from non-referable scans.

RESULTS

The generated BAMs highlighted known pathologic features including nonperfusion area and retinal fluid.

CONCLUSION/SIGNIFICANCE

A fully interpretable classifier based on these highlights could help clinicians better utilize and verify automated DR diagnosis.

Synthetic Data Improve Survival Status Prediction Models in Early-Onset Colorectal Cancer

PURPOSE

In artificial intelligence-based modeling, working with a limited number of patient groups is challenging. This retrospective study aimed to evaluate whether applying synthetic data generation methods to the clinical data of small patient groups can enhance the performance of prediction models.

MATERIALS AND METHODS

A data set collected by the Cancer Registry Library Project from the Yonsei Cancer Center (YCC), Severance Hospital, between January 2008 and October 2020 was reviewed. Patients with colorectal cancer younger than 50 years who started their initial treatment at YCC were included. A Bayesian network-based synthesizing model was used to generate a synthetic data set, combined with the differential privacy (DP) method.

RESULTS

A synthetic population of 5,005 was generated from a data set of 1,253 patients with 93 clinical features. The Hellinger distance and correlation difference metric were below 0.3 and 0.5, respectively, indicating no statistical difference. The overall survival by disease stage did not differ between the synthetic and original populations. Training with the synthetic data and validating with the original data showed the highest performances of 0.850, 0.836, and 0.790 for the Decision Tree, Random Forest, and XGBoost models, respectively. Comparison of synthetic data sets with different epsilon parameters from the original data sets showed improved performance >0.1%. For extremely small data sets, models using synthetic data outperformed those using only original data sets. The reidentification risk measures demonstrated that the epsilons between 0.1 and 100 fell below the baseline, indicating a preserved privacy state.

CONCLUSION

The synthetic data generation approach enhances predictive modeling performance by maintaining statistical and clinical integrity, and simultaneously reduces privacy risks through the application of DP techniques.

CVII: Enhancing Interpretability in Intelligent Sensor Systems via Computer Vision Interpretability Index

In the realm of intelligent sensor systems, the dependence on Artificial Intelligence (AI) applications has heightened the importance of interpretability. This is particularly critical for opaque models such as Deep Neural Networks (DNN), as understanding their decisions is essential, not only for ethical and regulatory compliance, but also for fostering trust in AI-driven outcomes. This paper introduces the novel concept of a Computer Vision Interpretability Index (CVII). The CVII framework is designed to emulate human cognitive processes, specifically in tasks related to vision. It addresses the intricate challenge of quantifying interpretability, a task that is inherently subjective and varies across domains. The CVII is rigorously evaluated using a range of computer vision models applied to the COCO (Common Objects in Context) dataset, a widely recognized benchmark in the field. The findings established a robust correlation between image interpretability, model selection, and CVII scores. This research makes a substantial contribution to enhancing interpretability for human comprehension, as well as within intelligent sensor applications. By promoting transparency and reliability in AI-driven decision-making, the CVII framework empowers its stakeholders to effectively harness the full potential of AI technologies.

Systematic review: fluid biomarkers and machine learning methods to improve the diagnosis from mild cognitive impairment to Alzheimer's disease

Mild cognitive impairment (MCI) is often considered an early stage of dementia, with estimated rates of progression to dementia up to 80-90% after approximately 6 years from the initial diagnosis. Diagnosis of cognitive impairment in dementia is typically based on clinical evaluation, neuropsychological assessments, cerebrospinal fluid (CSF) biomarkers, and neuroimaging. The main goal of diagnosing MCI is to determine its cause, particularly whether it is due to Alzheimer's disease (AD). However, only a limited percentage of the population has access to etiological confirmation, which has led to the emergence of peripheral fluid biomarkers as a diagnostic tool for dementias, including MCI due to AD. Recent advances in biofluid assays have enabled the use of sophisticated statistical models and multimodal machine learning (ML) algorithms for the diagnosis of MCI based on fluid biomarkers from CSF, peripheral blood, and saliva, among others. This approach has shown promise for identifying specific causes of MCI, including AD. After a PRISMA analysis, 29 articles revealed a trend towards using multimodal algorithms that incorporate additional biomarkers such as neuroimaging, neuropsychological tests, and genetic information. Particularly, neuroimaging is commonly used in conjunction with fluid biomarkers for both cross-sectional and longitudinal studies. Our systematic review suggests that cost-effective longitudinal multimodal monitoring data, representative of diverse cultural populations and utilizing white-box ML algorithms, could be a valuable contribution to the development of diagnostic models for AD due to MCI. Clinical assessment and biomarkers, together with ML techniques, could prove pivotal in improving diagnostic tools for MCI due to AD.

Artificial intelligence (AI) and machine learning (ML) in precision oncology: a review on enhancing discoverability through multiomics integration

Multiomics data including imaging radiomics and various types of molecular biomarkers have been increasingly investigated for better diagnosis and therapy in the era of precision oncology. Artificial intelligence (AI) including machine learning (ML) and deep learning (DL) techniques combined with the exponential growth of multiomics data may have great potential to revolutionize cancer subtyping, risk stratification, prognostication, prediction and clinical decision-making. In this article, we first present different categories of multiomics data and their roles in diagnosis and therapy. Second, AI-based data fusion methods and modeling methods as well as different validation schemes are illustrated. Third, the applications and examples of multiomics research in oncology are demonstrated. Finally, the challenges regarding the heterogeneity data set, availability of omics data, and validation of the research are discussed. The transition of multiomics research to real clinics still requires consistent efforts in standardizing omics data collection and analysis, building computational infrastructure for data sharing and storing, developing advanced methods to improve data fusion and interpretability, and ultimately, conducting large-scale prospective clinical trials to fill the gap between study findings and clinical benefits.

Frugal day-ahead forecasting of multiple local electricity loads by aggregating adaptive models

This paper focuses on day-ahead electricity load forecasting for substations of the distribution network in France; therefore, the corresponding problem lies between the instability of a single consumption and the stability of a countrywide total demand. Moreover, this problem requires to forecast the loads of over one thousand substations; consequently, it belongs to the field of multiple time series forecasting. To that end, the paper applies an adaptive methodology that provided excellent results at a national scale; the idea is to combine generalized additive models with state-space representations. However, extending this methodology to the prediction of over a thousand time series raises a computational issue. It is solved by developing a frugal variant that reduces the number of estimated parameters: forecasting models are estimated only for a few time series and transfer learning is achieved by relying on aggregation of experts. This approach yields a reduction of computational needs and their associated emissions. Several variants are built, corresponding to different levels of parameter transfer, to find the best trade-off between accuracy and frugality. The selected method achieves competitive results compared to individual models. Finally, the paper highlights the interpretability of the models, which is important for operational applications.

An interpretable machine learning framework for diagnosis and prognosis of COVID-19

Coronaviruses have affected the lives of people around the world. Increasingly, studies have indicated that the virus is mutating and becoming more contagious. Hence, the pressing priority is to swiftly and accurately predict patient outcomes. In addition, physicians and patients increasingly need interpretability when building machine models in healthcare. We propose an interpretable machine framework(KISM) that can diagnose and prognose patients based on blood test datasets. First, we use k-nearest neighbors, isolated forests, and SMOTE to pre-process the original blood test datasets. Seven machine learning tools Support Vector Machine, Extra Tree, Random Forest, Gradient Boosting Decision Tree, eXtreme Gradient Boosting, Logistic Regression, and ensemble learning were then used to diagnose and predict COVID-19. In addition, we used SHAP and scikit-learn post-hoc interpretability to report feature importance, allowing healthcare professionals and artificial intelligence models to interact to suggest biomarkers that some doctors may have missed. The 10-fold cross-validation of two public datasets shows that the performance of KISM is better than that of the current state-of-the-art methods. In the diagnostic COVID-19 task, an AUC value of 0.9869 and an accuracy of 0.9787 were obtained, and ultimately Leukocytes, platelets, and Proteina C reativa mg/dL were found to be the most indicative biomarkers for the diagnosis of COVID-19. An AUC value of 0.9949 and an accuracy of 0.9677 were obtained in the prognostic COVID-19 task and Age, LYMPH, and WBC were found to be the most indicative biomarkers for identifying the severity of the patient.

ArtEMon: Artificial Intelligence and Internet of Things Powered Greenhouse Gas Sensing for Real-Time Emissions Monitoring

Greenhouse gas (GHG) emissions reporting and sustainability are increasingly important for businesses around the world. Yet the lack of a single standardised method of measurement, when coupled with an inability to understand the true state of emissions in complex logistics activities, presents enormous barriers for businesses to understanding the extent of their emissions footprint. One of the traditional approaches to accurately capturing and monitoring gas emissions in logistics is through using gas sensors. However, connecting, maintaining, and operating gas sensors on moving vehicles in different road and weather conditions is a large and costly challenge. This paper presents the development and evaluation of a reliable and accurate sensing technique for GHG emissions collection (or monitoring) in real-time, employing the Internet of Things (IoT) and Artificial Intelligence (AI) to eliminate or reduce the usage of gas sensors, using reliable and cost-effective solutions.

Explainability agreement between dermatologists and five visual explanations techniques in deep neural networks for melanoma AI classification

Introduction

The use of deep convolutional neural networks for analyzing skin lesion images has shown promising results. The identification of skin cancer by faster and less expensive means can lead to an early diagnosis, saving lives and avoiding treatment costs. However, to implement this technology in a clinical context, it is important for specialists to understand why a certain model makes a prediction; it must be explainable. Explainability techniques can be used to highlight the patterns of interest for a prediction.

Methods

Our goal was to test five different techniques: Grad-CAM, Grad-CAM++, Score-CAM, Eigen-CAM, and LIME, to analyze the agreement rate between features highlighted by the visual explanation maps to 3 important clinical criteria for melanoma classification: asymmetry, border irregularity, and color heterogeneity (ABC rule) in 100 melanoma images. Two dermatologists scored the visual maps and the clinical images using a semi-quantitative scale, and the results were compared. They also ranked their preferable techniques.

Results

We found that the techniques had different agreement rates and acceptance. In the overall analysis, Grad-CAM showed the best total+partial agreement rate (93.6%), followed by LIME (89.8%), Grad-CAM++ (88.0%), Eigen-CAM (86.4%), and Score-CAM (84.6%). Dermatologists ranked their favorite options: Grad-CAM and Grad-CAM++, followed by Score-CAM, LIME, and Eigen-CAM.

Discussion

Saliency maps are one of the few methods that can be used for visual explanations. The evaluation of explainability with humans is ideal to assess the understanding and applicability of these methods. Our results demonstrated that there is a significant agreement between clinical features used by dermatologists to diagnose melanomas and visual explanation techniques, especially Grad-Cam.

Interpreting the Mechanism of Synergism for Drug Combinations Using Attention-Based Hierarchical Graph Pooling

Synergistic drug combinations provide huge potentials to enhance therapeutic efficacy and to reduce adverse reactions. However, effective and synergistic drug combination prediction remains an open question because of the unknown causal disease signaling pathways. Though various deep learning (AI) models have been proposed to quantitatively predict the synergism of drug combinations, the major limitation of existing deep learning methods is that they are inherently not interpretable, which makes the conclusions of AI models untransparent to human experts, henceforth limiting the robustness of the model conclusion and the implementation ability of these models in real-world human-AI healthcare. In this paper, we develop an interpretable graph neural network (GNN) that reveals the underlying essential therapeutic targets and the mechanism of the synergy (MoS) by mining the sub-molecular network of great importance. The key point of the interpretable GNN prediction model is a novel graph pooling layer, a self-attention-based node and edge pool (henceforth SANEpool), that can compute the attention score (importance) of genes and connections based on the genomic features and topology. As such, the proposed GNN model provides a systematic way to predict and interpret the drug combination synergism based on the detected crucial sub-molecular network. Experiments on various well-adopted drug-synergy-prediction datasets demonstrate that (1) the SANEpool model has superior predictive ability to generate accurate synergy score prediction, and (2) the sub-molecular networks detected by the SANEpool are self-explainable and salient for identifying synergistic drug combinations.

Characterization of Synthetic Health Data Using Rule-Based Artificial Intelligence Models

The aim of this study is to apply and characterize eXplainable AI (XAI) to assess the quality of synthetic health data generated using a data augmentation algorithm. In this exploratory study, several synthetic datasets are generated using various configurations of a conditional Generative Adversarial Network (GAN) from a set of 156 observations related to adult hearing screening. A rule-based native XAI algorithm, the Logic Learning Machine, is used in combination with conventional utility metrics. The classification performance in different conditions is assessed: models trained and tested on synthetic data, models trained on synthetic data and tested on real data, and models trained on real data and tested on synthetic data. The rules extracted from real and synthetic data are then compared using a rule similarity metric. The results indicate that XAI may be used to assess the quality of synthetic data by (i) the analysis of classification performance and (ii) the analysis of the rules extracted on real and synthetic data (number, covering, structure, cut-off values, and similarity). These results suggest that XAI can be used in an original way to assess synthetic health data and extract knowledge about the mechanisms underlying the generated data.

Improvement of a prediction model for heart failure survival through explainable artificial intelligence

Cardiovascular diseases and their associated disorder of heart failure (HF) are major causes of death globally, making it a priority for doctors to detect and predict their onset and medical consequences. Artificial Intelligence (AI) allows doctors to discover clinical indicators and enhance their diagnoses and treatments. Specifically, "eXplainable AI" (XAI) offers tools to improve the clinical prediction models that experience poor interpretability of their results. This work presents an explainability analysis and evaluation of two HF survival prediction models using a dataset that includes 299 patients who have experienced HF. The first model utilizes survival analysis, considering death events and time as target features, while the second model approaches the problem as a classification task to predict death. The model employs an optimization data workflow pipeline capable of selecting the best machine learning algorithm as well as the optimal collection of features. Moreover, different post hoc techniques have been used for the explainability analysis of the model. The main contribution of this paper is an explainability-driven approach to select the best HF survival prediction model balancing prediction performance and explainability. Therefore, the most balanced explainable prediction models are Survival Gradient Boosting model for the survival analysis and Random Forest for the classification approach with a c-index of 0.714 and balanced accuracy of 0.74 (std 0.03) respectively. The selection of features by the SCI-XAI in the two models is similar where "serum_creatinine", "ejection_fraction", and "sex" are selected in both approaches, with the addition of "diabetes" for the survival analysis model. Moreover, the application of post hoc XAI techniques also confirm common findings from both approaches by placing the "serum_creatinine" as the most relevant feature for the predicted outcome, followed by "ejection_fraction". The explainable prediction models for HF survival presented in this paper would improve the further adoption of clinical prediction models by providing doctors with insights to better understand the reasoning behind usually "black-box" AI clinical solutions and make more reasonable and data-driven decisions.

Optimization of Arrhythmia-based ECG-lead Selection for Computer-interpreted Heart Rhythm Classification

The 12-lead ECG only has 8 independent ECG leads, which leads to diagnostic redundancy when using all 12 leads for heart arrhythmias classification. We have previously developed a deep learning (DL)-based computer-interpreted ECG (CIE) approach to identify an optimal 4-lead ECG subset for classifying heart arrhythmias. However, the clinical diagnostic criteria of cardiac arrhythmia types are often lead-specific, so this study is going to explore the selection of arrhythmia-based ECG-lead subsets rather than one general optimal ECG-lead subset, which could improve the classification performance for the CIE. The DL-based CIE model previously developed was used to learn 4 common types of heart arrhythmias (LBBB, RBBB, AF, and I-AVB) for identifying corresponding optimal ECG-lead subsets. A public dataset that splits into training (approx. 70%), validation (approx. 15%), and test (approx. 15%) sets from the PhysioNet Cardiology Challenge 2020 was used to explore the study. The results demonstrated that the DL-based CIE model identified an optimal ECG-lead subset for each arrhythmia: I, II, aVR, aVL, V1, V3, and V5 for I-AVB; I, II, aVR, and V3 for AF; I, II, aVR, aVF, V1, V3, and V4 for LBBB; and I, II, III, aVR, V1, V4, and V6 for RBBB. For each arrhythmia classification, the DL-based CIE model using the optimal ECG-lead subset significantly outperformed the model using the full 12-lead ECG set on the validation set and on the external test dataset.The results support the hypothesis that using an optimal ECG-lead subset instead of the full 12-lead ECG set can improve the classification performance of a specific arrhythmia when using the DL-based CIE approach.Clinical Relevance- Using an arrhythmia-based optimal ECG-lead subset, the classification performance of a deep-learning-based model can be achieved without loss of accuracy in comparison with the full 12-lead set (p<0.05).

Combining Machine Learning with Physical Knowledge in Thermodynamic Modeling of Fluid Mixtures

Thermophysical properties of fluid mixtures are important in many fields of science and engineering. However, experimental data are scarce in this field, so prediction methods are vital. Different types of physical prediction methods are available, ranging from molecular models over equations of state to models of excess properties. These well-established methods are currently being complemented by new methods from the field of machine learning (ML). This review focuses on the rapidly developing interface between these two approaches and gives a structured overview of how physical modeling and ML can be combined to yield hybrid models. We illustrate the different options with examples from recent research and give an outlook on future developments.

Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review

This paper provides a comprehensive review of the literature concerning the utilization of Natural Language Processing (NLP) techniques, with a particular focus on transformer-based large language models (LLMs) trained using Big Code, within the domain of AI-assisted programming tasks. LLMs, augmented with software naturalness, have played a crucial role in facilitating AI-assisted programming applications, including code generation, code completion, code translation, code refinement, code summarization, defect detection, and clone detection. Notable examples of such applications include the GitHub Copilot powered by OpenAI's Codex and DeepMind AlphaCode. This paper presents an overview of the major LLMs and their applications in downstream tasks related to AI-assisted programming. Furthermore, it explores the challenges and opportunities associated with incorporating NLP techniques with software naturalness in these applications, with a discussion on extending AI-assisted programming capabilities to Apple's Xcode for mobile software development. This paper also presents the challenges of and opportunities for incorporating NLP techniques with software naturalness, empowering developers with advanced coding assistance and streamlining the software development process.

WARNING: A Wearable Inertial-Based Sensor Integrated with a Support Vector Machine Algorithm for the Identification of Faults during Race Walking

Due to subjectivity in refereeing, the results of race walking are often questioned. To overcome this limitation, artificial-intelligence-based technologies have demonstrated their potential. The paper aims at presenting WARNING, an inertial-based wearable sensor integrated with a support vector machine algorithm to automatically identify race-walking faults. Two WARNING sensors were used to gather the 3D linear acceleration related to the shanks of ten expert race-walkers. Participants were asked to perform a race circuit following three race-walking conditions: legal, illegal with loss-of-contact and illegal with knee-bent. Thirteen machine learning algorithms, belonging to the decision tree, support vector machine and k-nearest neighbor categories, were evaluated. An inter-athlete training procedure was applied. Algorithm performance was evaluated in terms of overall accuracy, F1 score and G-index, as well as by computing the prediction speed. The quadratic support vector was confirmed to be the best-performing classifier, achieving an accuracy above 90% with a prediction speed of 29,000 observations/s when considering data from both shanks. A significant reduction of the performance was assessed when considering only one lower limb side. The outcomes allow us to affirm the potential of WARNING to be used as a referee assistant in race-walking competitions and during training sessions.

Machine Learning Techniques for Developing Remotely Monitored Central Nervous System Biomarkers Using Wearable Sensors: A Narrative Literature Review

BACKGROUND

Central nervous system (CNS) disorders benefit from ongoing monitoring to assess disease progression and treatment efficacy. Mobile health (mHealth) technologies offer a means for the remote and continuous symptom monitoring of patients. Machine Learning (ML) techniques can process and engineer mHealth data into a precise and multidimensional biomarker of disease activity.

OBJECTIVE

This narrative literature review aims to provide an overview of the current landscape of biomarker development using mHealth technologies and ML. Additionally, it proposes recommendations to ensure the accuracy, reliability, and interpretability of these biomarkers.

METHODS

This review extracted relevant publications from databases such as PubMed, IEEE, and CTTI. The ML methods employed across the selected publications were then extracted, aggregated, and reviewed.

RESULTS

This review synthesized and presented the diverse approaches of 66 publications that address creating mHealth-based biomarkers using ML. The reviewed publications provide a foundation for effective biomarker development and offer recommendations for creating representative, reproducible, and interpretable biomarkers for future clinical trials.

CONCLUSION

mHealth-based and ML-derived biomarkers have great potential for the remote monitoring of CNS disorders. However, further research and standardization of study designs are needed to advance this field. With continued innovation, mHealth-based biomarkers hold promise for improving the monitoring of CNS disorders.

Time series big data: a survey on data stream frameworks, analysis and algorithms

Big data has a substantial role nowadays, and its importance has significantly increased over the last decade. Big data's biggest advantages are providing knowledge, supporting the decision-making process, and improving the use of resources, services, and infrastructures. The potential of big data increases when we apply it in real-time by providing real-time analysis, predictions, and forecasts, among many other applications. Our goal with this article is to provide a viewpoint on how to build a system capable of processing big data in real-time, performing analysis, and applying algorithms. A system should be designed to handle vast amounts of data and provide valuable knowledge through analysis and algorithms. This article explores the current approaches and how they can be used for the real-time operations and predictions.

Development of prediction models for one-year brain tumour survival using machine learning: a comparison of accuracy and interpretability

BACKGROUND AND OBJECTIVE

Prediction of survival in patients diagnosed with a brain tumour is challenging because of heterogeneous tumour behaviours and treatment response. Advances in machine learning have led to the development of clinical prognostic models, but due to the lack of model interpretability, integration into clinical practice is almost non-existent. In this retrospective study, we compare five classification models with varying degrees of interpretability for the prediction of brain tumour survival greater than one year following diagnosis.

METHODS

1028 patients aged ≥16 years with a brain tumour diagnosis between April 2012 and April 2020 were included in our study. Three intrinsically interpretable 'glass box' classifiers (Bayesian Rule Lists [BRL], Explainable Boosting Machine [EBM], and Logistic Regression [LR]), and two 'black box' classifiers (Random Forest [RF] and Support Vector Machine [SVM]) were trained on electronic patients records for the prediction of one-year survival. All models were evaluated using balanced accuracy (BAC), F1-score, sensitivity, specificity, and receiver operating characteristics. Black box model interpretability and misclassified predictions were quantified using SHapley Additive exPlanations (SHAP) values and model feature importance was evaluated by clinical experts.

RESULTS

The RF model achieved the highest BAC of 78.9%, closely followed by SVM (77.7%), LR (77.5%) and EBM (77.1%). Across all models, age, diagnosis (tumour type), functional features, and first treatment were top contributors to the prediction of one year survival. We used EBM and SHAP to explain model misclassifications and investigated the role of feature interactions in prognosis.

CONCLUSION

Interpretable models are a natural choice for the domain of predictive medicine. Intrinsically interpretable models, such as EBMs, may provide an advantage over traditional clinical assessment of brain tumour prognosis by weighting potential risk factors and their interactions that may be unknown to clinicians. An agreement between model predictions and clinical knowledge is essential for establishing trust in the models decision making process, as well as trust that the model will make accurate predictions when applied to new data.

An Efficient Iterative Approach to Explainable Feature Learning

This article introduces a new iterative approach to explainable feature learning. During each iteration, new features are generated, first by applying arithmetic operations on the input set of features. These are then evaluated in terms of probability distribution agreements between values of samples belonging to different classes. Finally, a graph-based approach for feature selection is proposed, which allows for selecting high-quality and uncorrelated features to be used in feature generation during the next iteration. As shown by the results, the proposed method improved the accuracy of all tested classifiers, where the best accuracies were achieved using random forest. In addition, the method turned out to be insensitive to both of the input parameters, while superior performances in comparison to the state of the art were demonstrated on nine out of 15 test sets and achieving comparable results in the others. Finally, we demonstrate the explainability of the learned feature representation for knowledge discovery.