Explainable artificial intelligence (XAI) in radiology and nuclear medicine: a literature review

Enhancing Deep Learning Model Explainability in Brain Tumor Datasets Using Post-Heuristic Approaches

The application of deep learning models in medical diagnosis has showcased considerable efficacy in recent years. Nevertheless, a notable limitation involves the inherent lack of explainability during decision-making processes. This study addresses such a constraint by enhancing the interpretability robustness. The primary focus is directed towards refining the explanations generated by the LIME Library and LIME image explainer. This is achieved through post-processing mechanisms based on scenario-specific rules. Multiple experiments have been conducted using publicly accessible datasets related to brain tumor detection. Our proposed post-heuristic approach demonstrates significant advancements, yielding more robust and concrete results in the context of medical diagnosis.

Developing an ensemble machine learning study: Insights from a multi-center proof-of-concept study

BACKGROUND

To address the numerous unmeet clinical needs, in recent years several Machine Learning models applied to medical images and clinical data have been introduced and developed. Even when they achieve encouraging results, they lack evolutionary progression, thus perpetuating their status as autonomous entities. We postulated that different algorithms which have been proposed in the literature to address the same diagnostic task, can be aggregated to enhance classification performance. We suggested a proof of concept to define an ensemble approach useful for integrating different algorithms proposed to solve the same clinical task.

METHODS

The proposed approach was developed starting from a public database consisting of radiomic features extracted from CT images relating to 535 patients suffering from lung cancer. Seven algorithms were trained independently by participants in the AI4MP working group on Artificial Intelligence of the Italian Association of Physics in Medicine to discriminate metastatic from non-metastatic patients. The classification scores generated by these algorithms are used to train SVM classifier. The Explainable Artificial Intelligence approach is applied to the final model. The ensemble model was validated following an 80-20 hold-out and leave-one-out scheme on the training set.

RESULTS

Compared to individual algorithms, a more accurate result was achieved. On the independent test the ensemble model achieved an accuracy of 0.78, a F1-score of 0.57 and a log-loss of 0.49. Shapley values representing the contribution of each algorithm to the final classification result of the ensemble model were calculated. This information represents an added value for the end user useful for evaluating the appropriateness of the classification result on a particular case. It also allows us to evaluate on a global level which methodological approaches of the individual algorithms are likely to have the most impact.

CONCLUSION

Our proposal represents an innovative approach useful for integrating different algorithms that populate the literature and which lays the foundations for future evaluations in broader application scenarios.

Machine learning for (non-)epileptic tissue detection from the intraoperative electrocorticogram

OBJECTIVE

Clinical visual intraoperative electrocorticography (ioECoG) reading intends to localize epileptic tissue and improve epilepsy surgery outcome. We aimed to understand whether machine learning (ML) could complement ioECoG reading, how subgroups affected performance, and which ioECoG features were most important.

METHODS

We included 91 ioECoG-guided epilepsy surgery patients with Engel 1A outcome. We allocated 71 training and 20 test set patients. We trained an extra trees classifier (ETC) with 14 spectral features to classify ioECoG channels as covering resected or non-resected tissue. We compared the ETC's performance with clinical ioECoG reading and assessed whether patient subgroups affected performance. Explainable artificial intelligence (xAI) unveiled the most important ioECoG features learnt by the ETC.

RESULTS

The ETC outperformed clinical reading in five test set patients, was inferior in six, and both were inconclusive in nine. The ETC performed best in the tumor subgroup (area under ROC curve: 0.84 [95%CI 0.79-0.89]). xAI revealed predictors of resected (relative theta, alpha, and fast ripple power) and non-resected tissue (relative beta and gamma power).

CONCLUSIONS

Combinations of subtle spectral ioECoG changes, imperceptible by the human eye, can aid healthy and pathological tissue discrimination.

SIGNIFICANCE

ML with spectral ioECoG features can support, rather than replace, clinical ioECoG reading, particularly in tumors.

Multi-Quantifying Maxillofacial Traits via a Demographic Parity-Based AI Model

Objective and Impact Statement: The multi-quantification of the distinct individualized maxillofacial traits, that is, quantifying multiple indices, is vital for diagnosis, decision-making, and prognosis of the maxillofacial surgery. Introduction: While the discrete and demographically disproportionate distributions of the multiple indices restrict the generalization ability of artificial intelligence (AI)-based automatic analysis, this study presents a demographic-parity strategy for AI-based multi-quantification. Methods: In the aesthetic-concerning maxillary alveolar basal bone, which requires quantifying a total of 9 indices from length and width dimensional, this study collected a total of 4,000 cone-beam computed tomography (CBCT) sagittal images, and developed a deep learning model composed of a backbone and multiple regression heads with fully shared parameters to intelligently predict these quantitative metrics. Through auditing of the primary generalization result, the sensitive attribute was identified and the dataset was subdivided to train new submodels. Then, submodels trained from respective subsets were ensembled for final generalization. Results: The primary generalization result showed that the AI model underperformed in quantifying major basal bone indices. The sex factor was proved to be the sensitive attribute. The final model was ensembled by the male and female submodels, which yielded equal performance between genders, low error, high consistency, satisfying correlation coefficient, and highly focused attention. The ensemble model exhibited high similarity to clinicians with minor processing time. Conclusion: This work validates that the demographic parity strategy enables the AI algorithm with greater model generalization ability, even for the highly variable traits, which benefits for the appearance-concerning maxillofacial surgery.

RadShap: An Explanation Tool for Highlighting the Contributions of Multiple Regions of Interest to the Prediction of Radiomic Models

Explaining the decisions made by a radiomic model is of significant interest, as it can provide valuable insights into the information learned by complex models and foster trust in well-performing ones, thereby facilitating their clinical adoption. Promising radiomic approaches that aggregate information from multiple regions within an image currently lack suitable explanation tools that could identify the regions that most significantly influence their decisions. Here we present a model- and modality-agnostic tool (RadShap, https://github.com/ncaptier/radshap), based on Shapley values, that explains the predictions of multiregion radiomic models by highlighting the contribution of each individual region. Methods: The explanation tool leverages Shapley values to distribute the aggregative radiomic model's output among all the regions of interest of an image, highlighting their individual contribution. RadShap was validated using a retrospective cohort of 130 patients with advanced non-small cell lung cancer undergoing first-line immunotherapy. Their baseline PET scans were used to build 1,000 synthetic tasks to evaluate the degree of alignment between the tool's explanations and our data generation process. RadShap's potential was then illustrated through 2 real case studies by aggregating information from all segmented tumors: the prediction of the progression-free survival of the non-small cell lung cancer patients and the classification of the histologic tumor subtype. Results: RadShap demonstrated strong alignment with the ground truth, with a median frequency of 94% for consistently explained predictions in the synthetic tasks. In both real-case studies, the aggregative models yielded superior performance to the single-lesion models (average [±SD] time-dependent area under the receiver operating characteristic curve was 0.66 ± 0.02 for the aggregative survival model vs. 0.55 ± 0.04 for the primary tumor survival model). The tool's explanations provided relevant insights into the behavior of the aggregative models, highlighting that for the classification of the histologic subtype, the aggregative model used information beyond the biopsy site to correctly classify patients who were initially misclassified by a model focusing only on the biopsied tumor. Conclusion: RadShap aligned with ground truth explanations and provided valuable insights into radiomic models' behaviors. It is implemented as a user-friendly Python package with documentation and tutorials, facilitating its smooth integration into radiomic pipelines.

Absolute ground truth-based validation of computer-aided nodule detection and volumetry in low-dose CT imaging

PURPOSE

To validate the performance of computer-aided detection (CAD) and volumetry software using an anthropomorphic phantom with a ground truth (GT) set of 3D-printed nodules.

METHODS

The Kyoto Kaguku Lungman phantom, containing 3D-printed solid nodules including six diameters (4 to 9 mm) and three morphologies (smooth, lobulated, spiculated), was scanned at varying CTDIvol levels (6.04, 1.54 and 0.20 mGy). Combinations of reconstruction algorithms (iterative and deep learning image reconstruction) and kernels (soft and hard) were applied. Detection, volumetry and density results recorded by a commercially available AI-based algorithm (AVIEW LCS + ) were compared to the absolute GT, which was determined through µCT scanning at 50 µm resolution. The associations between image acquisition parameters or nodule characteristics and accuracy of nodule detection and characterization were analyzed with chi square tests and multiple linear regression.

RESULTS

High levels of detection sensitivity and precision (minimal 83 % and 91 % respectively) were observed across all acquisitions. Neither reconstruction algorithm nor radiation dose showed significant associations with detection. Nodule diameter however showed a highly significant association with detection (p < 0.0001). Volumetric measurements for nodules > 6 mm were accurate within 10 % absolute range from volumeGT, regardless of dose and reconstruction. Nodule diameter and morphology are major determinants of volumetric accuracy (p < 0.001). Density assignment was not significantly influenced by any parameters.

CONCLUSIONS

Our study confirms the software's accurate performance in nodule volumetry, detection and density characterization with robustness for variations in CT imaging protocols. This study suggests the incorporation of similar phantom setups in quality assurance of CAD tools.

The transition from genomics to phenomics in personalized population health

Modern health care faces several serious challenges, including an ageing population and its inherent burden of chronic diseases, rising costs and marginal quality metrics. By assessing and optimizing the health trajectory of each individual using a data-driven personalized approach that reflects their genetics, behaviour and environment, we can start to address these challenges. This assessment includes longitudinal phenome measures, such as the blood proteome and metabolome, gut microbiome composition and function, and lifestyle and behaviour through wearables and questionnaires. Here, we review ongoing large-scale genomics and longitudinal phenomics efforts and the powerful insights they provide into wellness. We describe our vision for the transformation of the current health care from disease-oriented to data-driven, wellness-oriented and personalized population health.

Measuring the binary thickness of buccal bone of anterior maxilla in low-resolution cone-beam computed tomography via a bilinear convolutional neural network

Background

The thickness of the buccal bone of the anterior maxilla is an important aesthetic-determining factor for dental implant, which is divided into the thick (≥1 mm) and thin type (<1 mm). However, as a micro-scale structure that is evaluated through low-resolution cone-beam computed tomography (CBCT), its thickness measurement is error-prone under the circumstance of enormous patients and relatively inexperienced primary dentists. Further, the challenges of deep learning-based analysis of the binary thickness of buccal bone include the substantial real-world variance caused by pixel error, the extraction of fine-grained features, and burdensome annotations.

Methods

This study built bilinear convolutional neural network (BCNN) with 2 convolutional neural network (CNN) backbones and a bilinear pooling module to predict the binary thickness of buccal bone (thick or thin) of the anterior maxilla in an end-to-end manner. The methods of 5-fold cross-validation and model ensemble were adopted at the training and testing stages. The visualization methods of Gradient Weighted Class Activation Mapping (Grad-CAM), Guided Grad-CAM, and layer-wise relevance propagation (LRP) were used for revealing the important features on which the model focused. The performance metrics and efficacy were compared between BCNN, dentists of different clinical experience (i.e., dental student, junior dentist, and senior dentist), and the fusion of BCNN and dentists to investigate the clinical feasibility of BCNN.

Results

Based on the dataset of 4,000 CBCT images from 1,000 patients (aged 36.15±13.09 years), the BCNN with visual geometry group (VGG)16 backbone achieved an accuracy of 0.870 [95% confidence interval (CI): 0.838-0.902] and an area under the receiver operating characteristic (ROC) curve (AUC) of 0.924 (95% CI: 0.896-0.948). Compared with the conventional CNNs, BCNN precisely located the buccal bone wall over irrelevant regions. The BCNN generally outperformed the expert-level dentists. The clinical diagnostic performance of the dentists was improved with the assistance of BCNN.

Conclusions

The application of BCNN to the quantitative analysis of binary buccal bone thickness validated the model's excellent ability of subtle feature extraction and achieved expert-level performance. This work signals the potential of fine-grained image recognition networks to the precise quantitative analysis of micro-scale structures.

Automated diagnosis of 7 canine skin tumors using machine learning on H&E-stained whole slide images

Microscopic evaluation of hematoxylin and eosin-stained slides is still the diagnostic gold standard for a variety of diseases, including neoplasms. Nevertheless, intra- and interrater variability are well documented among pathologists. So far, computer assistance via automated image analysis has shown potential to support pathologists in improving accuracy and reproducibility of quantitative tasks. In this proof of principle study, we describe a machine-learning-based algorithm for the automated diagnosis of 7 of the most common canine skin tumors: trichoblastoma, squamous cell carcinoma, peripheral nerve sheath tumor, melanoma, histiocytoma, mast cell tumor, and plasmacytoma. We selected, digitized, and annotated 350 hematoxylin and eosin-stained slides (50 per tumor type) to create a database divided into training, n = 245 whole-slide images (WSIs), validation (n = 35 WSIs), and test sets (n = 70 WSIs). Full annotations included the 7 tumor classes and 6 normal skin structures. The data set was used to train a convolutional neural network (CNN) for the automatic segmentation of tumor and nontumor classes. Subsequently, the detected tumor regions were classified patch-wise into 1 of the 7 tumor classes. A majority of patches-approach led to a tumor classification accuracy of the network on the slide-level of 95% (133/140 WSIs), with a patch-level precision of 85%. The same 140 WSIs were provided to 6 experienced pathologists for diagnosis, who achieved a similar slide-level accuracy of 98% (137/140 correct majority votes). Our results highlight the feasibility of artificial intelligence-based methods as a support tool in diagnostic oncologic pathology with future applications in other species and tumor types.

Predictive value of radiomics-based machine learning for the disease-free survival in breast cancer: a systematic review and meta-analysis

Purpose

This study summarized the previously-published studies regarding the use of radiomics-based predictive models for the identification of breast cancer-associated prognostic factors, which can help clinical decision-making and follow-up strategy.

Materials and methods

This study has been pre-registered on PROSPERO. PubMed, Embase, Cochrane Library, and Web of Science were searched, from inception to April 23, 2022, for studies that used radiomics for prognostic prediction of breast cancer patients. Then the search was updated on July 18, 2023. Quality assessment was conducted using the Radiomics Quality Score, and meta-analysis was performed using R software.

Results

A total of 975 articles were retrieved, and 13 studies were included, involving 5014 participants and 35 prognostic models. Among the models, 20 models were radiomics-based and the other 15 were based on clinical or pathological information. The primary outcome was Disease-free Survival (DFS). The retrieved studies were screened using LASSO, and Cox Regression was applied for modeling. The mean RQS was 18. The c-index of radiomics-based models for DFS prediction was 0.763 (95%CI 0.718-0.810) in the training set and 0.702 (95%CI 0.637-0.774) in the validation set. The c-index of combination models was 0.807 (95%CI0.736-0.885) in the training set and 0.840 (95%CI 0.794-0.888) in the validation set. There was no significant change in the c-index of DFS at 1, 2, 3, and over 5 years of follow-up.

Conclusion

This study has proved that radiomics-based prognostic models are of great predictive performance for the prognosis of breast cancer patients. combination model shows significantly enhanced predictive performance.

Systematic review registration

https://www.crd.york.ac.uk/PROSPERO/, identifier CRD42022332392.