Expert-enhanced machine learning for cardiac arrhythmia classification

A review of evaluation approaches for explainable AI with applications in cardiology

Explainable artificial intelligence (XAI) elucidates the decision-making process of complex AI models and is important in building trust in model predictions. XAI explanations themselves require evaluation as to accuracy and reasonableness and in the context of use of the underlying AI model. This review details the evaluation of XAI in cardiac AI applications and has found that, of the studies examined, 37% evaluated XAI quality using literature results, 11% used clinicians as domain-experts, 11% used proxies or statistical analysis, with the remaining 43% not assessing the XAI used at all. We aim to inspire additional studies within healthcare, urging researchers not only to apply XAI methods but to systematically assess the resulting explanations, as a step towards developing trustworthy and safe models.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10462-024-10852-w.

The influence of atrial flutter in automated detection of atrial arrhythmias - are we ready to go into clinical practice?"

OBJECTIVE

To investigate the impact of atrial flutter (A_fl) in the atrial arrhythmias classification task. We additionally advocate the use of a subject-based split for future studies in the field in order to avoid within-subject correlation which may lead to over-optimistic inferences. Finally, we demonstrate the effectiveness of the classifiers outside of the initially studied circumstances, by performing an inter-dataset model evaluation of the classifiers in data from different sources.

METHODS

ECG signals of two private and three public (two MIT-BIH and Chapman ecgdb) databases were preprocessed and divided into 10s segments which were then subject to feature extraction. The created datasets were divided into a training and test set in two ways, based on a random split and a patient split. Classification was performed using the XGBoost classifier, as well as two benchmark classification models using both data splits. The trained models were then used to make predictions on the test data of the remaining datasets.

RESULTS

The XGBoost model yielded the best performance across all datasets compared to the remaining benchmark models, however variability in model performance was seen across datasets, with accuracy ranging from 70.6% to 89.4%, sensitivity ranging from 61.4% to 76.8%, and specificity ranging from 87.3% to 95.5%. When comparing the results between the patient and the random split, no significant difference was seen in the two private datasets and the Chapman dataset, where the number of samples per patient is low. Nonetheless, in the MIT-BIH dataset, where the average number of samples per patient is approximately 1300, a noticeable disparity was identified. The accuracy, sensitivity, and specificity of the random split in this dataset of 93.6%, 86.4%, and 95.9% respectively, were decreased to 88%, 61.4%, and 89.8% in the patient split, with the largest drop being in A_fl sensitivity, from 71% to 5.4%. The inter-dataset scores were also significantly lower than their intra-dataset counterparts across all datasets.

CONCLUSIONS

CAD systems have great potential in the assistance of physicians in reliable, precise and efficient detection of arrhythmias. However, although compelling research has been done in the field, yielding models with excellent performances on their datasets, we show that these results may be over-optimistic. In our study, we give insight into the difficulty of detection of A_fl on several datasets and show the need for a higher representation of A_fl in public datasets. Furthermore, we show the necessity of a more structured evaluation of model performance through the use of a patient-based split and inter-dataset testing scheme to avoid the problem of within-subject correlation which may lead to misleadingly high scores. Finally, we stress the need for the creation and use of datasets with a higher number of patients and a more balanced representation of classes if we are to progress in this mission.

Machine learning in the detection and management of atrial fibrillation

Machine learning has immense novel but also disruptive potential for medicine. Numerous applications have already been suggested and evaluated concerning cardiovascular diseases. One important aspect is the detection and management of potentially thrombogenic arrhythmias such as atrial fibrillation. While atrial fibrillation is the most common arrhythmia with a lifetime risk of one in three persons and an increased risk of thromboembolic complications such as stroke, many atrial fibrillation episodes are asymptomatic and a first diagnosis is oftentimes only reached after an embolic event. Therefore, screening for atrial fibrillation represents an important part of clinical practice. Novel technologies such as machine learning have the potential to substantially improve patient care and clinical outcomes. Additionally, machine learning applications may aid cardiologists in the management of patients with already diagnosed atrial fibrillation, for example, by identifying patients at a high risk of recurrence after catheter ablation. We summarize the current state of evidence concerning machine learning and, in particular, artificial neural networks in the detection and management of atrial fibrillation and describe possible future areas of development as well as pitfalls.