1
|
Ross J, Hammouche S, Chen Y, Rockall AG. Beyond regulatory compliance: evaluating radiology artificial intelligence applications in deployment. Clin Radiol 2024; 79:338-345. [PMID: 38360516 DOI: 10.1016/j.crad.2024.01.026] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Revised: 01/24/2024] [Accepted: 01/29/2024] [Indexed: 02/17/2024]
Abstract
The implementation of artificial intelligence (AI) applications in routine practice, following regulatory approval, is currently limited by practical concerns around reliability, accountability, trust, safety, and governance, in addition to factors such as cost-effectiveness and institutional information technology support. When a technology is new and relatively untested in a field, professional confidence is lacking and there is a sense of the need to go above the baseline level of validation and compliance. In this article, we propose an approach that goes beyond standard regulatory compliance for AI apps that are approved for marketing, including independent benchmarking in the lab as well as clinical audit in practice, with the aims of increasing trust and preventing harm.
Collapse
Affiliation(s)
- J Ross
- Department of Cancer and Surgery, Imperial College London, UK.
| | - S Hammouche
- Department of Cancer and Surgery, Imperial College London, UK
| | - Y Chen
- School of Medicine, University of Nottingham, UK
| | - A G Rockall
- Department of Cancer and Surgery, Imperial College London, UK
| |
Collapse
|
2
|
Abdulaal L, Maiter A, Salehi M, Sharkey M, Alnasser T, Garg P, Rajaram S, Hill C, Johns C, Rothman AMK, Dwivedi K, Kiely DG, Alabed S, Swift AJ. A systematic review of artificial intelligence tools for chronic pulmonary embolism on CT pulmonary angiography. FRONTIERS IN RADIOLOGY 2024; 4:1335349. [PMID: 38654762 PMCID: PMC11035730 DOI: 10.3389/fradi.2024.1335349] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/08/2023] [Accepted: 03/26/2024] [Indexed: 04/26/2024]
Abstract
Background Chronic pulmonary embolism (PE) may result in pulmonary hypertension (CTEPH). Automated CT pulmonary angiography (CTPA) interpretation using artificial intelligence (AI) tools has the potential for improving diagnostic accuracy, reducing delays to diagnosis and yielding novel information of clinical value in CTEPH. This systematic review aimed to identify and appraise existing studies presenting AI tools for CTPA in the context of chronic PE and CTEPH. Methods MEDLINE and EMBASE databases were searched on 11 September 2023. Journal publications presenting AI tools for CTPA in patients with chronic PE or CTEPH were eligible for inclusion. Information about model design, training and testing was extracted. Study quality was assessed using compliance with the Checklist for Artificial Intelligence in Medical Imaging (CLAIM). Results Five studies were eligible for inclusion, all of which presented deep learning AI models to evaluate PE. First study evaluated the lung parenchymal changes in chronic PE and two studies used an AI model to classify PE, with none directly assessing the pulmonary arteries. In addition, a separate study developed a CNN tool to distinguish chronic PE using 2D maximum intensity projection reconstructions. While another study assessed a novel automated approach to quantify hypoperfusion to help in the severity assessment of CTEPH. While descriptions of model design and training were reliable, descriptions of the datasets used in training and testing were more inconsistent. Conclusion In contrast to AI tools for evaluation of acute PE, there has been limited investigation of AI-based approaches to characterising chronic PE and CTEPH on CTPA. Existing studies are limited by inconsistent reporting of the data used to train and test their models. This systematic review highlights an area of potential expansion for the field of AI in medical image interpretation.There is limited knowledge of A systematic review of artificial intelligence tools for chronic pulmonary embolism in CT. This systematic review provides an assessment on research that examined deep learning algorithms in detecting CTEPH on CTPA images, the number of studies assessing the utility of deep learning on CTPA in CTEPH was unclear and should be highlighted.
Collapse
Affiliation(s)
- Lojain Abdulaal
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
- Faculty of Applied Medical Science, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Ahmed Maiter
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
- Respiratory Physiology Department, Sheffield Pulmonary Vascular Disease Unit, Sheffield, United Kingdom
- Department of Clinical Radiology, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, United Kingdom
| | - Mahan Salehi
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
| | - Michael Sharkey
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
| | - Turki Alnasser
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
| | - Pankaj Garg
- Faculty of Medicine and Health Sciences, Norwich Medical School, University of East Anglia, Norwich, United Kingdom
| | - Smitha Rajaram
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
- Respiratory Physiology Department, Sheffield Pulmonary Vascular Disease Unit, Sheffield, United Kingdom
- Department of Clinical Radiology, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, United Kingdom
| | - Catherine Hill
- Respiratory Physiology Department, Sheffield Pulmonary Vascular Disease Unit, Sheffield, United Kingdom
- Department of Clinical Radiology, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, United Kingdom
| | - Christopher Johns
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
- Respiratory Physiology Department, Sheffield Pulmonary Vascular Disease Unit, Sheffield, United Kingdom
- Department of Clinical Radiology, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, United Kingdom
| | - Alex Matthew Knox Rothman
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
| | - Krit Dwivedi
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
- Respiratory Physiology Department, Sheffield Pulmonary Vascular Disease Unit, Sheffield, United Kingdom
| | - David G. Kiely
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
- Faculty of Engineering, INSIGNEO Institute, Institute for in Silico Medicine, The University of Sheffield, Sheffield, United Kingdom
- Sheffield Biomedical Research Centre, National Institute for Health Research, Sheffield, United Kingdom
| | - Samer Alabed
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
- Department of Clinical Radiology, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, United Kingdom
- Faculty of Engineering, INSIGNEO Institute, Institute for in Silico Medicine, The University of Sheffield, Sheffield, United Kingdom
| | - Andrew James Swift
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
- Department of Clinical Radiology, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, United Kingdom
- Faculty of Engineering, INSIGNEO Institute, Institute for in Silico Medicine, The University of Sheffield, Sheffield, United Kingdom
- Sheffield Biomedical Research Centre, National Institute for Health Research, Sheffield, United Kingdom
| |
Collapse
|
3
|
Schilling M, Unterberg-Buchwald C, Lotz J, Uecker M. Assessment of deep learning segmentation for real-time free-breathing cardiac magnetic resonance imaging at rest and under exercise stress. Sci Rep 2024; 14:3754. [PMID: 38355969 PMCID: PMC10866998 DOI: 10.1038/s41598-024-54164-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2023] [Accepted: 02/09/2024] [Indexed: 02/16/2024] Open
Abstract
In recent years, a variety of deep learning networks for cardiac MRI (CMR) segmentation have been developed and analyzed. However, nearly all of them are focused on cine CMR under breathold. In this work, accuracy of deep learning methods is assessed for volumetric analysis (via segmentation) of the left ventricle in real-time free-breathing CMR at rest and under exercise stress. Data from healthy volunteers (n = 15) for cine and real-time free-breathing CMR at rest and under exercise stress were analyzed retrospectively. Exercise stress was performed using an ergometer in the supine position. Segmentations of two deep learning methods, a commercially available technique (comDL) and an openly available network (nnU-Net), were compared to a reference model created via the manual correction of segmentations obtained with comDL. Segmentations of left ventricular endocardium (LV), left ventricular myocardium (MYO), and right ventricle (RV) are compared for both end-systolic and end-diastolic phases and analyzed with Dice's coefficient. The volumetric analysis includes the cardiac function parameters LV end-diastolic volume (EDV), LV end-systolic volume (ESV), and LV ejection fraction (EF), evaluated with respect to both absolute and relative differences. For cine CMR, nnU-Net and comDL achieve Dice's coefficients above 0.95 for LV and 0.9 for MYO, and RV. For real-time CMR, the accuracy of nnU-Net exceeds that of comDL overall. For real-time CMR at rest, nnU-Net achieves Dice's coefficients of 0.94 for LV, 0.89 for MYO, and 0.90 for RV and the mean absolute differences between nnU-Net and the reference are 2.9 mL for EDV, 3.5 mL for ESV, and 2.6% for EF. For real-time CMR under exercise stress, nnU-Net achieves Dice's coefficients of 0.92 for LV, 0.85 for MYO, and 0.83 for RV and the mean absolute differences between nnU-Net and reference are 11.4 mL for EDV, 2.9 mL for ESV, and 3.6% for EF. Deep learning methods designed or trained for cine CMR segmentation can perform well on real-time CMR. For real-time free-breathing CMR at rest, the performance of deep learning methods is comparable to inter-observer variability in cine CMR and is usable for fully automatic segmentation. For real-time CMR under exercise stress, the performance of nnU-Net could promise a higher degree of automation in the future.
Collapse
Affiliation(s)
- Martin Schilling
- Institute for Diagnostic and Interventional Radiology, Universitätsmedizin Göttingen, Göttingen, Germany
| | - Christina Unterberg-Buchwald
- Institute for Diagnostic and Interventional Radiology, Universitätsmedizin Göttingen, Göttingen, Germany
- German Centre for Cardiovascular Research (DZHK), Partner Site Göttingen, Göttingen, Germany
- Clinic of Cardiology and Pneumology, Universitätsmedizin Göttingen, Göttingen, Germany
| | - Joachim Lotz
- Institute for Diagnostic and Interventional Radiology, Universitätsmedizin Göttingen, Göttingen, Germany
| | - Martin Uecker
- Institute for Diagnostic and Interventional Radiology, Universitätsmedizin Göttingen, Göttingen, Germany.
- German Centre for Cardiovascular Research (DZHK), Partner Site Göttingen, Göttingen, Germany.
- Institute of Biomedical Imaging, Graz University of Technology, Graz, Austria.
| |
Collapse
|
4
|
Alnasser TN, Abdulaal L, Maiter A, Sharkey M, Dwivedi K, Salehi M, Garg P, Swift AJ, Alabed S. Advancements in cardiac structures segmentation: a comprehensive systematic review of deep learning in CT imaging. Front Cardiovasc Med 2024; 11:1323461. [PMID: 38317865 PMCID: PMC10839106 DOI: 10.3389/fcvm.2024.1323461] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Accepted: 01/08/2024] [Indexed: 02/07/2024] Open
Abstract
Background Segmentation of cardiac structures is an important step in evaluation of the heart on imaging. There has been growing interest in how artificial intelligence (AI) methods-particularly deep learning (DL)-can be used to automate this process. Existing AI approaches to cardiac segmentation have mostly focused on cardiac MRI. This systematic review aimed to appraise the performance and quality of supervised DL tools for the segmentation of cardiac structures on CT. Methods Embase and Medline databases were searched to identify related studies from January 1, 2013 to December 4, 2023. Original research studies published in peer-reviewed journals after January 1, 2013 were eligible for inclusion if they presented supervised DL-based tools for the segmentation of cardiac structures and non-coronary great vessels on CT. The data extracted from eligible studies included information about cardiac structure(s) being segmented, study location, DL architectures and reported performance metrics such as the Dice similarity coefficient (DSC). The quality of the included studies was assessed using the Checklist for Artificial Intelligence in Medical Imaging (CLAIM). Results 18 studies published after 2020 were included. The DSC scores median achieved for the most commonly segmented structures were left atrium (0.88, IQR 0.83-0.91), left ventricle (0.91, IQR 0.89-0.94), left ventricle myocardium (0.83, IQR 0.82-0.92), right atrium (0.88, IQR 0.83-0.90), right ventricle (0.91, IQR 0.85-0.92), and pulmonary artery (0.92, IQR 0.87-0.93). Compliance of studies with CLAIM was variable. In particular, only 58% of studies showed compliance with dataset description criteria and most of the studies did not test or validate their models on external data (81%). Conclusion Supervised DL has been applied to the segmentation of various cardiac structures on CT. Most showed similar performance as measured by DSC values. Existing studies have been limited by the size and nature of the training datasets, inconsistent descriptions of ground truth annotations and lack of testing in external data or clinical settings. Systematic Review Registration [www.crd.york.ac.uk/prospero/], PROSPERO [CRD42023431113].
Collapse
Affiliation(s)
- Turki Nasser Alnasser
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
- College of Applied Medical Science, King Saud bin Abdulaziz University for Health Science, Riyadh, Saudi Arabia
| | - Lojain Abdulaal
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
| | - Ahmed Maiter
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
- Department of Clinical Radiology, Sheffield Teaching Hospitals, Sheffield, United Kingdom
| | - Michael Sharkey
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
| | - Krit Dwivedi
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
- Department of Clinical Radiology, Sheffield Teaching Hospitals, Sheffield, United Kingdom
| | - Mahan Salehi
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
| | - Pankaj Garg
- Norwich Medical School, Faculty of Medicine and Health Sciences, University of East Anglia, Norwich, United Kingdom
| | - Andrew James Swift
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
- Insigneo Institute, Faculty of Engineering, The University of Sheffield, Sheffield, United Kingdom
| | - Samer Alabed
- Department of Infection, Immunity and Cardiovascular Disease, The University of Sheffield, Sheffield, United Kingdom
- Department of Clinical Radiology, Sheffield Teaching Hospitals, Sheffield, United Kingdom
- Insigneo Institute, Faculty of Engineering, The University of Sheffield, Sheffield, United Kingdom
| |
Collapse
|