1
|
Buch K. Editorial Comment: The Critical Need to Prospectively Evaluate the Utility of Artificial Intelligence Tools in Radiology-Do These Algorithms Always Help Us? AJR Am J Roentgenol 2024. [PMID: 39356486 DOI: 10.2214/ajr.24.32049] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/03/2024]
Affiliation(s)
- Karen Buch
- Dept of Radiology, Division of Neuroradiology, Massachusetts General Hospital, Boston, MA 02114
| |
Collapse
|
2
|
Forghani R. Real-World Adoption of Artificial Intelligence in Radiology: Opportunities and Barriers. AJNR Am J Neuroradiol 2024; 45:1175-1176. [PMID: 39122466 PMCID: PMC11392376 DOI: 10.3174/ajnr.a8422] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/12/2024]
|
3
|
Savage CH, Tanwar M, Elkassem AA, Sturdivant A, Hamki O, Sotoudeh H, Sirineni G, Singhal A, Milner D, Jones J, Rehder D, Li M, Li Y, Junck K, Tridandapani S, Rothenberg SA, Smith AD. Prospective Evaluation of Artificial Intelligence Triage of Intracranial Hemorrhage on Noncontrast Head CT Examinations. AJR Am J Roentgenol 2024. [PMID: 39230402 DOI: 10.2214/ajr.24.31639] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/05/2024]
Abstract
Background: Retrospective studies evaluating artificial intelligence (AI) algorithms for intracranial hemorrhage (ICH) detection on noncontrast CT (NCCT) have shown promising results but lack prospective validation. Objective: To evaluate the impact on radiologists' real-world aggregate performance for ICH detection and report turnaround times for ICH-positive examinations of a radiology department's implementation of an AI triage and notification system for ICH detection on head NCCT examinations. Methods: This prospective single-center study included adult patients who underwent head NCCT examinations from May 12, 2021 to June 30, 2021 (phase 1) or September 30, 2021 to December 4, 2021 (phase 2). Before phase 1, the radiology department implemented a commercial AI triage system for ICH detection that processed head NCCT examinations and notified radiologists of positive results through a widget with a floating pop-up display. Examinations were interpreted by neuroradiologists or emergency radiologists, who evaluated examinations without and with AI assistance in phase 1 and phase 2, respectively. A panel of radiologists conducted a review process for all examinations with discordance between the radiology report and AI and a subset of remaining examinations, to establish the reference standard. Diagnostic performance and report turnaround times were compared using Pearson chi-square test and Wilcoxon rank-sum test, respectively. Bonferroni correction was used to account for five diagnostic performance metrics (adjusted significance threshold, .01 [α=.05/5]). Results: A total of 9954 examinations from 7371 patients (mean age, 54.8±19.8 years; 3773 female, 3598 male) were included. In phases 1 and 2, 19.8% (735/3716) and 21.9% (1368/6238) of examinations, respectively, were positive for ICH (P=.01). Radiologists without versus with AI showed no significant difference in accuracy (99.5% vs 99.2%), sensitivity (98.6% vs 98.9%), PPV (99.0% vs 99.7%), or NPV (99.7% vs 99.7%) (all P>.01); specificity was higher for radiologists without than with AI (99.8% vs 99.3%, respectively, P=.004). Mean report turnaround time for ICH-positive examinations was 147.1 minutes without AI versus 149.9 minutes with AI (P=.11). Conclusion: An AI triage system for ICH detection did not improve radiologists' diagnostic performance or report turnaround times. Clinical Impact: This large prospective real-world study does not support use of AI assistance for ICH detection.
Collapse
Affiliation(s)
- Cody H Savage
- University of Maryland Medical Intelligent Imaging (UM2ii) Center, Department of Diagnostic Radiology & Nuclear Medicine, University of Maryland School of Medicine
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine
| | - Manoj Tanwar
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine
| | - Asser Abou Elkassem
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine
| | - Adam Sturdivant
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine
| | - Omar Hamki
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine
| | - Houman Sotoudeh
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine
| | - Gopi Sirineni
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine
| | - Aparna Singhal
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine
| | - Desmin Milner
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine
| | - Jesse Jones
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine
| | - Dirk Rehder
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine
| | - Mei Li
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine
| | - Yufeng Li
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine
| | - Kevin Junck
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine
| | - Srini Tridandapani
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine
| | - Steven A Rothenberg
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine
| | - Andrew D Smith
- Department of Diagnostic Imaging, St. Jude Children's Research Hospital
| |
Collapse
|
4
|
Rajagopal S, Bogaard HJ, Elbaz MSM, Freed BH, Remy-Jardin M, van Beek EJR, Gopalan D, Kiely DG. Emerging multimodality imaging techniques for the pulmonary circulation. Eur Respir J 2024:2401128. [PMID: 39209480 DOI: 10.1183/13993003.01128-2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2024] [Accepted: 06/11/2024] [Indexed: 09/04/2024]
Abstract
Pulmonary hypertension (PH) remains a challenging condition to diagnose, classify and treat. Current approaches to the assessment of PH include echocardiography, ventilation/perfusion scintigraphy, cross-sectional imaging using computed tomography and magnetic resonance imaging, and right heart catheterisation. However, these approaches only provide an indirect readout of the primary pathology of the disease: abnormal vascular remodelling in the pulmonary circulation. With the advent of newer imaging techniques, there is a shift toward increased utilisation of noninvasive high-resolution modalities that offer a more comprehensive cardiopulmonary assessment and improved visualisation of the different components of the pulmonary circulation. In this review, we explore advances in imaging of the pulmonary vasculature and their potential clinical translation. These include advances in diagnosis and assessing treatment response, as well as strategies that allow reduced radiation exposure and implementation of artificial intelligence technology. These emerging modalities hold the promise of developing a deeper understanding of pulmonary vascular disease and the impact of comorbidities. They also have the potential to improve patient outcomes by reducing time to diagnosis, refining classification, monitoring treatment response and improving our understanding of disease mechanisms.
Collapse
Affiliation(s)
| | - Harm J Bogaard
- Department of Pulmonology, Amsterdam University Medical Center, Location VU Medical Center, Amsterdam, The Netherlands
| | - Mohammed S M Elbaz
- Department of Radiology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Benjamin H Freed
- Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | | | - Edwin J R van Beek
- Edinburgh Imaging, Queens Medical Research Institute, University of Edinburgh, Edinburgh, UK
| | - Deepa Gopalan
- Department of Radiology, Imperial College Healthcare NHS Trust, London, UK
| | - David G Kiely
- Sheffield Pulmonary Vascular Disease Unit and NIHR Biomedical Research Centre Sheffield, Royal Hallamshire Hospital, Sheffield, UK
| |
Collapse
|
5
|
Forman HP. Large Language Models as an Inexpensive and Effective Extra Set of Eyes in Radiology Reporting. Radiology 2024; 311:e240844. [PMID: 38625009 DOI: 10.1148/radiol.240844] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/17/2024]
Affiliation(s)
- Howard P Forman
- From the Department of Radiology, Yale University School of Medicine, 333 Cedar St, New Haven, CT 06510; Yale School of Management, New Haven, Conn; Department of Economics, Yale College, New Haven, Conn; and Yale School of Public Health, New Haven, Conn
| |
Collapse
|
6
|
McNamara SL, Yi PH, Lotter W. The clinician-AI interface: intended use and explainability in FDA-cleared AI devices for medical image interpretation. NPJ Digit Med 2024; 7:80. [PMID: 38531952 DOI: 10.1038/s41746-024-01080-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2023] [Accepted: 03/14/2024] [Indexed: 03/28/2024] Open
Abstract
As applications of AI in medicine continue to expand, there is an increasing focus on integration into clinical practice. An underappreciated aspect of this clinical translation is where the AI fits into the clinical workflow, and in turn, the outputs generated by the AI to facilitate clinician interaction in this workflow. For instance, in the canonical use case of AI for medical image interpretation, the AI could prioritize cases before clinician review or even autonomously interpret the images without clinician review. A related aspect is explainability - does the AI generate outputs to help explain its predictions to clinicians? While many clinical AI workflows and explainability techniques have been proposed, a summative assessment of the current scope in clinical practice is lacking. Here, we evaluate the current state of FDA-cleared AI devices for medical image interpretation assistance in terms of intended clinical use, outputs generated, and types of explainability offered. We create a curated database focused on these aspects of the clinician-AI interface, where we find a high frequency of "triage" devices, notable variability in output characteristics across products, and often limited explainability of AI predictions. Altogether, we aim to increase transparency of the current landscape of the clinician-AI interface and highlight the need to rigorously assess which strategies ultimately lead to the best clinical outcomes.
Collapse
Affiliation(s)
| | - Paul H Yi
- University of Maryland Medical Intelligent Imaging (UM2ii) Center, Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, Baltimore, MD, USA
| | - William Lotter
- Harvard Medical School, Boston, MA, USA.
- Department of Data Science, Dana-Farber Cancer Institute, Boston, MA, USA.
- Department of Pathology, Brigham & Women's Hospital, Boston, MA, USA.
| |
Collapse
|
7
|
Vallée A, Quint R, Laure Brun A, Mellot F, Grenier PA. A deep learning-based algorithm improves radiology residents' diagnoses of acute pulmonary embolism on CT pulmonary angiograms. Eur J Radiol 2024; 171:111324. [PMID: 38241853 DOI: 10.1016/j.ejrad.2024.111324] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2023] [Revised: 12/08/2023] [Accepted: 01/15/2024] [Indexed: 01/21/2024]
Abstract
PURPOSE To compare radiology residents' diagnostic performances to detect pulmonary emboli (PEs) on CT pulmonary angiographies (CTPAs) with deep-learning (DL)-based algorithm support and without. METHODS Fully anonymized CTPAs (n = 207) of patients suspected of having acute PE served as input for PE detection using a previously trained and validated DL-based algorithm. Three residents in their first three years of training, blinded to the index report and clinical history, read the CTPAs first without, and 2 months later with the help of artificial intelligence (AI) output, to diagnose PE as present, absent or indeterminate. We evaluated concordances and discordances with the consensus-reading results of two experts in chest imaging. RESULTS Because the AI algorithm failed to analyze 11 CTPAs, 196 CTPAs were analyzed; 31 (15.8 %) were PE-positive. Good-classification performance was higher for residents with AI-algorithm support than without (AUROCs: 0.958 [95 % CI: 0.921-0.979] vs. 0.894 [95 % CI: 0.850-0.931], p < 0.001, respectively). The main finding was the increased sensitivity of residents' diagnoses using the AI algorithm (92.5 % vs. 81.7 %, respectively). Concordance between residents (kappa: 0.77 [95 % CI: 0.76-0.78]; p < 0.001) improved with AI-algorithm use (kappa: 0.88 [95 % CI: 0.87-0.89]; p < 0.001). CONCLUSION The AI algorithm we used improved between-resident agreements to interpret CTPAs for suspected PE and, hence, their diagnostic performances.
Collapse
Affiliation(s)
- Alexandre Vallée
- Department of Epidemiology and Public Health, Hôpital Foch. 40 rue Worth 92150 Suresnes, France.
| | - Raphaelle Quint
- Department of Medical Imaging, Hôpital Foch. 40 rue Worth 92150 Suresnes, France.
| | - Anne Laure Brun
- Department of Medical Imaging, Hôpital Foch. 40 rue Worth 92150 Suresnes, France.
| | - François Mellot
- Department of Medical Imaging, Hôpital Foch. 40 rue Worth 92150 Suresnes, France.
| | - Philippe A Grenier
- Department of Clinical Research and Innovation, Hôpital Foch. 40 rue Worth 92150 Suresnes, France.
| |
Collapse
|
8
|
Grenier PA, Brun AL, Mellot F. [The contribution of artificial intelligence (AI) subsequent to the processing of thoracic imaging]. Rev Mal Respir 2024; 41:110-126. [PMID: 38129269 DOI: 10.1016/j.rmr.2023.12.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Accepted: 11/27/2023] [Indexed: 12/23/2023]
Abstract
The contribution of artificial intelligence (AI) to medical imaging is currently the object of widespread experimentation. The development of deep learning (DL) methods, particularly convolution neural networks (CNNs), has led to performance gains often superior to those achieved by conventional methods such as machine learning. Radiomics is an approach aimed at extracting quantitative data not accessible to the human eye from images expressing a disease. The data subsequently feed machine learning models and produce diagnostic or prognostic probabilities. As for the multiple applications of AI methods in thoracic imaging, they are undergoing evaluation. Chest radiography is a practically ideal field for the development of DL algorithms able to automatically interpret X-rays. Current algorithms can detect up to 14 different abnormalities present either in isolation or in combination. Chest CT is another area offering numerous AI applications. Various algorithms have been specifically formed and validated for the detection and characterization of pulmonary nodules and pulmonary embolism, as well as segmentation and quantitative analysis of the extent of diffuse lung diseases (emphysema, infectious pneumonias, interstitial lung disease). In addition, the analysis of medical images can be associated with clinical, biological, and functional data (multi-omics analysis), the objective being to construct predictive approaches regarding disease prognosis and response to treatment.
Collapse
Affiliation(s)
- P A Grenier
- Délégation à la recherche clinique et l'innovation, hôpital Foch, Suresnes, France.
| | - A L Brun
- Service de radiologie, hôpital Foch, Suresnes, France
| | - F Mellot
- Service de radiologie, hôpital Foch, Suresnes, France
| |
Collapse
|
9
|
Forman HP, Bhalla S. Subsegmental Pulmonary Emboli and Chronic Pulmonary Emboli Should Not Be Ignored. Radiology 2024; 310:e232873. [PMID: 38411509 DOI: 10.1148/radiol.232873] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/28/2024]
Affiliation(s)
- Howard P Forman
- * Department of Radiology and Biomedical Imaging, Yale School of Medicine, 333 Cedar St, Tompkins East 2-204, New Haven, CT 06520
- Yale School of Public Health, New Haven, Conn
| | - Sanjeev Bhalla
- Department of Cardiothoracic Imaging, Mallinckrodt Institute of Radiology, Washington University School of Medicine in St. Louis, St Louis, Mo
| |
Collapse
|